ashankgupta/grpc_llm_template

A production-ready template for serving Large Language Models via gRPC with streaming token generation. Built with Python, PyTorch, Hugging Face Transformers, and gRPC. Supports any causal language model from HuggingFace with configurable sampling parameters (temperature, top_p, top_k).

14
/ 100
Experimental
No License No Package No Dependents
Maintenance 13 / 25
Adoption 0 / 25
Maturity 1 / 25
Community 0 / 25

How are scores calculated?

Stars

—

Forks

—

Language

Python

License

—

Last pushed

Apr 06, 2026

Commits (30d)

0

GitHub

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/ashankgupta/grpc_llm_template"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

campfirein/byterover-cli

ByteRover CLI (brv) - The portable memory layer for autonomous coding agents (formerly Cipher)

88

mistralai/client-python

Python client library for Mistral AI platform

88

openai/openai-python

The official Python library for the OpenAI API

86

pydantic/pydantic

Data validation using Python type hints

83

milla-jovovich/mempalace

The highest-scoring AI memory system ever benchmarked. And it's free.

82

Explore LLM Tools

All categories Trending LLM Tool directory Insights