Rohit2sali/vllm-multi-tenant-llm-gateway
This is vllm multi tenant large language model gateway. This system is created to serve lot of requests at same time to lot of users. It uses vllm as it's engine to run llm, it has scheduler to schedule the queries of users and limiter to limit the use of specific user. It also uses LoRA adapters in vllm.
Stars
—
Forks
—
Language
Jupyter Notebook
License
—
Category
Last pushed
Mar 05, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/Rohit2sali/vllm-multi-tenant-llm-gateway"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AlexsJones/llmfit
Hundreds of models & providers. One command to find what runs on your hardware.
livehl/aimirror
🚀 200倍速！AI时代的下载神器 | Docker/PyPI/HuggingFace/CRAN 全加速 | 并行分片+智能缓存，让下载飞起来
Chen-zexi/vllm-cli
A command-line interface tool for serving LLM using vLLM.
victordibia/llmx
An API for Chat Fine-Tuned Large Language Models (llm)
InftyAI/llmaz
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!