NetEase-Media/grps_trtllm

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

42
/ 100
Emerging

158 stars.

No Package No Dependents
Maintenance 6 / 25
Adoption 10 / 25
Maturity 16 / 25
Community 10 / 25

How are scores calculated?

Stars

158

Forks

11

Language

Python

License

Apache-2.0

Category

llm-framework-abstractions

Last pushed

Dec 08, 2025

Commits (30d)

0

GitHub
Llm Framework Abstractions · 11 models

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/transformers/NetEase-Media/grps_trtllm"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

hassancs91/SimplerLLM

Simplify interactions with Large Language Models

68

avilum/minrlm

Token-efficient Recursive Language Model. 3.6x fewer tokens than vanilla LLMs. Data never enters...

55

kyegomez/SingLoRA

This repository provides a minimal, single-file implementation of SingLoRA (Single Matrix...

52

tylerelyt/LLM-Workshop

🌟 Learn Large Language Model development through hands-on projects and real-world implementations

51

gtausa197-svg/-Project-Nord-Spiking-Neural-Network-Language-Model

The first pure SNN language model trained from scratch with a fully original architecture. 144M...

39

Explore Transformer Models

All categories Trending Transformer directory Insights