Mattbusel/llm-cpp
The C++ LLM toolkit. 26 single-header libraries for streaming, caching, cost estimation, retry, and structured output. Drop in what you need.
This is a collection of C++ libraries designed for developers who are building applications that use large language models (LLMs). It provides tools for various LLM-related tasks, like managing conversational turns, implementing robust retries, or estimating token costs. Developers can pick and choose specific functionalities to integrate LLM capabilities directly into their C++ applications without external dependencies like Python.
Use this if you are a C++ developer building a native application and need fine-grained control over LLM interactions, streaming, caching, or structured output.
Not ideal if you are not a C++ developer or prefer using high-level SDKs and package managers for LLM integration.
Stars
8
Forks
1
Language
—
License
—
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/Mattbusel/llm-cpp"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Blaizzy/mlx-vlm
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac...
b4rtaz/distributed-llama
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM...
armbues/SiLLM
SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple...
microsoft/batch-inference
Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
armbues/SiLLM-examples
Examples for using the SiLLM framework for training and running Large Language Models (LLMs) on...