rustformers/llm

Built on the GGML tensor library, this project provided modular Rust crates for CPU-based LLM inference supporting models like LLaMA, GPT-J, and BLOOM with both library and CLI interfaces. The architecture separated concerns across `llm-base`, model-specific crates, and a `ggml-sys` FFI layer, enabling quantization and perplexity computation alongside interactive chat modes. It integrated with Hugging Face tokenizers by default and supported GGML model formats, though later branches explored GGUF compatibility.

6,152 stars. No commits in the last 6 months.