mlc-ai/mlc-llm
Universal LLM Deployment Engine with ML Compilation
Compiles LLMs to optimized machine code via TVM's ML compilation framework, then executes them through MLCEngine—a unified inference runtime supporting diverse backends (CUDA, ROCm, Metal, WebGPU, OpenCL) across GPUs, mobile devices, and browsers. Exposes OpenAI-compatible REST and language-specific APIs (Python, JavaScript, iOS, Android) from the same compiled engine, enabling model-agnostic deployment without framework lock-in.
22,185 stars. Actively maintained with 15 commits in the last 30 days.
Stars
22,185
Forks
1,960
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 09, 2026
Commits (30d)
15
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/mlc-ai/mlc-llm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Related models
PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
skyzh/tiny-llm
A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny...
ServerlessLLM/ServerlessLLM
Serverless LLM Serving for Everyone.
AXERA-TECH/ax-llm
Explore LLM model deployment based on AXera's AI chips
VectorInstitute/vector-inference
Efficient LLM inference on Slurm clusters.