MNN and xllm
These are competitors—both are inference engines optimizing LLM execution on hardware accelerators, targeting the same use case of efficient on-device model deployment, though MNN has achieved significantly wider adoption and maturity.
About MNN
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
Supports inference and training across multiple frameworks (TensorFlow, Caffe, ONNX, TorchScript) with specialized runtimes for LLMs via MNN-LLM and diffusion models via MNN-Diffusion. Employs aggressive optimization strategies including FP16/Int8 quantization (50-70% size reduction), minimal dependencies, and platform-specific backends to achieve sub-2MB executable overhead on iOS and 800KB core library on Android. Integrates with MNN Workbench for model visualization and one-click deployment across mobile, embedded, and IoT devices.
About xllm
jd-opensource/xllm
A high-performance inference engine for LLMs, optimized for diverse AI accelerators.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work