thu-pacman/chitu

Supports heterogeneous CPU+GPU inference and on-the-fly quantization conversion (FP4→FP8/BF16), enabling single-card deployment of 671B models. Provides production-grade stability across diverse hardware platforms—NVIDIA, Ascend, Moxin, and Hygon—with scalability from pure CPU setups to large distributed clusters. Optimized for models like DeepSeek, Qwen, and GLM with integrated batching and long-context efficiency techniques.

3,418 stars and 13 monthly downloads. Actively maintained with 128 commits in the last 30 days. Available on PyPI.