mistralai/mistral-inference

Provides efficient multi-GPU distributed inference via `torchrun` for large models like Mixtral 8x7B/8x22B, with built-in support for function calling across all models. Leverages `xformers` for optimized transformer operations and exposes both Python and CLI interfaces (`mistral-demo`, `mistral-chat`) for interactive testing and deployment. Supports diverse model families including specialized variants (Codestral for code, Mathstral for math, Pixtral for vision) alongside standard base and instruction-tuned versions.

10,705 stars.