vllm-project/vllm-omni

Extends vLLM's text-focused architecture to handle any-to-any multimodal inference—text, image, video, and audio—with support for non-autoregressive models like Diffusion Transformers alongside traditional autoregressive generation. Uses fully disaggregated pipeline execution with OmniConnector and dynamic resource allocation across heterogeneous stages, enabling pipelined overlapping for high throughput. Provides tensor/pipeline/data/expert parallelism, OpenAI-compatible serving APIs, and integrates seamlessly with Hugging Face models including Qwen-Omni and multimodal generation variants.

3,197 stars and 16,988 monthly downloads. Actively maintained with 353 commits in the last 30 days. Available on PyPI.