vllm-project/vllm

Implements PagedAttention for efficient KV cache management and continuous request batching to maximize GPU utilization. Supports multiple quantization schemes (GPTQ, AWQ, INT4/8, FP8), speculative decoding, and tensor/pipeline parallelism across NVIDIA, AMD, Intel, and TPU hardware. Provides OpenAI-compatible API endpoints and integrates directly with Hugging Face models, including multi-modal and mixture-of-expert architectures.

73,007 stars and 7,953,905 monthly downloads. Used by 43 other packages. Actively maintained with 996 commits in the last 30 days. Available on PyPI.