kvcache-ai/Mooncake

Implements a disaggregated KVCache architecture that decouples prefill and decode stages via efficient cross-device/cross-machine transfer, with components like the Transfer Engine (RDMA-optimized KV cache movement) and Mooncake Store (hierarchical distributed cache pool). Integrates with vLLM, SGLang, TensorRT-LLM, and LMDeploy as a backend connector for multi-node inference pipelines, enabling zero-copy embeddings sharing and dynamic KV cache offloading across GPU, host, and remote storage tiers.

4,911 stars. Actively maintained with 119 commits in the last 30 days.