Tencent/PatrickStar

Implements dynamic chunk-based memory scheduling to enable heterogeneous CPU-GPU training, allowing larger models (18B+ parameters) on fewer GPUs by intelligently offloading non-active model components to CPU memory. Integrates with PyTorch via a drop-in `initialize_engine()` API and supports multi-node distributed training with collective communication optimizations. Demonstrates 6x larger model capacity than DeepSpeed on equivalent hardware while maintaining faster training throughput.

779 stars.