EvolvingLMMs-Lab/lmms-engine

Supports 20+ model architectures including vision-language (Qwen3-VL, LLaVA), diffusion (WanVideo, SiT), and MoE variants with unified training abstractions. Built on PyTorch FSDP2 with composition-friendly distributed training, featuring production optimizations like Ulysses sequence parallelism for long contexts, Liger fused kernels (30% memory reduction), and bin-packing for 35-40% MFU gains. Integrates with torchrun/Accelerate launchers and provides Docker environments for rapid deployment across single-GPU to multi-node distributed setups.

740 stars. Actively maintained with 7 commits in the last 30 days.