mosaicml/llm-foundry

Implements end-to-end training, finetuning, evaluation, and inference pipelines with built-in support for efficiency techniques like Flash Attention and Mixture-of-Experts architectures. Integrates with Composer for distributed training optimization and MosaicML's platform for scalable workload orchestration, while supporting both HuggingFace and proprietary models (MPT, DBRX) from 125M to 132B parameters. Includes data preparation utilities for StreamingDataset format, inference export to ONNX/HuggingFace, and in-context learning evaluation on academic benchmarks.

4,397 stars and 4,165 monthly downloads. Available on PyPI.