huggingface/alignment-handbook

Implements a full post-training pipeline spanning continued pretraining, supervised fine-tuning, and preference alignment techniques including DPO, ORPO, and Constitutional AI. Training scripts support distributed training via DeepSpeed ZeRO-3 and parameter-efficient approaches (LoRA/QLoRA), with reproducible YAML-based recipes for models like Zephyr and SmolLM. Integrates with Hugging Face Hub for dataset and model management, supporting both human feedback and AI preference signals.

5,523 stars and 151 monthly downloads. No commits in the last 6 months. Available on PyPI.