SafeAILab/EAGLE

Employs speculative decoding via a lightweight draft model that extrapolates multi-level semantic features (low, mid, and high-layer) from the base LLM, enabling 5.6x speedup on 13B models while maintaining output distribution equivalence. Progressively evolved across three versions—EAGLE-1 uses second-layer feature extrapolation, EAGLE-2 adds confidence-based dynamic tree adjustment, and EAGLE-3 removes feature prediction constraints through training-time simulation. Integrates with major LLM serving frameworks including vLLM, TensorRT-LLM, SGLang, and AMD ROCm, with trainable checkpoints available for Llama, Vicuna, Qwen, and DeepSeek models.

2,213 stars.