Tebmer/Awesome-Knowledge-Distillation-of-LLMs

Organized taxonomically across Knowledge Elicitation (labeling, expansion, curation, feedback, self-knowledge) and Distillation Algorithms (SFT, divergence-based, RL, rank optimization), the collection maps how to extract and transfer both general capabilities and domain-specific skills. Covers practical applications across skill dimensions (instruction-following, alignment, agent behavior, task specialization, multimodality) and vertical domains (legal, medical, finance, science), alongside encoder-based KD approaches. Integrates with open-source LLM ecosystems like LLaMA and Mistral, addressing capability transfer from proprietary models (GPT-4, Claude) through synthetic data generation and self-improvement techniques.

1,264 stars. No commits in the last 6 months.