datawhalechina/tiny-universe

Implements core LLM components from first principles using PyTorch—including Tiny Diffusion for image generation, Tiny Llama3 for pretraining, and Tiny Transformer architecture—alongside practical systems for RAG, Agent orchestration, and evaluation. Focuses on interpretable, minimal implementations with detailed code comments that decouple learning from high-level frameworks like Hugging Face, enabling independent system modification. Covers the complete pipeline from tokenizer training through inference, GraphRAG construction, and domain-specific evaluation metrics.

4,598 stars.