litgpt and llm-gpt
One project offers a comprehensive framework for training and deploying high-performance LLMs at scale, while the other provides a step-by-step, hands-on guide to building foundational NLP algorithms and early language models, making them complementary for different learning and implementation goals.
About litgpt
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Implements models from scratch without abstraction layers and combines Flash Attention with FSDP for distributed training across 1-1000+ GPUs/TPUs. Supports parameter-efficient finetuning via LoRA/QLoRA with mixed-precision quantization (fp4/8/16/32) to reduce GPU memory requirements, while integrating with PyTorch Lightning and Lightning Cloud infrastructure for end-to-end pretraining, finetuning, and deployment workflows through declarative YAML recipes.
About llm-gpt
huangjia2019/llm-gpt
From classic NLP to modern LLMs: building language models step by step. 异步图书：《 GPT图解 大模型是怎样构建的》- 这套代码是AI Coder出现之前，自己用纯手工搭建的一套简单有效的NLP经典算法集合。在大语言模型推动的AI Coder兴起之后，很少有机会再创作这么有“手工风”的代码了，不知道这是值得开心还是值得遗憾的事情。
Implements foundational NLP algorithms and transformer architecture components from scratch, including tokenization, embeddings, attention mechanisms, and decoding strategies, designed as hands-coded educational implementations rather than production frameworks. Structured as a companion to the "GPT图解" textbook and video course, progressing through classical NLP techniques toward modern large language model construction with emphasis on understanding core principles through direct implementation.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work