LLMs-from-scratch and llm-scratch-pytorch
These are competitors offering similar educational approaches to implementing transformers from scratch in PyTorch, with the former being a far more established and comprehensive resource (87k+ stars vs 100 stars) that would likely supersede the latter for most learners.
About LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Covers the complete pipeline from tokenization and attention mechanisms through pretraining on unlabeled data and finetuning for classification and instruction-following tasks. Includes practical implementations of multi-head attention, causal masking, and parameter-efficient techniques like LoRA, alongside code for loading pretrained model weights. Organized as Jupyter notebooks and standalone Python scripts that progressively build a functional GPT architecture while explaining each component's role in modern LLM training.
About llm-scratch-pytorch
skyloevil/llm-scratch-pytorch
lm-scratch-pytorch - The code is designed to be beginner-friendly, with a focus on understanding the fundamentals of PyTorch and implementing LLMs from scratch,step by step.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work