LLMs-from-scratch and Building-LLMs-from-scratch
These are competitors offering similar educational implementations of GPT-style language models in PyTorch, where the significantly more established rasbt repository would be the primary choice for learning transformer architecture from scratch.
About LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Covers the complete pipeline from tokenization and attention mechanisms through pretraining on unlabeled data and finetuning for classification and instruction-following tasks. Includes practical implementations of multi-head attention, causal masking, and parameter-efficient techniques like LoRA, alongside code for loading pretrained model weights. Organized as Jupyter notebooks and standalone Python scripts that progressively build a functional GPT architecture while explaining each component's role in modern LLM training.
About Building-LLMs-from-scratch
codewithdark-git/Building-LLMs-from-scratch
This repository guides you through the process of building a GPT-style Large Language Model (LLM) from scratch using PyTorch. The structure and approach are inspired by the book Build a Large Language Model (From Scratch) by Sebastian Raschka.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work