LLMs-from-scratch and create-million-parameter-llm-from-scratch
The first is a comprehensive educational guide covering the full LLM architecture and training pipeline, while the second is a focused implementation of a specific model variant (LLaMA 1 with 2.3M parameters), making them complements that serve different depths of learning rather than competitors.
About LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Covers the complete pipeline from tokenization and attention mechanisms through pretraining on unlabeled data and finetuning for classification and instruction-following tasks. Includes practical implementations of multi-head attention, causal masking, and parameter-efficient techniques like LoRA, alongside code for loading pretrained model weights. Organized as Jupyter notebooks and standalone Python scripts that progressively build a functional GPT architecture while explaining each component's role in modern LLM training.
About create-million-parameter-llm-from-scratch
FareedKhan-dev/create-million-parameter-llm-from-scratch
Building a 2.3M-parameter LLM from scratch with LLaMA 1 architecture.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work