LLM Implementation Tutorials LLM Tools

Educational repositories and step-by-step guides for building LLMs from scratch using frameworks like PyTorch and Transformers. Includes basic implementations, minimal examples, and learning-focused projects. Does NOT include production LLM frameworks, pre-trained model fine-tuning, or inference optimization tools.

There are 102 llm implementation tutorials tools tracked. 1 score above 70 (verified tier). The highest-rated is Lightning-AI/litgpt at 81/100 with 13,225 stars and 15,196 monthly downloads. 1 of the top 10 are actively maintained.

Get all 102 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-tools&subcategory=llm-implementation-tutorials&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	Lightning-AI/litgpt 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.	81	Verified	13,225	Python
2	SPUTNIKAI/LeechTransformer Leech-Lila: A Geometric Attention Transformer(Language Model) with the Leech...	37	Emerging	4	Jupyter Notebook
3	liangyuwang/Tiny-DeepSpeed Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library	36	Emerging	50	Python
4	Yigtwxx/PredictaLM PredictaLM is a lightweight Turkish language model designed for next-word...	35	Emerging	3	Python
5	catherinesyeh/attention-viz Visualizing query-key interactions in language + vision transformers (VIS 2023)	35	Emerging	162	HTML
6	viralcode/superGPT Train your own LLM from scratch	35	Emerging	7	Python
7	microsoft/Text2Grad 🚀 Text2Grad: Converting natural language feedback into gradient signals for...	35	Emerging	31	Python
8	huangjia2019/llm-gpt From classic NLP to modern LLMs: building language models step by step....	33	Emerging	194	Jupyter Notebook
9	FareedKhan-dev/Building-llama3-from-scratch LLaMA 3 is one of the most promising open-source model after Mistral, we...	33	Emerging	203	Jupyter Notebook
10	sondotpin/llm-scratch ChatGPT-like LLM in PyTorch from scratch, step by step	32	Emerging	17	Jupyter Notebook
11	FareedKhan-dev/train-tiny-llm Train a 29M parameter GPT from Scratch	32	Emerging	34	Python
12	better-py/learn-py Test code was written for research and verification of some Python libraries.	31	Emerging	14	Jupyter Notebook
13	CoinCheung/gdGPT Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline...	30	Emerging	97	Python
14	Arezkiiiii/mini_llm 🚀 Build and understand a Large Language Model from scratch using PyTorch...	29	Experimental	3	Jupyter Notebook
15	JoelDeonDsouza/Zenpool_LLM Zenpool is a compact, fine-tuned MLL (Mini Language Learner) model	27	Experimental	5	Jupyter Notebook
16	RWKV-Wiki/MultilingualShareGPT MultilingualShareGPT, the free multi-language corpus for LLM training	26	Experimental	73	—
17	loke-x/jam-gpt An Experimental Reimplementation of LLM models for research and development process	26	Experimental	21	Jupyter Notebook
18	GURPREETKAURJETHRA/Multi-GPU-Fine-Training-LLMs Multi GPU Fine Training LLMs using DeepSpeed and Accelerate.	26	Experimental	2	Jupyter Notebook
19	Ajax0564/VyomAI VyomAI: state-of-the-art NLP LLM Vision MultiModel transformers ...	26	Experimental	5	Python
20	MistyozAI/CosmicFish CosmicFish is a fully original, efficient transformer-based language model...	25	Experimental	3	Python
21	MuzzammilShah/Road-to-AI A structured documentation hub for AI and ML concepts, based on Andrej...	25	Experimental	19	—
22	Shinichi0713/LLM-fundamental-study this site is the fundamental page of LLM-mechanism	25	Experimental	3	Jupyter Notebook
23	sheikhcoders/SheikhLM Blazingly fast and remarkably powerful.	24	Experimental	1	Python
24	wozeparrot/tinyrwkv tinygrad port of the RWKV large language model.	24	Experimental	45	Python
25	harshtiwari01/llm-heatmap-visualizer A set of scripts to generate full attention-head heatmaps for transformer-based LLMs	24	Experimental	2	Jupyter Notebook
26	MDalamin5/Build-and-Finetune-LLM-From-Scratch-Deploy-via-vLLM-AWS-GCP A complete end-to-end learning repo covering everything from building Large...	24	Experimental	—	Jupyter Notebook
27	liangyuwang/train-large-model-from-scratch A minimal, hackable pre-training stack for GPT-style language models	23	Experimental	7	Python
28	mtkaya/transformer-edge-optimization Optimize Transformer models for edge devices	23	Experimental	1	Jupyter Notebook
29	mytechnotalent/SARAN SARAN: Shallow Auto-Regressive Attention Network	23	Experimental	6	Python
30	max1ab/smol-training-playbook-zh Hugging Face 文章《The Smol Training Playbook》中文翻译	23	Experimental	1	HTML
31	Matonice/LLM_From_Scratch Implementing all the parts and architecture of a large language model from scratch	22	Experimental	—	Python
32	filipsedivy/SLiM-CZ-V1 🇨🇿 Slavic Linguistic integrated Micro-model for Czechia	22	Experimental	—	Python
33	logic-OT/Decoder-Only-LLM This repository features a custom-built decoder-only language model (LLM)...	22	Experimental	16	Jupyter Notebook
34	fahadsid1770/Aesop-LLM custom Bengali Language Model built from scratch using PyTorch, featuring...	22	Experimental	—	Jupyter Notebook
35	ag-sc/CRAM-Generation-LLM Using ChatGPT & GPT-4 to generate CRAM & PyCRAM designators in a one-shot fashion	22	Experimental	4	Python
36	Michael-Jackson666/Zero2Hero-AI From first principles to state-of-the-art: A hands-on journey implementing...	21	Experimental	2	Python
37	IAAR-Shanghai/Awesome-Attention-Heads An awesome repository & A comprehensive survey on interpretability of LLM...	21	Experimental	400	TeX
38	ranpy13/Learning-LLM Learning to build LLM from scratch, following rasbt/LLMs-from-scratch footsteps.	21	Experimental	8	Jupyter Notebook
39	kreasof-ai/Homunculus-Project Long term project about a custom AI architecture. Consist of cutting-edge...	21	Experimental	8	Python
40	musa-kal/smoll-language-model Educational language model implementation from scratch, featuring Bigram and...	20	Experimental	1	Python
41	AnkitaMungalpara/Building-LLM-From-Scratch This repository provides a step-by-step guide to creating your own large...	20	Experimental	1	Jupyter Notebook
42	adorow/LlmFromScratch Notes and code from reading "Build a Large Language Model (From Scratch)"	19	Experimental	—	Jupyter Notebook
43	ashrithssreddy/llm-from-scratch Building GPTs from the ground up. A hands-on journey through attention...	19	Experimental	—	Jupyter Notebook
44	Tek233/Build_a_Large_Language_Model_From_Scratch Testing and experimenting with code from Sebastian Raschka’s book Build a...	19	Experimental	—	Jupyter Notebook
45	mcbieda/llm-from-scratch 1. A simple implementation of the LLM gpt-2 "from scratch" that allows easy...	19	Experimental	—	Jupyter Notebook
46	wln20/Attention-Viewer A plug-and-play tool for visualizing attention-score heatmap in generative...	19	Experimental	51	Python
47	SreeEswaran/Train-your-LLM This repository contains code and resources for training, fine-tuning, and...	18	Experimental	6	Python
48	benthecoder/AI learning AI from scratch	17	Experimental	15	Jupyter Notebook
49	njmarko/llm-gpt-sort A new type of sorting algorithm. Use large language model (llm like gpt,...	17	Experimental	12	Jupyter Notebook
50	hulkiciray/llm_from_scratch Comprehensive fresh start for how LLMs work	16	Experimental	4	Jupyter Notebook
51	simply-pouria/The-LMs-Book My study notes, code implementations, etc. while reading The Hundred-Page...	16	Experimental	1	Jupyter Notebook
52	ikun-llm/ikun-2.5B 练习时长两年半的 AI 大模型 (实际 26M params，2.5B = 两年半) \| ikun meme-culture chatbot 🐔🏀	16	Experimental	2	Jinja
53	snnclsr/chatgpt-from-scratch A full-stack ChatGPT-like application built (almost) from scratch	16	Experimental	7	Python
54	aryagxr/llm-from-scratch Implementation of LLM ✨from scratch✨	15	Experimental	2	Jupyter Notebook
55	ariannamethod/RRPRAM RRPRAM: Recurcsive Resonant Pattern Recognition Attention Mechanism by Arianna Method	15	Experimental	1	C
56	MAlshurbaji/LLM-Playground-From-Scratch A hands-on educational repository demonstrating how to build a miniature LLM...	15	Experimental	—	Jupyter Notebook
57	MaharshPatelX/multi-model-conda-environment This GitHub repository provides a comprehensive guide on setting up a Conda...	15	Experimental	6	—
58	ikun-llm/ikun-pretrain 从零预训练语言模型 \| Pretrain LLM from scratch 📚	14	Experimental	—	—
59	RajdeepKushwaha5/LLMs-from-scratch This repository contains the code for developing, pretraining, and...	14	Experimental	—	Jupyter Notebook
60	Kitsunp/Small-lenguaje-Model-Hybrid-Norm-Furier-Formers A compact language model implementing HybridNorm and Fourier-based...	14	Experimental	4	Python
61	moorebrett0/microformer A minimal, dual-memory transformer language model built from scratch in...	14	Experimental	4	Python
62	itxmjr/LLM-From-Scratch A step-by-step Guide: Build a GPT-like LLM From Scratch using PyTorch	14	Experimental	—	Jupyter Notebook
63	rudyon/pipeline Training pipeline for LLMs in PyTorch.	14	Experimental	—	Python
64	nachammai779/Large-Language-Models-from-Scratch 🧠 Building Large Language Models from the ground up — BPE tokenization,...	14	Experimental	—	Jupyter Notebook
65	Adwerse/Mini_LLM 🧠 Transformer built from scratch — RoPE, SwiGLU, KV-Cache, Flash Attention....	14	Experimental	—	Jupyter Notebook
66	CastorYu/train-hybrid-llm-from-scratch A simplistic script for training your own hybrid llm (using autoregressive...	13	Experimental	2	Python
67	FreezB11/PsyDuck a 60M parameter LLM from scratch	13	Experimental	2	Jupyter Notebook
68	prasanna00019/Small-Language-Models A collection of Small Language Models (SLMs) built from scratch in PyTorch.	13	Experimental	2	Jupyter Notebook
69	JerryPan2718/flexgpt Tradeoff between runtime and RAM usage for large language model inference.	13	Experimental	6	Jupyter Notebook
70	CS-433/ml-project-2-mlp Advancing Homepage2Vec with LLM-Generated Datasets for Multilingual Website...	13	Experimental	5	Jupyter Notebook
71	ictup/Build-a-LLM-model-from-scratch-simple LLM pipeline: data→tokenizer→attention→GPT train/eval→instruction...	13	Experimental	2	Jupyter Notebook
72	pprp/smol_training_zh 《Smol 训练手册》：打造世界级大模型的秘诀	12	Experimental	9	Shell
73	bobo0810/MiniGPT-4-DeepSpeed MiniGPT-4基于DeepSpeed加速➕ 扩充模型规模 ➕ 实验分析	12	Experimental	4	—
74	UEFI-code/PyTorch_For_PoorGuys This is a special PyTorch For Poor Guys Who can't afford big GPU	12	Experimental	4	C++
75	Antoinechss/LLM-from-scratch Building a LLM (GPT-3 like architecture) from scratch in Python and training...	12	Experimental	1	Python
76	FranxYao/Language-Model-Pretraining-for-Text-Generation LM pretraining for generation, reading list, resources, conference mappings.	12	Experimental	20	—
77	feljost/fgpt Building a conversational LLM from scratch	12	Experimental	1	Python
78	daniel-mehta/CooperLM-354M A 354M parameter GPT-2 model trained on filtered Wikipedia, BookCorpus, and...	12	Experimental	1	Jupyter Notebook
79	ajheshbasnet/model-reincarnated a collection of re-implementations of renowned artificial intelligence...	12	Experimental	1	Jupyter Notebook
80	Daegybyte/LLM_Training_Pipeline Modular text-generation training pipeline using PyTorch, Hugging Face, and...	11	Experimental	—	Jupyter Notebook
81	AndrewKM210/micro-llm A micro-LLM implementation that should be able to train and run on a standard laptop	11	Experimental	—	Jupyter Notebook
82	hichamcharhi/llm-from-scratch Implémentez un LLM from scratch en Python — cours pédagogique en...	11	Experimental	—	Jupyter Notebook
83	sugam24/LLM-From-Scratch Building a Large Language Model from the ground up.	11	Experimental	—	Jupyter Notebook
84	falseywinchnet/ConvexGPT Mostly Convex ICNN based Large Language Model	11	Experimental	—	Jupyter Notebook
85	insooeric/LLM_Small Long Language Model from SCRATCH	11	Experimental	—	Jupyter Notebook
86	Bhardwaj-Saurabh/Build_A_Small_Language_Model A detail guide to build, train, fine-tune and deploye SLM. This repo...	11	Experimental	—	Jupyter Notebook
87	teatonedev/Llama3.2-From-Scratch This repository serves as an educational resource for building an...	11	Experimental	—	Python
88	omkarbg100/LLM-GPT This is a LLM Model with full Architecture Coded ans is based On GPT-2 Weights	11	Experimental	—	Jupyter Notebook
89	jvscursulim/build-a-llm-from-scratch Studies about LLM	11	Experimental	—	Jupyter Notebook
90	capecoder08/llm-playground Playing with tokenizers, transformers, and LLMs	11	Experimental	—	Jupyter Notebook
91	Samya-S/Building-LLMs-from-scratch A hands-on guide to implementing Large Language Models from scratch	11	Experimental	—	Jupyter Notebook
92	patrikwolf/ttt_theory Specialization after Generalization	11	Experimental	6	Python
93	storytellingengineer/Learning_LLM_using_Pytorch Hi, this repo is my learning to build LLMs from scratch \| Foundational...	11	Experimental	—	Python
94	Jkanishkha0305/LLMs-from-Scratch A curated collection of Large Language Models(LLMs), Small Language...	11	Experimental	—	Jupyter Notebook
95	jbergq/transformer-language-modeling Notebook showing how to implement and train a transformer for language modeling	10	Experimental	1	Jupyter Notebook
96	anafisa/Text2Text-Transformer Large Multi-Language Models for News Translation	10	Experimental	1	Jupyter Notebook
97	T-Mohamed-Shafeek/LLM-for-Language-Translation This repository contains a simple and beginner-level noteboook which employs...	10	Experimental	1	Jupyter Notebook
98	sasa5linkar/Spacy-Serbian-Transformer Using Jertehs corpuses to train spacy with tranformer and LLM	10	Experimental	1	Jupyter Notebook
99	aayushsharma-io/tensorix Tensorix is a minimalistic and highly compatible Python library designed to...	10	Experimental	1	—
100	gusye1234/gpt-adapter Collections of algorithms for efficiently fine-tuning LLM	10	Experimental	1	Python
101	asiff00/Text-Generation-with-TensorFlow-from-Scratch Text Generation with TensorFlow from Scratch	10	Experimental	1	Jupyter Notebook
102	aaaastark/NBART-Multilingual-Translator This repository contains a Python script that uses a pre-trained NBART...	10	Experimental	1	Jupyter Notebook

Comparisons in this category

litgpt and llm-gpt (81 vs 33) litgpt and jam-gpt (81 vs 26) litgpt and superGPT (81 vs 35)