Transformer Architecture Education LLM Tools

Educational implementations of transformer architectures from scratch using various frameworks (PyTorch, TensorFlow, NumPy), focusing on learning the core mechanisms. Does NOT include pre-trained model applications, fine-tuning frameworks, or production-ready inference systems.

There are 62 transformer architecture education tools tracked. 1 score above 50 (established tier). The highest-rated is LowinLi/transformers-stream-generator at 51/100 with 96 stars.

Get all 62 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-tools&subcategory=transformer-architecture-education&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	LowinLi/transformers-stream-generator This is a text generation method which returns a generator, streaming out...	51	Established	96	Python
2	jaymody/picoGPT An unnecessarily tiny implementation of GPT-2 in NumPy.	48	Emerging	3,454	Python
3	ystemsrx/mini-nanoGPT One-click training of your own GPT. Training a GPT has never been easier for...	48	Emerging	371	Python
4	Eamon2009/Codeformer-A.I A character-level GPT transformer built from scratch in PyTorch, trained on...	38	Emerging	3	Python
5	kyegomez/AttentionGrid A network of attention mechanisms at your fingertips. Unleash the potential...	37	Emerging	7	Python
6	kamalkraj/minGPT-TF A minimal TF2 re-implementation of the OpenAI GPT training	36	Emerging	58	Jupyter Notebook
7	NotShrirang/tinygpt 🎈 A series of lightweight GPT models featuring TinyGPT Base (~51M params)...	33	Emerging	16	Jupyter Notebook
8	abhaskumarsinha/MinimalGPT MinimalGPT is a concise, adaptable, and streamlined code framework that...	31	Emerging	24	Python
9	BobMcDear/trap Autoregressive transformers in APL	29	Experimental	107	APL
10	liangyuwang/Tiny-Megatron Tiny-Megatron, a minimalistic re-implementation of the Megatron library	28	Experimental	23	Python
11	Esmail-ibraheem/Transformer-pytorch Language to Language Transformer model from scartch using pure Pytorch where...	27	Experimental	9	Jupyter Notebook
12	naity/protein-transformer Implement, train, tune, and evaluate a transformer model for antibody...	27	Experimental	6	Jupyter Notebook
13	chrisjob1021/transformer-examples A collection of educational toy implementations and examples of key...	26	Experimental	3	Jupyter Notebook
14	akanyaani/minGPTF A TF re-implementation of the Karpathy's minGPT (Generative Pretrained...	26	Experimental	8	Python
15	pedrodiamel/gpt_mini_mini learning about llms	25	Experimental	3	Python
16	shamspias/Transformers-and-Large-Language-Models-From-Basics-to-Frontier-Research Dive into the transformative world of NLP with this guide on Transformers....	25	Experimental	5	—
17	DiogoNeves/mlx-gpt An approximate implementation of A. Karpathy's Let's build GPT, with MLX	23	Experimental	6	Jupyter Notebook
18	TolaniSilas/Mini-GPT This repository is an implementation of the GPT series.	23	Experimental	1	Jupyter Notebook
19	suvash/taxophoney GPT (Decoder only Transformer - from scratch) generated fake/phoney...	23	Experimental	2	Jupyter Notebook
20	JoshKeesee/Mini-GPT Train, run, and configure your own local AI from scratch! This includes a...	22	Experimental	—	Python
21	Gauravchy09/mini-gpt-from-scratch Mini GPT implementation from scratch to explore transformer architecture,...	22	Experimental	—	Jupyter Notebook
22	eonu/transformers-from-scratch Modular Python implementation of encoder-only, decoder-only and...	22	Experimental	1	Python
23	thejvdev/gpt-from-scratch Repository for Implementing a Decoder-Only GPT Model	22	Experimental	—	Jupyter Notebook
24	HSaurabh0919/CTransformers Implementing wide variety of transformers, fine tuning as well as trying...	22	Experimental	3	Jupyter Notebook
25	87owo/EasyGPT Easily train and inference on your personal computer, no need for large...	22	Experimental	35	Python
26	codewithdark-git/titans-transformer This repository contains an experimental implementation of the Titans...	17	Experimental	4	Jupyter Notebook
27	SvetimFM/transformer-training-interface Educational transformer neural network implementation with interactive...	17	Experimental	2	Python
28	MEHARKhaoula/transformers-text-coherence Fine-Tuning Five Types of Transformers and Incorporating Syntactic Features...	16	Experimental	4	Jupyter Notebook
29	ambideXtrous9/Transformer-from-Scratch Transformer from Scratch	16	Experimental	—	Python
30	TrainsFormerAI/TrainsFormerAI-1.0 TrainsFormer AI is a revolutionary desktop application that brings classic...	16	Experimental	1	Python
31	ashleysally00/transformers-and-attention Detailed guide to Transformer models that includes both technical and...	15	Experimental	—	—
32	lengyueit/gpt-mini OpenAI GPT的简单复现	15	Experimental	20	Python
33	scttfrdmn/local-code-model Pure Go implementation of a GPT-style transformer from scratch - educational...	15	Experimental	—	Go
34	UEFI-code/miniGPT An open-source project to show how to build a mini language model using PyTorch	15	Experimental	1	Python
35	emmetsite342/visual-transformer-guide Explain how Transformer AI models work with an interactive,...	14	Experimental	—	HTML
36	gatorduck/Creating_Custom_Decoder_Transformer Custom decoder Transformer that treats a patient's medical journey like a...	14	Experimental	—	Jupyter Notebook
37	LarissaTchomgang/miniGPT 🛠 Build a GPT-style language model from scratch with MiniGPT, offering an...	14	Experimental	—	Python
38	aitechnologies-it/gpt-mini Yet another minimalistic Tensorflow (re-)re-implementation of Karpathy's...	14	Experimental	14	Jupyter Notebook
39	rbbrdckybk/MiniGPT-4 Simplified local Windows OS setup of MiniGPT-4 running in an Anaconda...	14	Experimental	9	Python
40	FareedKhan-dev/best-introduction-to-transformer transformer again in the same manner as I did in my previous blog (for both...	13	Experimental	8	—
41	mcrimi/snakeformer Transformer model from scratch to simulate the good old game of Snake 🐍.	13	Experimental	2	Python
42	Ultron09/Numpy-Transformer A pure NumPy implementation of GPT built from scratch for educational...	13	Experimental	2	Python
43	fattorib/tritonformer Trainable transformer with fwd+bwd ops in Triton, matching the performance...	13	Experimental	6	Python
44	cpparnell/honeycrisp miniGPT built on Apple's MLX array framework	13	Experimental	5	Python
45	brianberns/MinGptSharp A F# .NET TorchSharp port of Andrej Karpathy's minGPT.	13	Experimental	10	F#
46	tsvlgd/gpt-from-scratch decoder-only Transformer (GPT) language model coded from scratch in pytorch	13	Experimental	2	Jupyter Notebook
47	ZZZ150751/cs336_spring2025_assignment1 Implementation of a Decoder-only Transformer language model from scratch for...	12	Experimental	1	Python
48	jongoiko/minigpt Training a tiny GPT-like Transformer language model	12	Experimental	1	Jupyter Notebook
49	sumony2j/SeedGPT SeedGPT is a minimal decoder-only LLM from scratch, inspired by andrej...	12	Experimental	1	Python
50	Anton3090/gpt2-mini-implementation 🧸 TinyStories Transformer — A lightweight 4-layer GPT-style model trained on...	11	Experimental	—	Jupyter Notebook
51	wahabzh/transformer-from-scratch 🤖 Complete Transformer implementation from scratch using PyTorch. Trained on...	11	Experimental	—	Jupyter Notebook
52	tensor-fusion/GPT-Haskell A pure Haskell implementation of a decoder-only transformer (GPT)	11	Experimental	21	Haskell
53	Hunain0786/miniTransformer Mini Transformer (Implemented From Scratch) A from-scratch implementation...	11	Experimental	—	Python
54	tulasinnd/Transformer-Decoder-Evolution This repository contains various decoder-only transformer versions built...	11	Experimental	—	Jupyter Notebook
55	sourize/Decodex This project implements a decoder-only GPT model from scratch using PyTorch.	11	Experimental	—	Jupyter Notebook
56	muhammadhussain-2009/Building-A-Transformer-From-Scratch Coding A Decoder Only Transformer Like ChatGPT From Scratch	11	Experimental	—	Jupyter Notebook
57	QuwsarOhi/PicoGPT A simple small GPT model for educative purpose that can be trained on CPU	11	Experimental	2	Python
58	am1tyadav/tensorflow-mingpt minGPT in TensorFlow	11	Experimental	—	Python
59	Rohan-Thoma/Coding-attention-from-scratch This repository consists code for executing attention mechanism from scratch...	11	Experimental	2	Jupyter Notebook
60	HMels/BassTab_Transformer This The Model scrapes huge amounts of bass music tableture from the...	10	Experimental	1	Python
61	thaisaraujom/about-gpt-and-transformer This repository provides a comprehensive and intuitive explanation of the...	10	Experimental	1	—
62	DataWorshipper/Machine_Translation This repository implements a Machine Translation system from scratch using...	10	Experimental	1	Jupyter Notebook

Comparisons in this category

minGPT-TF and minGPTF (36 vs 26)