GPT2 Pretraining Fine-tuning Transformer Models

Tools for pretraining, fine-tuning, and implementing GPT-2 models from scratch, including language-specific variants and inference optimization. Does NOT include downstream applications like question-answering or summarization, nor other model architectures beyond GPT-2 variants.

There are 128 gpt2 pretraining fine-tuning models tracked. 3 score above 50 (established tier). The highest-rated is tabularis-ai/be_great at 67/100 with 350 stars and 5,517 monthly downloads.

Get all 128 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=transformers&subcategory=gpt2-pretraining-fine-tuning&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	tabularis-ai/be_great A novel approach for synthesizing tabular data using pretrained large language models	67	Established	350	Python
2	EleutherAI/gpt-neox An implementation of model parallel autoregressive transformers on GPUs,...	58	Established	7,399	Python
3	shibing624/textgen TextGen: Implementation of Text Generation models, include LLaMA, BLOOM,...	53	Established	979	Python
4	AdityaNG/kan-gpt The PyTorch implementation of Generative Pre-trained Transformers (GPTs)...	48	Emerging	725	Python
5	EleutherAI/gpt-neo An implementation of model parallel GPT-2 and GPT-3-style models using the...	47	Emerging	8,286	Python
6	keith2018/TinyGPT Tiny C++ LLM inference implementation from scratch	46	Emerging	106	C++
7	zemlyansky/gpt-tfjs GPT in TensorFlow.js	46	Emerging	33	JavaScript
8	kyegomez/GPT4o Community Open Source Implementation of GPT4o in PyTorch	44	Emerging	26	Shell
9	ai-forever/ru-gpts Russian GPT3 models.	44	Emerging	2,093	Python
10	0hq/WebGPT Run GPT model on the browser with WebGPU. An implementation of GPT inference...	44	Emerging	3,784	JavaScript
11	kakaobrain/kogpt KakaoBrain KoGPT (Korean Generative Pre-trained Transformer)	41	Emerging	1,014	Python
12	kyegomez/Lets-Verify-Step-by-Step "Improving Mathematical Reasoning with Process Supervision" by OPENAI	40	Emerging	114	Python
13	eric-ai-lab/MiniGPT-5 Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language...	36	Emerging	863	Python
14	mytechnotalent/RE-GPT Inspired by Andrej Karpathy’s "Let’s Build GPT", this project guides you...	36	Emerging	27	Jupyter Notebook
15	cdpierse/script_buddy_v2 Script Buddy v2 is a film script text generation tool built using film...	36	Emerging	47	Jupyter Notebook
16	turtlesoupy/this-word-does-not-exist This Word Does Not Exist	36	Emerging	1,021	Python
17	ai-forever/mgpt Multilingual Generative Pretrained Model	35	Emerging	207	Jupyter Notebook
18	hyperonym/basaran Basaran is an open-source alternative to the OpenAI text completion API. It...	35	Emerging	1,290	Python
19	kyegomez/CNNGPT This CNN-based language model leverages causal and dilated convolutions,...	34	Emerging	4	Python
20	datadreamer-dev/DataDreamer DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models. 🤖💤	34	Emerging	1,100	Python
21	erogol/BlaGPT Experimental playground for benchmarking language model (LM) architectures,...	34	Emerging	102	Python
22	sytelus/nanuGPT Simple, reliable and well tested training code for quick experiments with...	33	Emerging	13	Python
23	saqib1707/gpt2-from-scratch PyTorch Implementation of GPT-2	33	Emerging	31	Python
24	soumyadip1995/BabyGPT Something in the middle of Karpathy's mingpt model and video lectures, ...	32	Emerging	24	Jupyter Notebook
25	mytechnotalent/falcongpt Simple GPT app that uses the falcon-7b-instruct model with a Flask front-end.	32	Emerging	8	Python
26	TatevKaren/BabyGPT-Build_GPT_From_Scratch BabyGPT: Build Your Own GPT Large Language Model from Scratch Pre-Training...	32	Emerging	116	Python
27	wpeebles/G.pt Official PyTorch Implementation of "Learning to Learn with Generative Models...	32	Emerging	345	Python
28	arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for...	31	Emerging	113	Jupyter Notebook
29	EagleW/Stage-wise-Fine-tuning Code for Stage-wise Fine-tuning for Graph-to-Text Generation	31	Emerging	26	Lex
30	iVishalr/GPT A minimal and efficient Pytorch implementation of OpenAI's GPT (Generative...	31	Emerging	18	Jupyter Notebook
31	potamides/uniformers Token-free Language Modeling with ByGPT5 & Friends!	30	Emerging	12	Python
32	readme-generator/alreadyme-ai-research Generate README.md with GPT-3 few-shot learning	30	Emerging	27	Python
33	arrmansa/Gpt-Neo-Limited-Vram-Cuda A notebook that runs GPT-Neo with low vram (6 gb) and cuda acceleration by...	30	Emerging	14	Jupyter Notebook
34	losttech/Torch.MinGPT A C# implementation of GPT	30	Emerging	20	C#
35	dreamingjudith/KoGPT2-personachat Fine-tuned KoGPT2 chatbot demo with translated PersonaChat (ongoing)	29	Experimental	13	Jupyter Notebook
36	arrmansa/Basic-UI-for-GPT-Neo-with-low-vram A basic ui for running gpt neo 2.7B on low vram (3 gb Vram minimum)	28	Experimental	36	Jupyter Notebook
37	FareedKhan-dev/gpt4o-from-scratch Implementation of a GPT-4o like Multimodal from Scratch using Python	28	Experimental	78	Jupyter Notebook
38	EvilFreelancer/rugpt3-custom Pre-training custom ruGPT3 model on books written by F.M. Dostoevski	27	Experimental	7	Python
39	Agora-Lab-AI/OmniByteGPT An implementation of an all-new foundation model architecture that trains on...	27	Experimental	9	Python
40	jseeio/gpt2-tfjs GPT2 with Tensorflow.js	26	Experimental	4	JavaScript
41	tanulsingh/Humour.ai-Language-model-that-can-crack-Jokes Language Model that makes you Laugh .	26	Experimental	41	Python
42	StarxSky/ANE-GPT-New New ANE GPT	25	Experimental	5	Python
43	EdvardOlsen/Horoscope_generator This is a horoscope generating code	25	Experimental	4	Python
44	s-macke/GoPT GPT-2 Model Inference	25	Experimental	5	Go
45	ant-louis/belgpt2 🇧🇪 BelGPT-2: the 1st GPT model pretrained in French.	24	Experimental	34	Python
46	JarvisPei/FuseGPT The implementation for the paper, FuseGPT: Learnable Layers Fusion of...	24	Experimental	4	Python
47	Any-Winter-4079/Nano-GPT-Speedrun-Track This repo represents my Nano-GPT speedrun playground, which started coding...	23	Experimental	6	Python
48	trekhleb/homemade-gpt-js A minimal TensorFlow.js re-implementation of Karpathy's minGPT (Generative...	23	Experimental	88	TypeScript
49	fcakyon/gpt2-shakespeare A tutorial on GPT2 language model training with texts from Shakespeare	23	Experimental	1	Jupyter Notebook
50	mrseanryan/gpt-local Local GPT (llama 2 or dolly or gpt etc.) via Python - using ctransforers project	23	Experimental	2	Python
51	neuronalin/gpt-from-scratch-pytorch A decoder-only GPT-style Transformer built from scratch with PyTorch —...	22	Experimental	—	Python
52	JAVO932/PyGPT2 🖥️ Explore GPT-2 text generation with PyGPT2, a user-friendly Python app...	22	Experimental	—	Python
53	uSaiPrashanth/gpt-j-finetune Parallelizes finetuning of gpt-j on P3 dataset across multiple gpu nodes	22	Experimental	1	Python
54	Atenrev/forocoches-language-generation This is a PyTorch implementation of a decoder only transformer inspired on...	22	Experimental	7	Python
55	procesaur/Scratch2LM Training transformer models (e.g. RoBERTa, GPT2 and GPT-J) from scratch.	22	Experimental	7	Python
56	mytechnotalent/MicroGPT MicroGPT is a clean, educational implementation of the GPT (Generative...	21	Experimental	2	Python
57	chizkidd/microGPT Minimal char-level GPT inspired by @karpathy's microGPT: multi-dataset...	21	Experimental	2	Jupyter Notebook
58	lorenzomaiuri-dev/quantum-gpt A hybrid Quantum-Classical Transformer implementation based on nanoGPT,...	20	Experimental	1	Python
59	kabachuha/nanoGPKANT Testing KAN-based text generation GPT models	20	Experimental	18	Jupyter Notebook
60	pablo-reyes8/implementing-gpt Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks,...	20	Experimental	1	Python
61	Navy10021/KRLawGPT KRLawGPT : Generative Pre-trained Transformer for producing Korean Legal Text	19	Experimental	9	Python
62	aarxshi/DsaGPT A minimal GPT-style transformer built from scratch for DSA-style Q&A	19	Experimental	—	Jupyter Notebook
63	pronzzz/atomgpt AtomGPT is a chaotic, evolutionary implementation of a Generative...	19	Experimental	—	Python
64	s-omranpour/Shirin-Sokhan A Persian Poet Transformer! (finetuned GPT2 on Ganjoor data)	18	Experimental	5	Jupyter Notebook
65	Andras7/gpt2-pytorch Extremely simple and understandable GPT2 implementation with minor tweaks	18	Experimental	21	Python
66	codiceSpaghetti/numpyGPT A from-scratch GPT built with NumPy and Python’s standard library. No...	18	Experimental	20	Python
67	NJX-njx/microgpt 🔬 The most atomic GPT-2 implementation in 265 lines of pure Python & CUDA. A...	17	Experimental	21	HTML
68	SIC98/GPT2-python-code-generator GPT2 finetuning with transformers 🤗	17	Experimental	28	Jupyter Notebook
69	Eden-Eldith/WiggleGPT WiggleGPT is an language model that integrates bio-inspired neural...	16	Experimental	1	Python
70	Vadimbuildercxx/NumpyGPT A lightweight educational implementation of GPT (Generative Pre-trained...	16	Experimental	4	Python
71	Amir-Hofo/GPT2 Implementation of the GPT-2 architecture using PyTorch, trained on the...	15	Experimental	—	Python
72	RahulSChand/gpt2_squad GPT2 training on squad dataset	15	Experimental	2	Python
73	kyegomez/TinyGPTV Simple Implementation of TinyGPTV in super simple Zeta lego blocks	15	Experimental	16	Python
74	fattorib/Little-GPT GPT* - Training faster small transformers using ALiBi, Parallel Residual...	15	Experimental	21	Python
75	god01215/GPT-From-Scratch Implementation of a GPT-style LLM from scratch, following "Build a Large ...	15	Experimental	1	Jupyter Notebook
76	lin826/nanoGPT-demo Training and finetuning local GPTs.	15	Experimental	—	Python
77	inkybubble/mi_01_attention_patterns_scratch MI-01 - Attention Patterns from Scratch: Finding Previous-Token and...	15	Experimental	—	Python
78	Alibubere/scene2story Scene2Story is an AI-powered system that generates creative stories from...	15	Experimental	—	Python
79	buhsnn/eli5-gpt2-language-model Decoder-only Transformer (GPT-2 style) trained from scratch on the ELI5...	15	Experimental	1	Jupyter Notebook
80	jaketae/lm-identifier A toolkit for identifying pretrained language models from potentially...	14	Experimental	9	Python
81	eshaaaan/tinygpt 🤖 Simplify understanding of large language models with TinyGPT, featuring a...	14	Experimental	—	Python
82	Ojas025/almostGPT A GPT implementation for training and generating text on custom datasets	14	Experimental	—	Jupyter Notebook
83	chandan11248/GPT-2 Learning and implementing GPT-2 from scratch, including architecture...	14	Experimental	—	Jupyter Notebook
84	Sumeet8726/Hyper_rw 🛠️ Access virtual memory with the GuestMemory class and utility functions...	14	Experimental	—	HTML
85	diixo/build-gpt A PyTorch library with educational re-implementation of GPT-models: GPT2, LLaMA	14	Experimental	—	Python
86	brendandagys/ChadGPT From-scratch GPT experiments in PyTorch, covering attention mechanisms,...	14	Experimental	—	Jupyter Notebook
87	alkatrazstudio/neodim-server Natural language model AI via HTTP	13	Experimental	7	Python
88	marlo-z/reversal_curse_analysis Code for 'Towards a Theoretical Understanding of the 'Reversal Curse' via...	13	Experimental	5	Python
89	Sairamg18814/GUBBALA-V3-TRUE Revolutionary Self-Evolving Language Model - 100% self-contained AI trained...	13	Experimental	2	Python
90	jiseokson/PageBrain Light-weight LLM Serving with PagedAttention	13	Experimental	15	Python
91	zTgx/DeepText A GPT Model To Generate Text	13	Experimental	2	Python
92	iangitonga/gten A minimal library to run transformer neural networks on CPU.	12	Experimental	3	C
93	shreydan/shakespeareGPT understanding language modeling by training a small GPT on Shakespeare plays.	12	Experimental	13	Jupyter Notebook
94	alperiox/bookbot A toy project for my generative AI studies on text data. Train generative...	12	Experimental	1	Python
95	Divyansh900/PyCodeGen A python code generation model with 75M parameter built from the ground up...	12	Experimental	1	Python
96	bellthomas/gpt.local A work-in-progress, from-scratch implementation of a generative pre-trained...	12	Experimental	4	Python
97	PromptlyCode/inline-completion-model PromptlyCode inline completion model by PyTorch	12	Experimental	4	Python
98	Akhan521/GPT-From-Scratch 🧸 A fully custom GPT-style language model built from scratch using PyTorch...	11	Experimental	—	Python
99	dedsecurity/dpt Repo for offsite scale work	11	Experimental	2	Python
100	Agora-Lab-AI/NeoCore NeoCore™ - Next Generation CPU-Native Transformer.	11	Experimental	2	Python
101	neemiasbsilva/developing-nanoGPT2-fineweb Developing a cusstom nano GPT-2 from scratch using PyTorch on the Fineweb dataset.	11	Experimental	2	Python
102	sumony2j/SeedGPT-22M SeedGPT is a lightweight, 22M-parameter Transformer LLM for efficient text...	11	Experimental	—	Python
103	tahmidmir/Graph-RAG Fine-tuning GPT-2 on domain-specific articles related to skin cancer, using...	11	Experimental	2	Jupyter Notebook
104	jjantas/neural-networks-zero-to-hero This repository is a personal, in-depth reworking of Andrej Karpathy's...	11	Experimental	—	Jupyter Notebook
105	ademyanchuk/gpt2-diy From-scratch reproduction of GPT-2 following Andrej Karpathy's "Zero to Hero" series.	11	Experimental	—	Jupyter Notebook
106	n9e6y/PPG Persian Poetry Generator: A fine-tuned GPT-2 model for generating Persian...	11	Experimental	—	Jupyter Notebook
107	btboilerplate/GPT-2 Fine-tunes the GPT-2 language model on Shakespearean text to generate...	11	Experimental	—	Jupyter Notebook
108	oskarfernlund/noskGPT Simple transformer-based language model which generates Shakespearian dialogue.	11	Experimental	—	Python
109	TomaszKaleczyc/scifi_book_generator The purpose of this project is to build a decoder only transformer...	11	Experimental	—	Python
110	ayus1234/Text-Generation-with-GPT-2 A comprehensive toolkit for fine-tuning GPT-2 language models and generating...	11	Experimental	—	Python
111	taljindergill78/AI-Indian-Recipe-Generator AI-powered system that generates authentic Indian recipes using GPT-2 and...	11	Experimental	—	Python
112	BenBenyamin/GPT2 My implementation GPT2 from scratch using the original GPT2 and GPT3 papers	11	Experimental	—	Python
113	Wojtekb30/GPT-2-B200-pre-trainier Code for pre-training a GPT-2 model on (eight) NVIDIA DGX B200 GPUs and...	11	Experimental	—	Python
114	thewh1teagle/g2p-byt5 g2p with byt5	11	Experimental	—	Python
115	3ConstArt3/AIQuoteGenerator Fine-tune GPT-2 models on philosophers’ quotes with semantic tagging,...	11	Experimental	—	Python
116	NimeshRawanage/TextGen-AI-Transformer-Trainer Create your own text-generation AI (GPT). Fine-tune GPT-2 or T5 on your own...	11	Experimental	2	Python
117	vishxl/WisdumbAI [WIP] WisdumbAI: Generate thoughts/tweets using GPT-Neo.	10	Experimental	1	Jupyter Notebook
118	jndiogo/gptbench A python package to experiment with GPT-like transformer models	10	Experimental	1	Jupyter Notebook
119	Adam-Bowen/nanoGPT 🧠 nanoGPT (Andrej Karpathy's Zero to Hero)	10	Experimental	1	Jupyter Notebook
120	sartq333/story-GPT a simple GPT model pre-trained from scratch on tiny stories dataset	10	Experimental	1	Jupyter Notebook
121	J3lly-Been/gpt2-story-generation This project fine-tunes GPT-2, a popular pre-trained transformer model, to...	10	Experimental	1	Jupyter Notebook
122	Harsh-2909/gpt-from-scratch The "GPT from Scratch" project is an endeavor to implement the Generative...	10	Experimental	1	Jupyter Notebook
123	baumandm/lorem-insight Tool to generate lorem ipsum-style Insights for Insights Explorer	10	Experimental	1	Python
124	YashrajBaila7/GPT2LM A implimentation of GPT2 varient.	10	Experimental	1	Python
125	juletx/gpt2-eus Pretraining GPT2 model on Basque language	10	Experimental	1	Python
126	SynthWomb/Synthia SynthiaGPT leverages Google's Gemini & the Hugging Face Transformers library...	10	Experimental	1	Python
127	Uni-Creator/NanoGPT NanoGPT is a lightweight GPT-style language model designed for text...	10	Experimental	1	Python
128	Med-Karim-Ben-Boubaker/gpt-2-from-scratch A repository that shows the code behind different LLMs architectures and...	10	Experimental	3	Python

Comparisons in this category

gpt-neox and gpt-neo (58 vs 47)