jaymody/picoGPT
An unnecessarily tiny implementation of GPT-2 in NumPy.
This project offers a bare-bones version of the GPT-2 language model, allowing developers to see its core mechanics with minimal code. You input a text prompt, and it generates a text completion. This is designed for software developers or machine learning engineers who want to understand the foundational structure of a transformer-based language model without the complexities of a full-scale implementation.
3,454 stars. No commits in the last 6 months.
Use this if you are a developer looking for an extremely simplified, readable implementation of GPT-2's forward pass to learn its architecture.
Not ideal if you need to train a model, perform batch inference, use advanced sampling techniques, or require fast text generation for real-world applications.
Stars
3,454
Forks
456
Language
Python
License
MIT
Category
Last pushed
Apr 24, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/jaymody/picoGPT"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
LowinLi/transformers-stream-generator
This is a text generation method which returns a generator, streaming out each token in...
ystemsrx/mini-nanoGPT
One-click training of your own GPT. Training a GPT has never been easier for beginners. /...
kamalkraj/minGPT-TF
A minimal TF2 re-implementation of the OpenAI GPT training
Eamon2009/Codeformer-A.I
A character-level GPT transformer built from scratch in PyTorch, trained on Linux kernel C...
kyegomez/AttentionGrid
A network of attention mechanisms at your fingertips. Unleash the potential of attention...