theosorus/GPT2-Hasktorch
GPT2 implementation in Haskell with the Hasktorch library, inspired by Andrej Karpathy's Pytorch implementation.
Implements the complete GPT-2 architecture (12 transformer blocks, 12 attention heads, 768 embedding dimensions) with full training pipeline including gradient accumulation, lazy data loading for large text files, real-time metric visualization, and CUDA support via Hasktorch. Leverages Haskell's type system and functional paradigm to replicate Karpathy's nanoGPT while integrating the OpenAI tokenizer for inference-ready model weights.
No commits in the last 6 months.
Stars
36
Forks
2
Language
Haskell
License
BSD-3-Clause
Category
Last pushed
Jun 17, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/theosorus/GPT2-Hasktorch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jingyaogong/minimind
🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!
kyegomez/TeraGPT
Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT
noah-hein/mazeGPT
AI model for making mazes that extends OpenAIs GPT2 model