datawhalechina/llms-from-scratch-cn
仅需Python基础,从0构建大语言模型;从0逐步构建GLM4\Llama3\RWKV6, 深入理解大模型原理
# Technical Summary Provides hands-on implementation of LLM core components using PyTorch, covering tokenization, attention mechanisms, and GPT-style architectures through progressive Jupyter notebooks alongside detailed theoretical explanations. Includes architecture dissections and implementation guides for multiple production models (ChatGLM3/4, Llama3, RWKV V2-V6, MiniCPM), enabling learners to understand both foundational transformer mechanics and variant design choices across different model families.
4,010 stars. No commits in the last 6 months.
Stars
4,010
Forks
552
Language
Jupyter Notebook
License
—
Category
Last pushed
Aug 15, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/datawhalechina/llms-from-scratch-cn"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
FareedKhan-dev/train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
facebookresearch/LayerSkip
Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
kmeng01/rome
Locating and editing factual associations in GPT (NeurIPS 2022)
analyticalrohit/llms-from-scratch
Build a ChatGPT like LLM from scratch in PyTorch, explained step by step.