wdndev/llm_interview_note

Covers Transformer architecture fundamentals (attention mechanisms, positional encoding, tokenization) alongside practical implementations like LLaMA and ChatGLM model internals, with dedicated sections on distributed training strategies (data/tensor/pipeline parallelism), inference optimization via vLLM and quantization, and alignment techniques including RLHF and DPO. Integrates with frameworks like DeepSpeed and Megatron, while companion projects (tiny-llm-zh, tiny-rag, tiny-mcp) provide hands-on implementations for pretraining, RAG systems, and MCP-based agents on resource-constrained hardware.

13,130 stars. No commits in the last 6 months.