yuanzhoulvpi2017/zero_nlp

Built on PyTorch and Hugging Face Transformers, this framework provides end-to-end training pipelines for Chinese NLP tasks spanning text classification, generation (GPT-2, Dolly, LLaMA, ChatGLM), and multimodal models (CLIP, vision-encoder-decoder). It handles production-scale data processing through memory-mapped I/O and multithreading for datasets up to 100GB+, and implements multi-GPU pipeline parallelism and tensor parallelism for models exceeding single-GPU VRAM limits. The project includes comprehensive tutorials covering data cleaning, model modification (vocabulary pruning/expansion), LoRA fine-tuning, and deployment strategies across 15+ model architectures including recent additions like Qwen2, InternLM, and LLaVA.

3,783 stars. No commits in the last 6 months.