Langboat/Mengzi

Combines linguistic-informed pretraining and training acceleration techniques to deliver lightweight, efficient Chinese language models that maintain BERT-compatible architectures for seamless integration into existing NLP pipelines. Offers specialized variants including distilled 60M-parameter models, domain-specific financial versions, T5-based generation models supporting zero-shot capabilities, and multimodal Oscar variants for vision-language tasks. Optimized for resource-constrained deployment with FP16 training, custom tokenizers reducing token overhead, and corpus-pruned BLOOM variants.

542 stars. No commits in the last 6 months.