brightmart/nlp_chinese_corpus

Comprises five structured datasets totaling millions of documents: Wikipedia entries, news articles with metadata, categorized Q&A pairs, community-sourced Q&A with engagement metrics, and parallel translation pairs—all in JSON format for direct preprocessing. Targets pre-training of large-scale models and downstream tasks like text classification, question answering, and semantic similarity through richly annotated schemas. Integrates with the CLUE benchmark ecosystem, enabling standardized evaluation of Chinese NLP models across 10 unified tasks with one-line execution.

9,869 stars.