mookiezi/dataset-cleaning-toolkit

A dataset toolbox for preparing and analyzing conversational datasets, including CSV splitting, CSV → Parquet conversion, dataset statistics, Parquet cleaning and sorting, HuggingFace–style metadata generation, and batched chain insertion into PostgreSQL — with Rich progress, multiprocessing, and 32 GB-RAM-friendly batching.

12
/ 100
Experimental

No commits in the last 6 months.

Stale 6m No Package No Dependents
Maintenance 2 / 25
Adoption 1 / 25
Maturity 9 / 25
Community 0 / 25

How are scores calculated?

Stars

1

Forks

—

Language

Python

License

MIT

Category

text-preprocessing-pipelines

Last pushed

Oct 02, 2025

Commits (30d)

0

GitHub
Text Preprocessing Pipelines · 45 tools

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/nlp/mookiezi/dataset-cleaning-toolkit"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

chartbeat-labs/textacy

NLP, before and after spaCy

70

nltk/nltk_data

NLTK Data

61

prasanthg3/cleantext

An open-source package for python to clean raw text data

59

brightertiger/pygarble

Python Package to detect garbled, gibberish text for EN

54

jfilter/clean-text

🧹 Python package for text cleaning

53

Explore NLP Tools

All categories Trending NLP directory Insights