mirth/chonky

Uses fine-tuned transformer models (ModernBERT, mBERT) that learn semantic boundaries directly from training data, outperforming rule-based and embedding similarity approaches on standard benchmarks. Integrates with RAG pipelines and supports markup removal across HTML, XML, and Markdown formats; multiple model variants range from 66M to 396M parameters with multilingual options available on Hugging Face.

407 stars and 312 monthly downloads. Available on PyPI.