koomri/text-segmentation

Implements supervised neural segmentation using sentence embeddings (max pooling strategy) trained on wiki-727K and Choi datasets with word2vec representations. Built on PyTorch with configurable model architectures, includes dataset preprocessing pipelines for Wikipedia dumps and evaluation metrics via segeval. Provides CLI tools for training, evaluation, and custom dataset generation with TensorBoard logging support.

265 stars. No commits in the last 6 months.