Text Segmentation Python Packages

symspellpy

Python port of SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

catalyst

Accelerated deep learning R&D

fast-sentence-segment

Fast and Efficient Sentence Segmentation

ekphrasis

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

harvesttext

文本挖掘和预处理工具（文本清洗、新词发现、情感分析、实体识别链接、关键词抽取、知识抽取、句法分析等），无监督或弱监督方法

deepsegment

Sentence Segmentation with sequece tagging

morphseg

A multilingual package for segmenting text into morphemes using supervised deep learning.

symspellcpppy

Fast SymSpell written in c++ and exposes to python via pybind11

tajik-text-segmentation

Tajik text segmentation algorithms

chunkseg

Evaluate chaptering quality for audio and video content in time space, supporting segmentation and title generation

lingpatlab

LingPatLab: Linguistic Pattern Laboratory

pawpaw

High Performance Text Processing & Segmentation Framework

mschunker

Smart text chunker for LLM preprocessing (sections → paragraphs → sentences → hard splits).

catalyst-pdm

Accelerated deep learning R&D

testmorphseg

An efficient and easy-to-use morpheme segmentation library