Text Cleaning Python Packages

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

harvesttext

文本挖掘和预处理工具（文本清洗、新词发现、情感分析、实体识别链接、关键词抽取、知识抽取、句法分析等），无监督或弱监督方法

squeakycleantext

Text preprocessing and PII anonymisation for NLP/ML. ONNX NER ensemble, language detection, stopword removal. Built for statistical ML and language models.

extractnet

A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one package

asciilint

ASCII and character policy checks for text files

watermark-cleaner

Remove AI text artifacts, hidden unicode characters and file metadata before you publish. Deterministic, offline, zero dependencies. Python and Node CLIs with one shared rule set.

textlasso

TextLasso is a Simple Python library for extracting structured data from raw text, with special focus on processing LLM (Large Language Model) responses.

wisup-e2m

E2M converts various file types (doc, docx, epub, html, htm, url, pdf, ppt, pptx, mp3, m4a) into Markdown. It’s easy to install, with dedicated parsers and converters, supporting custom configs. E2M offers an all-in-one, flexible, and open-source solution.

pnlp

NLP预/后处理工具。

valx

ValX is an open-source Python package for text cleaning tasks, including profanity detection and removal. Now also includes sensitive information detection, and removal.

araclean

Offset-preserving Arabic text normalization & cleaning for NLP — map cleaned spans back to the original. Pure-Python, typed, reproducible profiles.

nlp-preprocessing

Text Preprocessing Package includes cleaning, tokenization, dataset preparation ...etc

farsflow

Deterministic, dependency-free, and semantic-safe Persian text preprocessing library

rehuman

Unicode-safe text cleaning & typographic normalization for Rust

banglish-stopwords

A lightweight Python library to filter 350+ Banglish stopwords for NLP and text cleaning.

topicrankpy

A Python package to get useful information from documents using TopicRank Algorithm.

textcl

Text preprocessing package for use in NLP tasks https://pypi.org/project/textcl/

mim-nlp

A Python package with ready-to-use models for various NLP tasks and text preprocessing utilities. The implementation allows fine-tuning.

wmc-cleaner

Remove AI text artifacts, hidden unicode characters and file metadata before you publish. Deterministic, offline, zero dependencies. Python and Node CLIs with one shared rule set.

textprepro

👀 Everything Everyway All At Once Text Preprocessing for Natural Language Processing.

takin

A Python toolkit for file processing, text cleaning and data splitting. 文件处理，文本清洗和数据划分的python工具包。

hnlp

NLP预/后处理工具。

text-cleaner-yuvan

A lightweight Python package for cleaning, normalizing, and tokenizing text data.

ternaus-cleantext

Cleans text as in the CLIP model