Text Processing Python Packages

pyparsing

Python library for creating PEG parsers

pymupdf

PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.

jaconv

Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku

html-to-markdown

High performance and CommonMark compliant HTML to Markdown converter. Maintained by the Kreuzberg team. Kreuzberg is a fast, polyglot document intelligence engine with a Rust core. It extracts structured data from 98+ document formats using streaming parsers and built-in OCR.

pymupdfb

PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.

pythainlp

Thai natural language processing in Python

proces

🐨 text preprocess.

humanreadable

humanreadable is a Python library to convert human-readable values to other units.

rustling

A high-performance library for computational linguistics

wetextprocessing

Text Normalization & Inverse Text Normalization

twitter-text-parser

Twitter Text Libraries for Python

daachorse

🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. (Python wrapper for daachorse)

matcher-py

A high-performance matcher designed to solve LOGICAL and TEXT VARIATIONS problems in word matching, implemented in Rust.

flashtext2

Flashtext implementation in Rust

jpnorm

日本語テキスト正規化ライブラリ (Rust core + Python)。neologdn 互換・用途別プリセット・URL 保護・カスタム辞書 / Fast, configurable Japanese text normalization

split-markdown4gpt

A Python tool for splitting large Markdown files into smaller sections based on a specified token limit. This is particularly useful for processing large Markdown files with GPT models, as it allows the models to handle the data in manageable chunks.

hazm

Persian NLP Toolkit

yasbd-lib

A high-accuracy, multilingual rule-based Sentence Boundary Detector (SBD) with a drop-in adapter for pysbd, delivering faster and more accurate sentence segmentation.

deduce

Deduce: de-identification method for Dutch medical text

radixor-c

A multilingual stemming engine by Egothor.

nlpo3

Thai natural language processing library in Rust, with Python and Node bindings.

irregex

A full toolkit for building any type of search engine.

tfkit

🤖📇 handling multiple nlp task in one pipeline

mongol-norm

Shape-aware Unicode normalizer for Traditional Mongolian (Hudum): same visible word ⟹ same encoding. Pure-Python UTN #57 v4 shaping engine, zero dependencies. ᠮᠣᠩᠭᠣᠯ ᠪᠢᠴᠢᠭ