Corpus Tools Python Packages

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

treesearch-ud

High-performance toolkit for querying linguistic dependency parses

ms3

A parser for annotated MuseScore 3 files.

fundus

A very simple news crawler with a funny name

opusfilter

OpusFilter - Parallel corpus processing toolkit

align

Python library for extracting quantitative, reproducible metrics of multi-level alignment between speakers in naturalistic language corpora.

polars-corpus

Polars extensions for corpus linguistics

mfte

MFTE (Multi Feature Tagger of English) Python is the Python version based on Le Foll's MFTE written in Perl. It is extended to include semantic tags from Biber (2006) and Biber et al. (1999), including other specific tags.

concordancer

Searching in-memory corpus with Corpus Query Language (CQL)

audiomate

Python library for handling audio datasets.

mineru-html

MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.

ua-gec

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language

betastr

An open source reimplementation of Benny Brodda's BETA in Python

lyricscorpora

An unofficial Python API that allows users to create a corpus of lyrical text from their favorite artists and billboard charts

corpus-similarity

Measure the similarity of text corpora for 74 languages