Token Classification Python Packages

adaseq

AdaSeq: An All-in-One Library for Developing State-of-the-Art Sequence Understanding Models

lettucedetect

Span-level grounding verification for RAG, code, and tool-grounded AI outputs.

squeakycleantext

Text preprocessing and PII anonymisation for NLP/ML. ONNX NER ensemble, language detection, stopword removal. Built for statistical ML and language models.

meddeid

Local-first multilingual clinical text de-identification framework with Dutch and English models, pluggable language profiles, and Python, CLI, batch, and HTTP interfaces.

spanwend

Convert between sequence tags and spans, with validation and repair for BIO, BILOU, IOB1, and related schemes

transformers-keras

Transformer-based models implemented in tensorflow 2.x(using keras).

ctxprune

Context compression for AI agents: LLMLingua-2 successor that keeps IDs, paths and numbers intact. 68.8% vs 53.4% QA at 50% kept.

factgenie

Lightweight self-hosted span annotation tool

smile-datasets

Data pipelines for both TensorFlow and PyTorch!

ua-datasets

A collection of datasets for Ukrainian language

rapidnlp-datasets

Data pipelines for both TensorFlow and PyTorch!

naivenlp-datasets

Data pipelines for both TensorFlow and PyTorch!

temporal-taggers

Code and data form the paper BERT Got a Date: Introducing Transformers to Temporal Tagging