Chunking Python Packages

semchunk

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

quebra-frases

chunks strings into byte sized pieces

fastcdc

FastCDC implementation in Python https://pypi.org/project/fastcdc/

btx-lib-mail

Send email from Python or the shell: streamed large attachments (bounded memory), RFC 3030 BDAT/CHUNKING, STARTTLS, multi-host failover, and attachment security.

cinematlas

Fuse within a unit, chunk across units: joint-vector search for video, photos, slides and any multimodal records on MongoDB Atlas + Voyage AI. Tested on 4 corpora with pre-stated predictions.

chunk-your-tools

Chunk MCP tools: Tool with required properties; optional Properties and Enums

chunk-your-skills

Chunk Skills: Frontmatter; Index of markdown headers 2 nodes; recompose with PageIndex natural page form

chunkr-rs

A fast chunking library

blazechunk

⚡ The fastest semantic text chunking library a SIMD-accelerated Rust core with a sync + async Python API.

smart-slice

Fidelity-first document slicing for RAG pipelines: ~30 formats in, retrieval-ready paragraphs out.

chunklet-py

One library to split them all: Sentence, Code, Docs. Chunk smarter, not harder; built for LLMs, RAG pipelines, and beyond.

betterhtmlchunking

BetterHTMLChunking is a Python library for intelligent HTML segmentation. It builds a DOM tree from raw HTML and extracts content-rich regions of interest, making content analysis effortless. Great for LLM based processing.

tryworks

Turn PDF, Word, PowerPoint, Excel, HTML, Markdown and email into LLM-ready elements. unstructured's partition API with zero required dependencies.

contextifier

Contextify is a document processing library that converts raw documents into AI-understandable context. It analyzes, restructures, and normalizes content so that language models can reason over documents with higher accuracy and consistency.

konan

Blazingly fast text chunkers in Rust with pythonic bindings. Rayon-parallel, real async, char-accurate offsets, semantic chunking

docslicer

Fast, deterministic document parser and chunker for RAG and agent pipelines. Ships an MCP server for agents navigating long documents with precision.

chunkshop

Standalone ingest-to-pgvector with a built-in chunker × embedder bakeoff. Python + Rust.

chunkrank

Model-aware text chunking and answer re-ranking for LLM pipelines. Python 3.10+, async-first, 54 models.

pavedb

PaveDB is an inspectable retrieval database for AI applications: text-backed vector search with source provenance and query replay, embedded or over HTTP. Website: https://pavedb.org/

lumberjack-py

Structure-aware document splitter for RAG — parses any document into a unified AST and chunks by section/block boundaries. 面向 RAG 的结构感知文档分割器——将任何文档解析为统一的 AST，并按节/块边界进行分割。

nano-xet

A tiny Xet: fsspec nxet:// filesystem with gear-hash content-defined chunking, deduplication and xorbs, on any filesystem

retrieval-eval

Every re-chunk quietly invalidates part of your RAG test suite. retrieval-eval tells you which labels are still true, and repairs the ones it can.

langparse

LangParse is a universal document parsing and text chunking engine for LLM or Agent applications — Documents In, Knowledge Out.

chunklab

Tells you whether the difference between chunking strategies is real — on your own documents. Local, no API key, no telemetry.