Chunking Python Packages

semchunk

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

quebra-frases

chunks strings into byte sized pieces

fastcdc

FastCDC implementation in Python https://pypi.org/project/fastcdc/

catalyst-chunker

An invariant-first chunk formation package for retrieval, agentic software, and document/code workflows that need source lineage and auditability.

chunkshop

Standalone ingest-to-pgvector with a built-in chunker × embedder bakeoff. Python + Rust.

konan

Blazingly fast text chunkers in Rust with pythonic bindings. Rayon-parallel, real async, char-accurate offsets, semantic chunking

table2rules

Convert HTML tables into flat, self-contained facts for LLMs and RAG pipelines.

betterhtmlchunking

BetterHTMLChunking is a Python library for intelligent HTML segmentation. It builds a DOM tree from raw HTML and extracts content-rich regions of interest, making content analysis effortless. Great for LLM based processing.

contextinator

Turning messy repos into weapons of mass structured context.

pavedb

PaveDB — Lightweight Vector Search Microservice

chunklet-py

One library to split them all: Sentence, Code, Docs. Chunk smarter, not harder — built for LLMs, RAG pipelines, and beyond.

faiss-vector-aggregator

This Python library provides a suite of advanced methods for aggregating multiple embeddings associated with a single document or entity into a single representative embedding.

chunklet

One library to split them all: Sentence, Code, Docs. Chunk smarter, not harder — built for LLMs, RAG pipelines, and beyond.

omnichunk

Structure-aware text chunking library for code, prose, and markup files. Intelligently splits files into context-rich chunks while preserving semantic boundaries. Supports 15+ programming languages, deterministic output, and zero external dependencies. Perfect for RAG systems, code analysis, and LLM context optimization.

mcp-pager

Token-aware response paging for MCP servers — chunks large tool responses and delivers them page by page with agent-readable metadata

chunkrank

Model-aware text chunking and answer re-ranking for LLM pipelines. Python 3.10+, async-first, 54 models.

rag-tui

Interactive terminal debugger and optimizer for RAG chunking pipelines

llmdocparser

A package for parsing PDFs and analyzing their content using LLMs.

seedbraid

Reference-based file reconstruction with CDC chunking, SBD1 binary seed format, and IPFS transport

notebooklm-chunker

Heading-aware PDF chunking with resumable source and Studio workflows for turning long documents into interactive NotebookLM learning kits.

chonky

Fully neural approach for text chunking

django-query-chunk

Django Query Chunk is used to split big queries into multiple chunks for prevent (too high) memory usage.

longparser

Privacy-first document intelligence engine — parse PDFs, DOCX, PPTX, XLSX & CSV into AI-ready chunks for RAG pipelines. Includes HITL review, 3-layer memory chat, and a production FastAPI server.

structchunk

Markdown chunker for RAG. Structure-aware splitting preserves full semantic context; tables split at row boundaries.