Large Files Python Packages

geofileops

Python toolbox to process large geospatial vector files faster.

geosplit

Python CLI and library for splitting large GeoJSON/Geopackage file by feature count or file size. Supports conversion between geopackage and geojson.

hsort

High-performance sorting and merging engine for large data files. Python extension with CLI and API support.

fast-file-encryption

Fast, secure, and minimal file encryption for large files using RSA and AES.

csv-schema-inference

A tool to automatically infer columns data types in .csv files

hydrastream

Concurrent HTTP downloader with in-memory stream reordering (curl_cffi + uvloop).

csv-shuffler

A tool to automatically Shuffle lines in .csv files

uniqdiff

Stable comparison engine for datasets, files, streams, and connector-backed sources.