Large Files Python Packages

geofileops

Python toolbox to process large geospatial vector files faster.

geosplit

Python CLI and library for splitting large GeoJSON file by feature count or file size. Supports conversion between geopackage and geojson.

hsort

A high-performance sorting and merging engine optimized for processing large-scale data files, with stable multi-key sorting support for fixed-length, variable-length, and CSV formats.

csv-schema-inference

A tool to automatically infer columns data types in .csv files

fast-file-encryption

Fast, secure, and minimal file encryption for large files using RSA and AES.

uniqdiff

Stable comparison engine for datasets, files, streams, and connector-backed sources.

hydrastream

Concurrent HTTP downloader with in-memory stream reordering (httpx + uvloop).

csv-shuffler

A tool to automatically Shuffle lines in a csv file