Data Compression Python Packages

glyd

Lossless AI compression: 33% less GPU memory, bit for bit.

zxc-compress

Lossless compression C library built for ultra-fast decode. Faster than LZ4 (40-75%+ on ARM64), at a better ratio. Official Rust, Python, Node.js, Go and WASM bindings. Write-once/read-many, with optional seekable archives for O(1) random access.

tensorflow-compression

Data compression in TensorFlow

rgclib

Recursive Group Coding (RGC): fast entropy coder with better compression than AC/Huffman on large alphabets – C lib + Python package

nanopq

Pure python implementation of product quantization for nearest neighbor search

pyvq

A vector quantization library for Rust :crab: with Python bindings 🐍

lzhw

Compression suite for data frames and tabular data files, csv, excel etc. Using LZHW algorithm.

algorithmselector

Explore our extensive Algorithms Repository, featuring a diverse range of algorithms from computational methods to data structures, cryptographic techniques, and AI. Ideal for education, research, and practical application, each algorithm is well-documented, tested, and optimized for performance and readability.

dense-basis

Dense Basis SED fitting

rii

Fast and memory-efficient ANN with a subset-search functionality

tensorflow-compression-ops

Data compression in TensorFlow

py-rans

A lightweight rANSCoder meant for rapid prototyping.

pfc-jsonl

Cold storage that queries like hot storage. High-ratio JSONL compressor with block-level random access. Query cold storage in seconds — no restore, no full decompression. 5–13% ratio on JSONL logs, 25–57% smaller than gzip. 1-hour window via DuckDB in under 6s. Free for personal and open-source use.

blackhole-blkh

Bit-perfect neural implicit compression (SIREN + hybrid residual). Excellent scaling on smooth images, textures, 3D volumes & game assets. Includes Gradio web demo.

tt-sketch

Sketching algorithms for Tensor Train decompositions

cra5

A large compression model for weather and climate data, which compresses a 400+ TB ERA5 dataset into a new 0.8 TB CRA5 dataset.

ravdec

Ravdec is a Python module implementing a lossless data compression algorithm designed by Ravin Kumar on September 19, 2016. This algorithm is designed exclusively for textual data, including alphabets, numbers, and symbols.