PyRank

Binary Deduplication Python Packages

Python packages with the GitHub topic binary-deduplication. Sorted by monthly downloads, with stars and forks.
mr3od
bgate-unix

High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.

255 1 0
mr3od
triagehash

High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.

3 1 0