PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Binary Deduplication Python Packages

Python packages with the GitHub topic binary-deduplication. Sorted by relevance, with stars and monthly downloads.
mr3od
bgate-unix

High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.

448 1 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery