Duplicates Python Packages

jscpd

Copy/paste detector for source code. 220+ languages, Rust engine, SARIF/HTML/badge reporters, GitHub Action, MCP server for AI agents.

removedup

Remove duplicates from parallel corpora

immich-photo-manager

MCP server and Claude Code plugin for self-hosted Immich: CLIP and OCR search, geographic albums, duplicates, people and faces, metadata repair, PDF photobooks. 94 tools, tested live on Immich 2.x and 3.x.

mail-deduplicate

📧 CLI to deduplicate mails from mail boxes

pyunique

Pyunique helps you get rid of duplicate files

dupe-eraser

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

deplicate

Advanced Duplicate File Finder for Python

maildir-deduplicate

📧 CLI to deduplicate mails from mail boxes

findupe

Safe duplicate finder & reviewer for macOS — exact + perceptual image matching, review-first deletion to Trash

arrayhascher

Fast hash in 2D Arrays (Numpy/Pandas/lists/tuples)

a-pandas-ex-duplicates-to-df

Creates a DataFrame/Series from duplicates

dropduplicatesplanb

Drops duplicates in DataFrames with tedious dtypes

screwduplicates

provides a simple and efficient way to remove duplicates from an iterable (even with unhashable elements, optional order preservation)

uniqdiff

Stable comparison engine for datasets, files, streams, and connector-backed sources.

deplicate-cli

Command Line Interface for deplicate

imageduplicatefinder

Software that removes duplicated images from folder2 in respect to folder1

yadupe

Yet another tool to find and remove duplicate files.

duplicateindexer

Find duplicates in multiple lists and return their indices and values.

drop-duplicates-nested-list

Drops duplicates from nested list

listset

remove duplicates from lists

stridesduplicatefinder

Calculate overlapping values between two arrays and return the results as a DataFrame

searchdups

Search for duplicate files