Duplicates Python Packages

removedup

Remove duplicates from parallel corpora

4K 7 1

uniqdiff

Stable exact comparison engine foundation for Python and the UniqTools ecosystem.

615 1 0

deplicate

Advanced Duplicate File Finder for Python. Nothing is impossible to solve.

450 79 17

imageduplicatefinder

Simple duplication finder for Images, matches on names and then compares image hashes.

358 0 1

pyunique

Pyunique helps you get rid of duplicate files

354 0 0

thebear

Bear - the decluttering deduplicator

242 4 1

arrayhascher

Fast hash in 2D Arrays (Numpy/Pandas/lists/tuples)

178 1 0

a-pandas-ex-duplicates-to-df

Creates a DataFrame/Series from duplicates

161 0 0

deplicate-cli

Command Line Interface for deplicate.

142 3 1

dropduplicatesplanb

Drops duplicates in DataFrames with tedious dtypes

140 0 0

yadupe

Yet another tool to find and remove duplicate files.

133 0 1

screwduplicates

provides a simple and efficient way to remove duplicates from an iterable (even with unhashable elements, optional order preservation)

103 0 0

drop-duplicates-nested-list

Drops duplicates from nested list

93 0 0

duplicateindexer

Find duplicates in multiple lists and return their indices and values.

90 0 0

listset

remove duplicates from lists

82 0 0

dupe-eraser

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

75 13 0

stridesduplicatefinder

Calculate overlapping values between two arrays and return the results as a DataFrame

59 0 0

searchdups

Search for duplicate files

35 0 0