Data Curation Python Packages

fiftyone-db

Refine high-quality datasets and visual AI models

fiftyone

Refine high-quality datasets and visual AI models

cleanlab

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

lsst-rucio-register

Routines and commands to add Butler specific information to Rucio metadata

fastdup

fastdup is a powerful, free tool designed to rapidly generate valuable insights from image and video datasets. It helps enhance the quality of both images and labels, while significantly reducing data operation costs, all with unmatched scalability.

weightslab

PyTorch dataset debugger for computer vision — pause training, mine live loss signals to surface mislabels, class imbalance & outliers, then curate your image, video & LiDAR data without restarting

bids-manager

GUI and CLI tool for raw-to-BIDS conversion, curation, metadata editing, and validation for MRI, EEG, MEG, and physiological recordings.

fiftyone-db-ubuntu2204

Refine high-quality datasets and visual AI models

cleanlab-studio

Client interface to Cleanlab Studio

fiftyone-desktop

Refine high-quality datasets and visual AI models

omicsmeta

Automated harmonization of public omics metadata

synrbl

Rebalancing chemical reaction

sliceguard

A library for detecting critical data slices in structured and unstructured data based on features, metadata and model predictions.

selfclean

A holistic self-supervised data cleaning strategy to detect off-topic samples, near duplicates and label errors.

chamanp

CHAMANP is a modular, extensible, and production-grade system for the curated preparation of molecular databases.

cleanlab-cli

Command line interface for all things Cleanlab Studio

cubids

Curation of BIDS (CuBIDS): A sanity-preserving software package for processing BIDS datasets.

example-package-elisno

The standard package for data-centric AI, machine learning with label errors, and automatically finding and fixing dataset issues in Python.

distributed-curator

Partition-aware MinHash LSH deduplication for large-scale text data curation on Apache Spark

docta-ai

Docta.ai

ldcoolp-figshare

Python tool using the Figshare API for data curation

urlgenie

Python package to make URL extraction, generalization, validation, and filtration easy.

fiftyone-db-debian9

Refine high-quality datasets and visual AI models

invisible-rabbit

Scalable data pre processing and curation toolkit for LLMs