Data Quality Checks Python Packages

traceml

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

datatile

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

pandas-summary

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

cuallee

Possibly the fastest DataFrame-agnostic quality check library in town.

re-data

re_data - fix data issues before your users & CEO would discover them 😊

forge-robotics

Robotics Data Toolkit | Convert between robotics dataset formats (RLDS, LeRobot v2/v3, Zarr, HDF5, Rosbag). Inspect, visualize, and analyze datasets. Works with HuggingFace Hub. Built for OpenVLA, Octo, LeRobot, and Diffusion Policy workflows.

dqops

Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.

weiser-ai

Data Quality made simple.

sqlbucket

Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.

redflag

Safety net for machine learning pipelines. Plays nice with sklearn and pandas.

framepeek

Lightweight, structured exploratory data analysis for pandas DataFrames with overview, column profiling, missing values, duplicates, outliers, correlations, and data quality warnings in one function call.

sumeh

Sumeh — Unified Data Quality Framework Sumeh is a unified data quality validation framework supporting multiple backends (PySpark, Dask, Polars, DuckDB, Pandas) with centralized rule configuration.

grib-check

A tool that validates project-specific conventions of GRIB files

etlx-wrapper

ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.

dqc-toolkit

Quality Checks for Training Data in Machine Learning

data-watchtower

数据质量检查工具, 用于诊断数据的问题

qprofiler

profile tabular datasets, manage automatic validation for new datasets, automatic handling for quality issues.

dbt-dqlens

Data quality for dbt, without writing tests. Auto-generates dbt tests by profiling your database.

validatelite

ValidateLite: A lightweight CLI for database schema validation and data quality checks. Ideal for CI/CD, ETL, and data pipelines.

sql-guard

A small package for data quality rules using Standard SQL

geoqa

GeoQA: Geospatial Data Quality Assessment . one-liner profiling, quality scoring, interactive reports

reqdonkey

Distributed Collection, Local Intelligence - Stop LLMs from hallucinating with Kong in the Loop architecture

hooqu

hooqu is a library built on top of Pandas-like Dataframes for defining "unit tests for data". This is a spiritual port of Apache Deequ to Python

waterfall-logging

a Python package to log (distinct) column counts in a DataFrame, export it as a Markdown table and plot a Waterfall statistics figure.