Bias Detection Python Packages

aif360

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

dataeval

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

dataeval-plots

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

model-diagnostics

Tools for diagnostics and assessment of (machine learning) models

langfair

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

judicator

Who evaluates the evaluator? Judicator audits LLM-as-a-Judge systems for 7 documented bias types. Zero config. Works with any LLM.

trustlens

Open-source Python library for evaluating ML model reliability beyond accuracy — with calibration, failure, and fairness diagnostics for informed deployment decisions.

ai-safety-tester

LLM security testing framework with CVE-style severity scoring and multi-model benchmarking

daml

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

optipfair

Structured pruning and bias visualization for Large Language Models. Tools for LLM optimization and fairness analysis.

fairsense-agentix

An agentic fairness and AI-risk analysis platform: bias detection for text, images, and datasets.

wefe

The Word Embedding Fairness Evaluation Framework

rho-eval

Behavioral auditing toolkit for LLMs — audit any model across 8 dimensions (factual, toxicity, bias, sycophancy, reasoning, refusal, deception, over-refusal) using teacher-forced confidence probes.

knowledge-fidelity

Compress LLMs while auditing whether they still know truth vs myths. SVD compression + false-belief detection in one toolkit.

ai-fairness-toolkit

A comprehensive toolkit for evaluating and improving AI model fairness and explainability

trusteval-ai

Enterprise LLM Evaluation & Responsible AI Framework — Benchmark bias, hallucination, PII leakage, and toxicity across Healthcare, BFSI, Retail & Legal industries. Supports OpenAI, Anthropic, Gemini & HuggingFace. Python SDK + CLI + Web Dashboard. 191 tests. Compliance-ready reports.

aiscan

Scan your AI/ML models for problems before you put them into production.

seatbelt

Responsible AI auditing for LLMs and SLMs

unbias-plus

Python package that finds biased language in text, explains why, suggests neutral wording, and rewrites the whole text. Use from CLI, API, or Python.

the-fairly-project

Bias detection Toolkit: Chrome Extension, Python Package, SOTA research paper docs.

biasx

Python library detecting and explaining gender bias in face classification models

aif360-fork2

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

whis-ethical-ai

Ethical AI Validator detects bias and assesses fairness in AI models with statistical parity analysis, real-time monitoring, and automated GDPR/AI Act compliance reporting. Python 2.7+ compatible.

phenoqc

PhenoQC is a lightweight, efficient, and user-friendly toolkit designed to perform comprehensive quality control (QC) on phenotypic datasets. It ensures that data adheres to standardized formats, maintains consistency, and is harmonized with recognized ontologies.