Explainability Python Packages

shap

A game theoretic approach to explain the output of any machine learning model.

interpret-core

Fit interpretable models. Explain blackbox machine learning.

interpret

Fit interpretable models. Explain blackbox machine learning.

raiutils

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

wisent

This is an open-source version of the representation engineering framework for stopping harmful outputs or hallucinations on the level of activations. 100% free, self-hosted and open-source.

shapiq

Shapley Interactions and Shapley Values for Machine Learning

erroranalysis

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

responsibleai

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

sage-importance

For calculating global feature importance using Shapley values.

shapash

🔅 Shapash: User-friendly Explainability and Interpretability to Develop Reliable and Transparent Machine Learning Models

bnnr

XAI-driven augmentation & diagnostics for PyTorch vision - find model failures, fix with saliency-guided augmentation (ICD/AICD), prove with auditable reports.

raiwidgets

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

streamlit-shap

streamlit-shap provides a wrapper to display SHAP plots in Streamlit.

tf-keras-vis

Neural network visualization toolkit for tf.keras

acv-dev

ACV is a python library that provides explanations for any machine learning model or data. It gives local rule-based explanations for any model or data and different Shapley Values for tree-based models.

abrain

ES-HyperNEAT Python implementation with C++ computations for NeuroEvolution, Reinforcement Learning and VfMRI

zennit

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks using attribution methods like LRP.

effector

Effector - a Python package for global and regional effect methods

scomp-link

End-to-end ML toolkit automating the complete workflow — data profiling, preprocessing, feature engineering, model selection, training, validation, explainability, drift monitoring, fairness checks, and interactive HTML reporting. Includes a full CLI for zero-code ML pipelines. Python 3.10+.

glassbox-mech-interp

Open-source EU AI Act Annex IV documentation toolkit. Mechanistic interpretability + circuit discovery for transformers. One function call generates a structured, hash-chained evidence package.

codeboarding

Interactive architecture diagrams for codebases

imodelsx

Interpret text data with LLMs (sklearn compatible).

xper

A methodology designed to measure the contribution of the features to the predictive performance of any econometric or machine learning model.

antakia

AntakIA is THE tool to explain an ML model or replace it with a collection of basic explainable models.