Interpretability Python Packages

shap

A game theoretic approach to explain the output of any machine learning model.

interpret-core

Fit interpretable models. Explain blackbox machine learning.

captum

Model interpretability and understanding for PyTorch

aplr

APLR builds predictive, interpretable regression and classification models using Automatic Piecewise Linear Regression. It often rivals tree-based methods in predictive accuracy while offering smoother and interpretable predictions.

interpret

Fit interpretable models. Explain blackbox machine learning.

tensorflow-decision-forests

A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models in Keras.

ydf

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

grad-cam

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

raiutils

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

dalex

moDel Agnostic Language for Exploration and eXplanation

imodels

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

wisent

Monitor and influence AI Brains

shapiq

Shapley Interactions and Shapley Values for Machine Learning

nnsight

The nnsight package enables interpreting and manipulating the internals of deep learned models.

fasttreeshap

Fast SHAP value computation for interpreting tree-based models

erroranalysis

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

trainlens

TrainLens turns training runs into evidence: what changed, what went wrong, and what experiment to run next.

streamlit-shap

streamlit-shap provides a wrapper to display SHAP plots in Streamlit.

responsibleai

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

ydf-tf

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

torchcam

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM, Finer-CAM, LeGrad, RefineCAM)

hugiml-core

High-performance interpretable ML classifier using High Utility Gain patterns (IEEE Access 2024). C++ accelerated, scikit-learn compatible, with EBM-style explanations, adaptive binning, pattern pruning, and deployment tooling for regulated domains.

alibi

Algorithms for explaining machine learning models

shapash

🔅 Shapash: User-friendly Explainability and Interpretability to Develop Reliable and Transparent Machine Learning Models