Interpretability Python Packages

shap

A game theoretic approach to explain the output of any machine learning model.

interpret-core

Fit interpretable models. Explain blackbox machine learning.

tensorflow-decision-forests

A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models in Keras.

ydf

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

captum

Model interpretability and understanding for PyTorch

interpret

Fit interpretable models. Explain blackbox machine learning.

aplr

APLR builds predictive, interpretable regression and classification models using Automatic Piecewise Linear Regression. It often rivals tree-based methods in predictive accuracy while offering smoother and interpretable predictions.

grad-cam

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

raiutils

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

imodels

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

nnsight

The nnsight package enables interpreting and manipulating the internals of deep learned models.

dalex

moDel Agnostic Language for Exploration and eXplanation

alibi

Algorithms for explaining machine learning models

shapiq

Shapley Interactions and Shapley Values for Machine Learning

fasttreeshap

Fast SHAP value computation for interpreting tree-based models

etzchaim

A diagnosable brain for your LLM. Cognitive architecture in the SOAR/ACT-R/CLARION/LIDA lineage, for the LLM era. Apache 2.0.

erroranalysis

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

responsibleai

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

sage-importance

For calculating global feature importance using Shapley values.

torchcam

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

aether-core

Belief substrate for AI systems — persistent, contradiction-aware trust state that outlives any single model. The model is the mouth, the substrate is the self.

pyvene

Stanford NLP Python library for understanding and improving PyTorch models via interventions

raiwidgets

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

gamma-facet

Human-explainable AI.