Feature Attribution Python Packages

captum

Model interpretability and understanding for PyTorch

shapiq

Shapley Interactions and Shapley Values for Machine Learning

zennit

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks using attribution methods like LRP.

treeig

Exact Integrated Gradients for tree ensembles, including XGBoost, LightGBM, Random Forests, and Gradient Boosting. Computes fast, exact feature attributions for tree models without numerical gradients, numerical integration, Monte Carlo sampling, or path approximations.

unifiedig

Provides a unified approach to Integrated Gradients: a common API for many models, include tree-based models; and common baseline semantics with prediction-neutral baseline distributions as the default.

chessplainer

Explain chess positions with SHAP: which pieces make Stockfish think a position is winning, drawn or lost.

omicau

Omics Audit: reproducible, leakage-safe multi-omics data-audit and fusion-benchmarking CLI.

cub-tools

A set of notebooks as a guide to the process of fine-grained image classification of birds species, using PyTorch based deep neural networks.

thermostat-datasets

Collection of NLP model explanations and accompanying analysis tools

fpde

Open-source Python implementation of Feature Prototype Direction Explainer (FPDE).

torchxai-explain

Explainability and saliency maps for PyTorch vision models — GradCAM, EigenCAM, Attention Rollout for CNNs, Vision Transformers (ViT), CLIP, YOLO, DETR, DINO. One-line API.

ct-master-build

Model interpretability and understanding for PyTorch