Mechanistic Interpretability Python Packages

pyvene

Stanford NLP Python library for understanding and improving PyTorch models via interventions

glassbox-mech-interp

Open-source EU AI Act Annex IV documentation toolkit. Mechanistic interpretability + circuit discovery for transformers. One function call generates a structured, hash-chained evidence package.

nnterp

Unified access to Large Language Model modules using NNsight

steering-vectors

Steering vectors for transformer language models in Pytorch / Huggingface

safety-compass

Detect safety degradation during LLM fine-tuning before it becomes behavioral

llmoji

Making agents cuter

carl-studio

CARL (Coherence-Aware Reinforcement Learning) — information-theoretic reward signals for LLM training via token-level probability distributions

warden-interp

Circuit-level regression testing for AI systems. Catch mechanistic drift that behavioral evals miss.

mlxlmprobe

Universal probing and interpretability tool for MLX language models on Apple Silicon

gavagai

Quantify translation indeterminacy between sparse autoencoder feature dictionaries (Quine × Mechanistic Interpretability).

openinterp

openinterp — Python SDK + CLI. FabricationGuard hallucination probe + ProbeBench leaderboard + Atlas search + Trace generation. pip install openinterp

codebook-features

Sparse and discrete interpretability tool for neural networks

softmax-linear-unit

An implementation of softmax linear unit (solu) in PyTorch

neuro-scan

LLM Neuroanatomy Explorer — map what each transformer layer does

mlx-lens

Mechanistic interpretability on Apple Silicon: steering vectors, residual capture, and SAE analysis for MLX models

reptimeline

Track how discrete representations evolve during training

graphpatch

graphpatch is a library for activation patching on PyTorch neural network models.

recurrentlens

Mechanistic interpretability for State-Space Models: SAEs, feature visualization, and a Hub registry for Mamba/Mamba-2.

carl-core

Coherence math primitives for CARL (Coherence-Aware Reinforcement Learning)

gated-sae-tf

A TensorFlow/Keras Sparse Gated Autoencoder (Gated SAE) for dictionary learning and interpretability.

arrakis-mi

Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.

mechreward

Mechanistic interpretability as reward signal for RL training of LLMs

glassbox-mcp

MCP server for Glassbox — mechanistic interpretability + EU AI Act Annex IV compliance tools for Claude and any MCP-compatible client

todacomm

Topological Data Analysis Comparison of Multiple Models - A framework for characterizing transformer representations using persistent homology