Mechanistic Interpretability Python Packages

pyvene

Stanford NLP Python library for understanding and improving PyTorch models via interventions

llmoji

Making agents cuter

nnterp

Unified access to Large Language Model modules using NNsight

openinterp

openinterp — Python SDK + CLI. FabricationGuard hallucination probe + ProbeBench leaderboard + Atlas search + Trace generation. pip install openinterp

carl-studio

Coherence-Aware Reinforcement Learning (CARL) - breakthrough LLM post-training and test-time training paradigm. carl builds the world's most advanced and intelligent agent systems that are a step change over current gen agents

steering-vectors

Steering vectors for transformer language models in Pytorch / Huggingface

glassbox-mech-interp

Open-source EU AI Act Annex IV compliance toolkit. Mechanistic interpretability + circuit discovery for transformers. One function call generates a court-ready evidence package

mlx-lens

Mechanistic interpretability on Apple Silicon: steering vectors, residual capture, and SAE analysis for MLX models

softmax-linear-unit

An implementation of softmax linear unit (solu) in PyTorch

carl-core

Coherence-Aware Reinforcement Learning (CARL) - breakthrough LLM post-training and test-time training paradigm. carl builds the world's most advanced and intelligent agent systems that are a step change over current gen agents

reptimeline

Track how discrete representations evolve during neural network training — lifecycle events, phase transitions, ontology discovery, and causal verification

codebook-features

Sparse and discrete interpretability tool for neural networks

mlxlmprobe

Universal probing and interpretability tool for MLX language models on Apple Silicon

graphpatch

graphpatch is a library for activation patching on PyTorch neural network models.

neuro-scan

LLM Neuroanatomy Explorer — map what each transformer layer does

glassbox-mcp

Open-source EU AI Act Annex IV compliance toolkit. Mechanistic interpretability + circuit discovery for transformers. One function call generates a court-ready evidence package

mechreward

Mechanistic interpretability as reward signal for RL training of LLMs — SAE features + GRPO + anti-Goodhart framework

arrakis-mi

Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.

reprobe

Linear probes and activation steering for transformer models

todacomm

Topological Data Analysis Comparison of Multiple Models - A framework for characterizing transformer representations using persistent homology