Robustness Python Packages

gauntlet-robotics

Robustness evaluation harness for robot manipulation policies (VLA): perturbation sweeps, failure clustering, reproducible reports.

holisticai

This is an open-source tool to assess and improve the trustworthiness of AI systems.

invarlock

Verifiable evidence for model releases.

advertorch

A Toolbox for Adversarial Robustness Research

recentrifuge

Recentrifuge: robust comparative analysis and contamination removal for metagenomics

bound-propagation

Linear and interval bound propagation in Pytorch with easy-to-use API and GPU support.

stabilization-uplift

Stabilization of classification ML-models using synthetic data with outliers on Open Data. Metrics on PyPI: pip install stabilization-uplift. NeurIPS 2025 GenAI in Finance Workshop.

adversarial-vision-challenge

NIPS Adversarial Vision Challenge

graph-tiger

Python toolbox to evaluate graph vulnerability and robustness (CIKM 2021)

croma

A distributional robustness margin for pathology foundation models

deel-torchlip

Build and train Lipschitz-constrained networks: PyTorch implementation of 1-Lipschitz layers. For TensorFlow/Keras implementation, see https://github.com/deel-ai/deel-lip

adaptor

ACL 2022: Adaptor: a library to easily adapt a language model to your own task, domain, or custom objective(s).

deltatau-audit

Time Robustness Audit for RL agents — measures timing reliance, deployment robustness, and stress resilience

robustmodelmaker

Coupling Bootstrap Stability Selection with Leakage-Safe Nested Cross-Validation for Scientific Machine Learning

raitap

Fully integrated pipeline to assess the transparency & robustness of AI models

robustpixelmaker

Reproducible, leakage-safe feature selection that works directly on image pixels.

byzfl

ByzFL: A Python library for robust federated learning, offering Byzantine-resilient aggregators, attack simulations, and ML pipelines for distributed systems. Compatible with PyTorch and NumPy.

robustsignalmaker

NaN-aware, leakage-free stability selection for scientific signals and spectra

neuralsat

A neural network verification tool based on the DPLL(T) SMT Solving algorithm.

thetiscore

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

gaf-microbatch-pytorch

Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch

imagenet-c

Corruption and Perturbation Robustness (ICLR 2019)

auto-lirpa

auto_LiRPA: An Automatic Linear Relaxation based Perturbation Analysis Library for Neural Networks and General Computational Graphs

pylandscape

This project propose the loss landscape analysis as effective methodology to understand the robustness against natural perturbation of QNN.