Robustness Python Packages

holisticai

This is an open-source tool to assess and improve the trustworthiness of AI systems.

auto-lirpa

auto_LiRPA: An Automatic Linear Relaxation based Perturbation Analysis Library for Neural Networks and General Computational Graphs

raitap

Fully integrated pipeline to assess the transparency & robustness of AI models

invarlock

Edit-agnostic robustness reports for model weight edits (quantization, pruning, etc.)

deel-torchlip

Build and train Lipschitz-constrained networks: PyTorch implementation of 1-Lipschitz layers. For TensorFlow/Keras implementation, see https://github.com/deel-ai/deel-lip

adversarial-vision-challenge

NIPS Adversarial Vision Challenge

advertorch

A Toolbox for Adversarial Robustness Research

imagenet-c

Corruption and Perturbation Robustness (ICLR 2019)

recentrifuge

Recentrifuge: robust comparative analysis and contamination removal for metagenomics

deltatau-audit

Time Robustness Audit for RL agents — measures timing reliance, deployment robustness, and stress resilience

bound-propagation

Linear and interval bound propagation in Pytorch with easy-to-use API and GPU support.

robustmodelmaker

Coupling Bootstrap Stability Selection with Leakage-Safe Nested Cross-Validation for Scientific Machine Learning

thetiscore

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

adaptor

Adaptor: Objective-centric Adaptation Framework for Language Models.

neuralsat

A neural network verification tool based on the DPLL(T) SMT Solving algorithm.

gaf-microbatch-pytorch

Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch

byzfl

ByzFL: A Python library for robust federated learning, offering Byzantine-resilient aggregators, attack simulations, and ML pipelines for distributed systems. Compatible with PyTorch and NumPy.

ml-model-quality-analysis

PyPi package to perform quality analyses on ML models.

dspy-security-bench

Measure how DSPy prompt optimization affects the prompt-injection robustness of agentic LLM programs, using AgentDojo's attack suite.

pylandscape

This project propose the loss landscape analysis as effective methodology to understand the robustness against natural perturbation of QNN.

linqit

Extends python's list builtin with fun, robust functionality - .NET's Language Integrated Queries (Linq) and more. Write clean code with powerful syntax.

equitas-benchmark

Corruption-robustness benchmark for hierarchical multi-LLM committees

fiddler-auditor

Fiddler Auditor is a tool to evaluate language models.

perceptual-advex

Code for the ICLR 2021 paper "Perceptual Adversarial Robustness: Defense Against Unseen Threat Models"