Robustness Python Packages

raitap

Fully integrated pipeline to assess the transparency & robustness of AI models

auto-lirpa

auto_LiRPA: An Automatic Linear Relaxation based Perturbation Analysis Library for Neural Networks and General Computational Graphs

holisticai

This is an open-source tool to assess and improve the trustworthiness of AI systems.

adversarial-vision-challenge

NIPS Adversarial Vision Challenge

recentrifuge

Recentrifuge: robust comparative analysis and contamination removal for metagenomics

bound-propagation

Bound propagation in Pytorch

advertorch

A Toolbox for Adversarial Robustness Research

imagenet-c

Corruption and Perturbation Robustness (ICLR 2019)

invarlock

Edit-agnostic robustness reports for model weight edits (quantization, pruning, etc.)

robustmodelmaker

A reproducible model-building pipeline for small-to-medium scientific datasets

adaptor

ACL 2022: Adaptor: a library to easily adapt a language model to your own task, domain, or custom objective(s).

deel-torchlip

Build and train Lipschitz-constrained networks: PyTorch implementation of 1-Lipschitz layers. For TensorFlow/Keras implementation, see https://github.com/deel-ai/deel-lip

neuralsat

NeuralSAT: A DPLL(T) Framework for Verifying Deep Neural Networks

deltatau-audit

Time Robustness Audit for RL agents — measures timing reliance, deployment robustness, and stress resilience

gaf-microbatch-pytorch

Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch

thetiscore

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

linqit

Extend python lists operations using .NET's LINQ syntax for clean and fast coding.

byzfl

ByzFL Library

promptbench

PromptBench is a powerful tool designed to scrutinize and analyze the interaction of large language models with various prompts. It provides a convenient infrastructure to simulate **black-box** adversarial **prompt attacks** on the models and evaluate their performances.

pylandscape

This project propose the loss landscape analysis as effective methodology to understand the robustness against natural perturbation of QNN.

ml-model-quality-analysis

PyPi package to perform quality analyses on ML models.

fiddler-auditor

Auditing large language models made easy.

agentchaos-sdk

Evaluate agent system robustness through controlled, runtime, non-intrusive LLM API fault injection.

graph-tiger

Python toolbox to evaluate graph vulnerability and robustness (CIKM 2021)