Replicability Python Packages

reverse-argparse

A Python library to determine what exactly the user ran at the command line, along with default values, and any transformations that happened after parsing arguments.

moptipy

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

boba

Author and execute multiverse analysis

staged-script

A Python package enabling the development of robust automation scripts that are subdivided into stages.

reproducible

Keep track of your results

shell-logger-sandialabs

A tool for keeping track of Python's interactions with the shell.

repro-eval

A Python Interface to Reproducibility Measures of System-Oriented IR Experiments

replikant

A flexible toolkit to enable replicable subjective evaluations

ymdl

A framework-agnostic deep learning package and experiment manager