Replicability Python Packages

reverse-argparse

A Python library to determine what exactly the user ran at the command line, along with default values, and any transformations that happened after parsing arguments.

moptipy

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

boba

Specifying and executing multiverse analysis

reproducible

Keep track of your results

shell-logger-sandialabs

A Python package that allows you to interact with the shell, while logging various metadata, statistics, and trace information.

staged-script

A Python package enabling the development of robust automation scripts that are subdivided into stages.

repro-eval

A Python Interface to Reproducibility Measures of System-Oriented IR Experiments

replikant

A flexible evaluation platform to enable researchers to conduct replicable subjective evaluation

ymdl

A framework-agnostic deep learning package and experiment manager