Workflow Management Python Packages

hera

Hera makes Python code easy to orchestrate on Argo Workflows through native Python integrations. It lets you construct and submit your Workflows entirely in Python. ⭐️ Remember to star!

doit

CLI task management & automation tool

snakemake

This is the development home of the workflow management system Snakemake. For general information, see

hera-workflows

Hera makes Python code easy to orchestrate on Argo Workflows through native Python integrations. It lets you construct and submit your Workflows entirely in Python. ⭐️ Remember to star!

dirac

DIRAC Grid

covalent

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

datajoint

Relational Workflows: where database schemas define executable data pipelines.

diraccommon

DIRAC Grid

calkit-python

All stages of research in one place: agile, verifiable, reproducible.

pyiron-base

Core components of the pyiron integrated development environment (IDE) for computational materials science

dataexcept

A comprehensive Python library providing structured, hierarchical exception classes for data science, data engineering, and ML pipeline workflows. Features 40+ specialized exceptions, logging helpers, and production-ready error handling for robust data processing systems.

ck9

All stages of research in one place: agile, verifiable, reproducible.

cylc-flow

Cylc: a general purpose workflow engine with a gift for cycling

stepup

A dynamic Python build tool for reproducible workflows

calk9

All stages of research in one place: agile, verifiable, reproducible.

autosubmit-api

Autosubmit API is a package that consumes the information generated by Autosubmit and serves it as an API.

horus-runtime

Run reproducible scientific workflows anywhere (laptop, remote, or HPC cluster) from a single YAML file. The open-source runtime behind Temple Compute.

tierkreis

Tierkreis Hybrid Workflow Management System

iotaa

It's One Thing After Another: A Simple Workflow Engine

selinon

An advanced distributed task flow management on top of Celery

workflow-canvas

Reproducible computational pipelines: containerized methods, content-addressed caching, run lineage, and a visual pipeline builder. Compiles to Snakemake.

cova

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

checklistfabrik

Python tool that generates interactive HTML checklists from YAML templates. Jinja conditionals, reusable includes, pluggable modules, built-in web server. Ideal for SOPs, deployments & recurring ops.

stepup-queue

Integrate SLURM jobs into a StepUp workflow