Slurm Python Packages

submitit

Python 3.8+ toolbox for submitting jobs to Slurm

skypilot

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

clusterscope

Clusterscope is a CLI and python library to extract information from HPC Clusters and Jobs.

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

slurm-usage

🖥 Command to list the current cluster usage per user

skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

hydra-submitit-launcher

Python 3.8+ toolbox for submitting jobs to Slurm

pipefunc

Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪

dstack

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

executorlib

Up-scale python functions for high-performance computing (HPC)

toil

A WDL, CWL and Python API supporting easy-to-use workflow engine. It is scalable, efficient and cross-platform (Linux/macOS).

foamlib

✨ A modern Python package for interacting with OpenFOAM

pysqa

Simple HPC queuing system adapter for Python on based jinja templates to automate the submission script creation.

pympipool

Up-scale python functions for high-performance computing (HPC)

adaptive-scheduler

Run many functions (adaptively) on many cores (>10k-100k) using mpi4py.futures, ipyparallel, loky, or dask-mpi. :tada:

dpdispatcher

generate HPC scheduler systems jobs input scripts and submit these scripts to HPC systems and poke until they finish

trainy-skypilot-nightly

SkyPilot: An intercloud broker for the clouds

qs-dmss

QS-DMSS Studio: evidence-first simulation lab for reproducible runs, evidence bundles, campaign comparison, and research-object exports.

slurminade

A decorator-based slurm runner.

biomero

BIOMERO.analyzer - A python library for easy connecting between OMERO (jobs) and a Slurm cluster

slurmutils

Utilities and APIs for interfacing with the Slurm workload manager ⚙️🔌

codeflare-torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

benchbuild

Empirical-Research Toolkit