Luigi Python Packages

luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.

hep-flare

FLARE : FCCee b2Luigi Automated Reconstruction And Event processing

luigi-tools

Extra tools to work with the Luigi library

gluish

Utils around luigi.

sciluigi

A light-weight wrapper library around Spotify's Luigi workflow library to make writing scientific workflows more fluent, flexible and modular

jupyter-paperboy

A web frontend for scheduling Jupyter notebook reports

pipecutter

pipecutter provides a few tools for luigi such that it works better with data science libraries and environments such as pandas, scikit-learn, and Jupyter notebooks.

kubeluigi

Run Luigi tasks as Jobs on K8s - An updated Kubernetes contrib

luigi-soft-failures

Allow Luigi tasks to fail softly, without cancelling downstream tasks.

voxceleb-luigi

Luigi pipeline to download VoxCeleb(2) audio from YouTube and extract speaker segments