PyRank

Pipelines Python Packages

Python packages with the GitHub topic pipelines. Sorted by monthly downloads, with stars and forks.
pytorch
torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

225K 428 156
zenml-io
zenml

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

201K 6K 663
meltano
meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

178K 3K 273
AgnostiqHQ
covalent

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

61K 868 114
meta-pytorch
torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

58K 427 156
pepkit
pipestat

Pipeline results reporting package

47K 4 2
vish288
mcp-gitlab

MCP server for the GitLab REST API — 91 tools, 7 resources, 6 prompts for projects, MRs, draft reviews, repository files, pipelines, CI/CD, approvals and issues. MCP 2026-07-28 (MCP 2.0); GitLab.com and self-managed; read-only mode, OAuth 2.1 and local sign-in.

31K 6 3
polyaxon
polyaxon

AI Infra / AI Orchestration / AI Control Plane

27K 4K 330
rsgalloway
pyseq

PySeq is a Python library for detecting, parsing, and formatting numbered file sequences

20K 127 37
mage-ai
mage-ai

🧙 Build, run, and manage data pipelines for integrating and transforming data.

19K 9K 991
mmourafiq
vents

Open source connections catalog, integrations, tools, alerting, and notifications library.

19K 31 4
calkit
calkit-python

All stages of research in one place: agile, verifiable, reproducible.

16K 60 16
ecorkran
squadron-ai

user-configurable pipeline orchestration. decompose, review, automate — every level from broad architecture to precise code, your choice of models, structured artifacts human and ai readable. pairs with context-forge. you're in control.

16K 5 2
pypyr
pypyr

pypyr task-runner cli & api for automation pipelines. Automate anything by combining commands, different scripts in different languages & applications into one pipeline process.

13K 645 30
pipefunc
pipefunc

Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪

13K 482 20
ASEM000
pytreeclass

Visualize, create, and operate on pytrees in the most intuitive way possible.

10K 48 3
lsst
lsst-pex-config

Configuration interface and history-tracking for LSST Data Management.

10K 0 11
mabel-dev
mabel

😊 mabel is a platform for authoring data processing systems.

10K 7 3
lsst
lsst-pipe-base

LSST Data Management: base classes for data processing tasks

9K 12 11
tenzir
tenzir

Tenzir is the data pipeline engine for security teams.

9K 762 110
lsst
lsst-ctrl-bps

A PipelineTask execution framework for multi-node processing for the LSST Batch Production Service (BPS).

8K 5 6
lsst
lsst-ctrl-mpexec

Execution framework for PipelineTask

8K 2 3
zenml-io
zenml-nightly

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

8K 6K 663
XiaokaiCui
mlpipelineholder

MLPipelineHolder is a straightforward framework for reproducible machine-learning experimentation in Jupyter notebooks.

8K 0 0