Data Pipeline Python Packages

flupy

Fluent data pipelines for python and your shell

elementary-data

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

doit

CLI task management & automation tool

airbyte-source-declarative-manifest

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

gusty

Making DAG construction easier

airbyte-source-facebook-marketing

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbyte-source-s3

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

file-observer

Deterministic file observation for pipelines — one read-only pass over a directory emits a reproducible JSON manifest of every file's type, metadata, structure, and provenance.

airbyte-source-google-ads

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbyte-source-github

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

augraphy

Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes

dagster-rocky

A SQL transformation engine that type-checks your whole pipeline and catches breaking changes before they run — branches, replay, column-level lineage, compile-time contracts, per-model cost. Adapters: Databricks, Snowflake, BigQuery, DuckDB. Single static Rust binary. Apache 2.0.

airbyte-source-salesforce

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

cli-market-world

Cost-of-Living OS for LatAm — market optimize: basket compare, TCO, substitutes, action links. 80 retailers, 40 verified, 8 countries. PyPI: cli-market-world.

airbyte-source-shopify

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbyte-source-faker

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbyte-source-google-sheets

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

batchit

Batch any Python iterator by count, elapsed time, or both. Zero dependencies.

piperider-nightly

Code review for data in dbt

airbyte-source-gcs

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbyte-source-google-drive

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

ematix-flow

Move data between databases, files, and streams from Python.

airbyte-source-zendesk-support

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.