Data Engineering Pipeline Python Packages

quickstart-vdk

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-core

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-jupyterlab-extension

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

wxdata

A Python package of end-to-end weather data clients & raw data clients with VPN/Proxy Server support, data processors that decode variable keys from GRIB format into a plain-language format & various tools for assisting Python automated workflows, querying meteorological datasets and filling gaps in meteorological data.

vdk-control-cli

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

xontrib-pipeliner

Let your pipe lines flow thru the Python code in Xonsh shell.

marshmallow-pyspark

Marshmallow serializer integration with pyspark

vdk-impala

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

airflow-provider-vdk

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-trino

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-heartbeat

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-server

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-plugin-control-cli

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-ingest-http

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-test-utils

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

pybro-cli

yo, it's ya boy, pybro! 😎 | a personal collection of python hacks for 24.04 debian

vdk-huggingface

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

plotsim

Plotsim generates multi-table synthetic datasets with behavioral trajectories, correlations, and causal lags. Config-driven, deterministic, no real data required.

vdk-control-api-auth

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

tiny-blocks

Tiny Blocks to build large and complex data pipelines!

vdk-kerberos-auth

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-oracle

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

vdk-postgres

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

hydrastream

Concurrent HTTP downloader with in-memory stream reordering (curl_cffi + uvloop).