Data Lake Python Packages

dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

lakefs-sdk

lakeFS - Data version control for your data lake | Git for data

lakefs

lakeFS - Data version control for your data lake | Git for data

lakefs-client

lakeFS - Data version control for your data lake | Git for data

cnequity

中国 A 股数据基础设施。55个日更数据集：股票行情、基本面、期货数据、资金面、公告事件、指数行业、宏观与风险。行级溯源、PIT 语义、复权与历史成分内置，MCP 原生。自托管，零注册、零 API Token

aligned

The DBT of ML, as Aligned describes data dependencies in ML systems, and reduce technical data debt

nodestream

A Declarative framework for Building, Maintaining, and Analyzing Graph Data

dlt-core

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

omniload

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

dlt-cratedb

dlt destination adapter for CrateDB

timeseries-table-format

Rust-native time-series table format with gap/overlap tracking and SQL queries

dlt-filesystem

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

ashare-lake

中国 A 股数据基础设施。55个日更数据集：股票行情、基本面、期货数据、资金面、公告事件、指数行业、宏观与风险。行级溯源、PIT 语义、复权与历史成分内置，MCP 原生。自托管，零注册、零 API Token

irisdb

Ship the decoder with the data. Self-decoding datasets in Rust, with WebAssembly decoders that any engine can run.

lakexpress-mcp

MCP server for LakeXpress

etlx-wrapper

ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.

s4fs

GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% with no app changes (Rust, nvCOMP, zstd).

s4-codec

GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% with no app changes (Rust, nvCOMP, zstd).

virtual-parquet

🗜️Compressing Parquet files using functions (TRL @NeurIPS'24, EDBT Best Demo'25)

vulcan-sql

Data API Framework for AI Agents and Data Apps

dlt-dataops

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

lakefs-sdk-async

lakeFS - Data version control for your data lake | Git for data

ztract

Extract mainframe EBCDIC data using COBOL copybooks. Zero MIPS. Pure Python + Cobrix engine.