Lakehouse Python Packages

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

pydoris

Apache Doris is a real-time analytics and hybrid search database for AI agents.

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

apache-polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg

litelink

An embedded Iceberg storage engine for append-only data

pydoris-custom

Apache Doris is a real-time analytics and hybrid search database for AI agents.

qod-cli

Production-grade SQL gateway in front of DuckDB Quack + DuckLake + Iceberg. Multi-tenant pools, pluggable auth (DB/JWT/OIDC), table-level ACLs, role-aware routing, and a live admin console

dbt-fabric-samdebruyn

Former fork combining dbt-fabric and dbt-fabricspark

open-lakehouse-contract

Open Lakehouse Contract (OLC) is an open YAML specification for declaring a data product's schema, quality rules, transformations, lineage, ownership, service levels (SLAs and SLOs) and materialisation.

ailake

AI-Lake Format: self-contained Parquet + HNSW index — unifying tabular data, embeddings, and vector search in a single Iceberg-compatible file

dbt-doris

Apache Doris is a real-time analytics and hybrid search database for AI agents.

duckle

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.

apache-gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

pyfluss

Apache Fluss is a streaming storage built for real-time analytics.

lakebench

A multi-modal Python library for benchmarking lakehouse engines and ELT scenarios, supporting both industry-standard and novel benchmarks.

ftm-lakehouse

Data standard and archive storage for structured FollowTheMoney data, leaked data, private and public document collections.

lakehouse-plumber

The Metadata Driven framework for Databricks Lakeflow Declarative Pipelines (formerly Delta Live Tables). Metadata framework that generates production ready Pyspark code for Lakeflow Declarative Pipelines

ibm-watsonxdata-mcp-server

Model Context Protocol (MCP) server for IBM watsonx.data - enables AI assistants to query and explore lakehouse data Resources

erpl-rev

Standalone C++ SAP RFC server bridging ABAP RFC into DuckDB (the inverse of erpl)

ytsaurus-spyt

YTsaurus is a scalable and fault-tolerant open-source big data platform.

akko-mcp-trino

Read-only MCP server for Trino: six tools, writes refused on the SQL tree, one configured Trino identity, bounded rows and time. Any MCP host, any model.

lht

Bring Your Own Data Warehouse: open-source Salesforce ↔ Snowflake sync, reverse ETL and record merge using the Salesforce Bulk API 2.0.

lakehouse-engine

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

timeseries-table-format

Rust-native time-series table format with gap/overlap tracking and SQL queries