Lakehouse Python Packages

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

pydoris-custom

Apache Doris is an easy-to-use, high performance and unified analytics database.

pydoris

Apache Doris is an easy-to-use, high performance and unified analytics database.

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

dbt-fabric-samdebruyn

Maintained and extended fork combining dbt-fabric and dbt-fabricspark

sqllocks-spindle

Multi-domain, schema-aware synthetic data generator for Microsoft Fabric. 13 domains, billion-row scale, statistically calibrated. Lakehouse · Warehouse · SQL DB · Eventhouse writers.

lakehouse-engine

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

lakehouse-plumber

The Metadata Driven framework for Databricks Lakeflow Declarative Pipelines (formerly Delta Live Tables). Metadata framework that generates production ready Pyspark code for Lakeflow Declarative Pipelines

dbt-doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

apache-gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

ytsaurus-spyt

YTsaurus is a scalable and fault-tolerant open-source big data platform.

redpanda-polaris-catalog-python

Apache Polaris, the interoperable, open source catalog for Apache Iceberg

laketower

Oversee your lakehouse

databend

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

datacoolie

Metadata-driven ETL framework for portable data pipelines across Polars, Spark, Fabric, Databricks, and AWS.

lakebench

A multi-modal Python library for benchmarking lakehouse engines and ELT scenarios, supporting both industry-standard and novel benchmarks.

doris-mcp-server

Apache Doris MCP Server

ibm-watsonxdata-mcp-server

Model Context Protocol (MCP) server for IBM watsonx.data - enables AI assistants to query and explore lakehouse data Resources

timeseries-table-format

Rust-native time-series table format with gap/overlap tracking and SQL queries

space-datasets

Unified storage framework for the entire machine learning lifecycle

ftm-lakehouse

Data standard, storage and retrieval for structured and unstructured FollowTheMoney data

apache-polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg

etherealogic-aetheriaforge

Databricks-native intelligent data transformation engine — coherence-scored Bronze/Silver/Gold with entity resolution and temporal reconciliation in a single deployable product.

pyfluss

Rust Client for Apache Fluss (Incubating)