Delta Lake Python Packages

delta-spark

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

deltalake

A native Rust library for Delta Lake, with bindings into Python

delta-sharing

An open protocol for secure data sharing

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

koheesio

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

pydoris

Apache Doris is a real-time analytics and hybrid search database for AI agents.

hops-deltalake

A native Rust library for Delta Lake, with bindings into Python

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

cdk-emrserverless-with-delta-lake

This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you could also launch an EMR notebook via cluster template to check the outcome from the EMR Serverless application.

pydoris-custom

Apache Doris is a real-time analytics and hybrid search database for AI agents.

delta-lake-reader

Read Delta tables without any Spark

dask-deltatable

A Delta Lake reader for Dask

dbt-doris

Apache Doris is a real-time analytics and hybrid search database for AI agents.

ematix-flow

Move data between databases, files, and streams from Python.

beacondb

A high-performance climate data lake query engine supporting subsetting for zarr, netcdf, parquet, arrow ipc, csv, delta and iceberg

lakehouse-engine

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

caland

Small tools for the ugly gaps on Databricks: stevin, lely, leeghwater, caland

deltafunnel

Fast, lightweight Delta Lake to SQL Server loads without Spark or ODBC.

lely

Small tools for the ugly gaps on Databricks: stevin, lely, leeghwater, caland

roapi

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

roapi-http

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

sqllocks-spindle

Multi-domain, schema-aware synthetic data generator for Microsoft Fabric

sqlshell

A powerful SQL shell with GUI interface for data analysis

data-hub-gandalf

SCD (Types 0-6), upsert, surrogate key, checksum, and guarded Delta merge library for Databricks over Spark Connect — you shall not pass... bad data.