Spark Connect Python Packages

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

icegraph-client

Interactive Apache Iceberg debugger for exploring production tables and tracing metadata evolution through AI and real time graph visualization.

adbc-driver-spark

Apache Arrow ADBC driver for Apache Spark Connect (Go core, C-ABI shared library, Python DBAPI). Arrow-native, multi-language.

data-hub-gandalf

SCD (Types 0-6), upsert, surrogate key, checksum, and guarded Delta merge library for Databricks over Spark Connect — you shall not pass... bad data.

pyspark-client-rust

Apache Spark Connect Client for Rust (Rust core + Python wrapper)

sparglim

Sparglim✨ makes PySpark App Configurable and Deploy Spark Connect Server Easier!

spark-connect-proxy

A reverse proxy server which allows secure connectivity to a Spark Connect server