Spark Connect Python Packages

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

adbc-driver-spark

Apache Arrow ADBC driver for Apache Spark Connect (Go core, C-ABI shared library, Python DBAPI). Arrow-native, multi-language.

data-hub-gandalf

SCD (Types 0-6), upsert, surrogate key, checksum, and guarded Delta merge library for Databricks over Spark Connect — you shall not pass... bad data.

spark-connect-proxy

A reverse proxy server which allows secure connectivity to a Spark Connect server

sparglim

sparglim