Scala Python Packages

pyspark

Apache Spark - A unified analytics engine for large-scale data processing

papermill

📚 Parameterize, execute, and analyze notebooks

dbl-tempo

API for manipulating time series on top of Apache Spark: lagged time values, rolling statistics (mean, avg, sum, count, etc), AS OF joins, downsampling, and interpolation

mleap

MLeap: Deploy ML Pipelines to Production

apache-sedona

A cluster computing framework for processing large-scale geospatial data

synapseml

Simple and Distributed Machine Learning

sparkmeasure

This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simplifies collecting, aggregating, and exporting Spark task/stage metrics, and is designed for practical use by developers and data engineers in interactive analysis, testing, and production monitoring workflows.

pyspark-client

Apache Spark - A unified analytics engine for large-scale data processing

pyspark-hnsw

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

tree-sitter-scala

Scala grammar for tree-sitter

pyspark-extension

A library that provides useful extensions to Apache Spark and PySpark.

h2o-pysparkling-3-1

Sparkling Water provides H2O functionality inside Spark cluster

sagemaker-pyspark

A Spark library for Amazon SageMaker.

h2o-pysparkling-3-5

Sparkling Water provides H2O functionality inside Spark cluster

h2o-pysparkling-3-4

Sparkling Water provides H2O functionality inside Spark cluster

pantsbuild-pants

The Pants Build System

h2o-pysparkling-2-4

Sparkling Water provides H2O functionality inside Spark cluster

h2o-pysparkling-3-3

Sparkling Water provides H2O functionality inside Spark cluster

hsfs

Python - Java/Scala API for the Hopsworks feature store

tensorflowonspark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

pantsbuild-pants-testutil

The Pants Build System

geospark

A cluster computing framework for processing large-scale geospatial data

pyspark-connect

Apache Spark - A unified analytics engine for large-scale data processing

spylon

Utilities to work with Scala/Java code with py4j