Scala Python Packages

pyspark

Apache Spark - A unified analytics engine for large-scale data processing

papermill

📚 Parameterize, execute, and analyze notebooks

dbl-tempo

API for manipulating time series on top of Apache Spark: lagged time values, rolling statistics (mean, avg, sum, count, etc), AS OF joins, downsampling, and interpolation

pyspark-client

Apache Spark - A unified analytics engine for large-scale data processing

apache-sedona

A cluster computing framework for processing large-scale geospatial data

mleap

MLeap: Deploy ML Pipelines to Production

sparkmeasure

This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simplifies collecting, aggregating, and exporting Spark task/stage metrics, and is designed for practical use by developers and data engineers in interactive analysis, testing, and production monitoring workflows.

synapseml

Simple and Distributed Machine Learning

tree-sitter-scala

Scala grammar for tree-sitter

pyspark-hnsw

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

pyspark-extension

A library that provides useful extensions to Apache Spark and PySpark.

h2o-pysparkling-3-1

Sparkling Water provides H2O functionality inside Spark cluster

sagemaker-pyspark

A Spark library for Amazon SageMaker.

pantsbuild-pants

The Pants Build System

h2o-pysparkling-3-4

Sparkling Water provides H2O functionality inside Spark cluster

h2o-pysparkling-3-5

Sparkling Water provides H2O functionality inside Spark cluster

hsfs

Python - Java/Scala API for the Hopsworks feature store

h2o-pysparkling-3-3

Sparkling Water provides H2O functionality inside Spark cluster

tensorflowonspark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

h2o-pysparkling-2-4

Sparkling Water provides H2O functionality inside Spark cluster

pantsbuild-pants-testutil

Test support for writing Pants plugins.

geospark

A cluster computing framework for processing large-scale geospatial data

pyspark-connect

Apache Spark - A unified analytics engine for large-scale data processing

spylon

Utilities to work with Scala/Java code with py4j