Cluster Python Packages

hdijupyterutils

Jupyter magics and kernels for working with remote Spark clusters

autovizwidget

Jupyter magics and kernels for working with remote Spark clusters

skein

A tool and library for easily deploying applications on Apache YARN

sparkmagic

Jupyter magics and kernels for working with remote Spark clusters

openquake-engine

OpenQuake Engine: a software for Seismic Hazard and Risk Analysis

tensorflowonspark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

pepperize-cdk-serverless-cluster-from-snapshot

Deprecated: Use https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_rds.ServerlessClusterFromSnapshot.html

casty

A minimalist, type-safe actor framework for Python 3.12+ with built-in distributed clustering

aioredis-cluster

Redis Cluster support extension for aioredis

srunx

A modern Python library for SLURM workload manager integration with workflow orchestration capabilities.

bexhoma

This python tool helps managing DBMS benchmarking experiments in a Kubernetes-based HPC cluster environment. It enables users to configure hardware / software setups for easily repeating tests over varying configurations.

classicml

An easy-to-use ML framework

privatecloud

PrivateCloud: one-command installer for a Kubernetes-based private cloud (k3s + monitoring + ingress + storage).

libgossip

A C++17 implementation of the Gossip protocol, designed for decentralized distributed systems.

rediscluster

a Python interface to a Cluster of Redis key-value store

hipart

A hierarchical divisive clustering toolbox

sonar

Tool to profile usage of HPC resources by regularly probing processes.

gap-stat

Dynamically get the suggested clusters in the data for unsupervised learning.

classicml-python

An easy-to-use ML framework

laniakea

Laniakea is a utility for managing instances at various cloud providers and aids in setting up a fuzzing cluster.

clustering-benchmarks

A Framework for Benchmarking Clustering Algorithms

arvados-pam

Arvados PAM module

pixel-clusterizer

A fast, generic, and easy to use clusterizer to cluster hits of a pixel matrix in Python.

slurmgen

SlurmGen - Simple Manager for Slurm Jobs