Cluster Python Packages

hdijupyterutils

Jupyter magics and kernels for working with remote Spark clusters

autovizwidget

Jupyter magics and kernels for working with remote Spark clusters

skein

A tool and library for easily deploying applications on Apache YARN

sparkmagic

Jupyter magics and kernels for working with remote Spark clusters

pepperize-cdk-serverless-cluster-from-snapshot

Deprecated: Use https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_rds.ServerlessClusterFromSnapshot.html

openquake-engine

OpenQuake Engine: a software for Seismic Hazard and Risk Analysis

laniakea

Laniakea is a utility for managing instances at various cloud providers and aids in setting up a fuzzing cluster.

casty

A minimalist, type-safe actor framework for Python 3.12+ with built-in distributed clustering

tensorflowonspark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

bexhoma

This python tool helps managing DBMS benchmarking experiments in a Kubernetes-based HPC cluster environment. It enables users to configure hardware / software setups for easily repeating tests over varying configurations.

aioredis-cluster

Redis Cluster support extension for aioredis

classicml

简单易用的经典机器学习框架

srunx

A modern Python library for SLURM workload manager integration with workflow orchestration capabilities.

rediscluster

a Python interface to a Cluster of Redis key-value store

slurmwatch

Is your Slurm job actually using what you asked for? Watch it live.

sonar

Tool to profile usage of HPC resources by regularly probing processes.

gap-stat

Dynamically get the suggested clusters in the data for unsupervised learning.

hipart

Hierarchical divisive clustering algorithm execution, visualization and Interactive visualization.

userdocker

Allow admins to grant restricted docker/nvidia-docker commandline access to users.

slurmpast

Why your Slurm jobs failed, and what to ask for next time.

libgossip

A C++17 implementation of the Gossip protocol, designed for decentralized distributed systems.

slurmgen

SlurmGen - Simple Manager for Slurm Jobs

classicml-python

简单易用的经典机器学习框架

runjob

runjob is a program for managing a group of related jobs running on a compute cluster