PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Distributed Computing Python Packages

Python packages with the GitHub topic distributed-computing. Sorted by relevance, with stars and monthly downloads.
dask
distributed

A distributed task scheduler for Dask

8M 2K 757
fugue-project
fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

2.5M 2K 100
pytorch
torchrl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

1.6M 3K 455
fugue-project
fugue-sql-antlr

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

978K 2K 100
Eventual-Inc
daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

829K 5K 474
PennyLaneAI
pennylane-lightning

The Lightning plugin ecosystem provides fast quantum state-vector and tensor network simulators written in C++ for use with PennyLane.

748K 136 54
google
ydf

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

555K 656 80
CamDavidsonPilon
tdigest

t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark

316K 407 53
dask
dask-cloudprovider

Cloud provider cluster managers for Dask. Supports AWS, Google Cloud Azure and more...

219K 147 116
DIRACGrid
dirac

DIRAC Grid

132K 121 186
xorbitsai
xoscar

Python actor framework for heterogeneous computing.

45K 171 30
pytorch
torchrl-nightly

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

38K 3K 455
lakehq
pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

28K 3K 146
ThalesGroup
agi-env

Open-source platform for reproducible AI/ML workflows, from local experimentation to distributed workers and long-lived services.

26K 15 1
lablup
backend-ai-manager

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

26K 638 174
lablup
backend-ai-agent

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

24K 638 174
ThalesGroup
agi-node

Open-source platform for reproducible AI/ML workflows, from local experimentation to distributed workers and long-lived services.

24K 15 1
lablup
backend-ai-common

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

23K 638 174
lablup
backend-ai-client

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

23K 638 174
Attumm
redis-dict

Python dictionary with Redis as backend, built for large datasets. Simplifies Redis operations for large-scale and distributed systems. Supports various data types, namespacing, pipelining, and expiration.

22K 76 13
catalyst-team
catalyst

Accelerated deep learning R&D

22K 3K 400
ThalesGroup
agi-cluster

Open-source platform for reproducible AI/ML workflows, from local experimentation to distributed workers and long-lived services.

21K 15 1
SneaksAndData
spark-utils

Comfy Utilities for Spark Job Authoring

21K 6 0
SneaksAndData
hadoop-fs-wrapper

Python Wrappers for Hadoop FileSystem

21K 4 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery