Parallelization Python Packages

swifter

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner

gamma-pytools

Foundational tools for BCG X's data science packages.

parmap

Easy to use map and starmap python equivalents

sdaxen-python-utilities

Useful tools for common Python tasks.

bonobo

Extract Transform Load for Python 3.5+

sciris

Fast, flexible tools to simplify scientific Python

dprs

Directed percolation and related models implemented in Rust and wrapped in Python

covalent

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

qc-parallelizer

A Python module for optimally combining and distributing quantum circuits

cova

Covalent Workflow Tool

django-collectfaster

Parallel file copying for Django's collectstatic.

parl

A high-performance distributed training framework for Reinforcement Learning

bluepyparallel

Provides an embarrassingly parallel tool with sql backend.

covalent-ssh-plugin

Covalent SSH Plugin

covalent-awslambda-plugin

Covalent AWS Lambda Executor Plugin

aloisius

Create/Update/Delete AWS CloudFormation stacks in parallel

covalent-ecs-plugin

Covalent ECS Plugin

covalent-awsbatch-plugin

Covalent AWS Batch Plugin

covalent-braket-plugin

Covalent Braket Plugin

covalent-slurm-plugin

Covalent Slurm Plugin

covalent-gcpbatch-plugin

Covalent GCP Batch Plugin

ml-impute

A package for synthetic data generation for imputation using single and multiple imputation methods.

gcn10py

Python interface to high-performance, MPI-Parallelized C binary `gcn10`

pystream-pipeline

Python package to create and manage fast parallelized data processing pipeline for real-time application