Bigdata Python Packages

fastwarc

A robust web archive analytics toolkit

resiliparse

A robust web archive analytics toolkit

uproot

ROOT I/O in pure Python and NumPy.

pybloom-live

Scalable Bloom Filter implemented in Python

uproot3

ROOT I/O in pure Python and NumPy.

cuallee

Possibly the fastest DataFrame-agnostic quality check library in town.

timehash

An algorithm for creating user configurable, variable-precision sliding windows of time. Useful for binning time values in large collections of data.

legend-pydataobj

LEGEND Python Data Objects

uproot4

ROOT I/O in pure Python and NumPy.

optimuspyspark

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

rayforce-py

Python bindings for RayforceDB

pysparkgateway

Connect to remote Spark clusters seamlessly.

athenacli

AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

visualpython

Visual Python is a GUI-based Python code generator, developed on the Jupyter Notebook as an extension.

airavata-django-portal-sdk

The Airavata Django Portal SDK is a library that makes it easier to develop Airavata Django Portal customizations.

quantms-utils

Python scripts and helpers for the quantMS workflow

jupyterlab-visualpython

GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.

databend

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

bigartm

Fast topic modeling platform

hadeploy

An Hadoop Application deployment tool

airavata-jupyter-magic

Jupyter magics for running notebook cells on remote HPC resources. Powered by Apache Airavata.

dpark

Python clone of Spark, a MapReduce alike framework in Python

arvados-pam

Arvados PAM module

anovos

An Open Source tool for Feature Engineering in Machine Learning