Bigdata Python Packages

avro

Apache Avro is a data serialization system.

fastwarc

A robust web archive analytics toolkit

resiliparse

A robust web archive analytics toolkit

uproot

ROOT I/O in pure Python and NumPy.

pybloom-live

Scalable Bloom Filter implemented in Python

cuallee

Possibly the fastest DataFrame-agnostic quality check library in town.

uproot3

ROOT I/O in pure Python and NumPy.

timehash

An algorithm for creating user configurable, variable-precision sliding windows of time. Useful for binning time values in large collections of data.

rayforce-py

High-performance and powerful Python DataFrame library built on top of RayforceDB

legend-pydataobj

LEGEND Python Data Objects

athenacli

AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

uproot4

ROOT I/O in pure Python and NumPy.

optimuspyspark

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

grapetree

GrapeTree is a fully interactive, tree visualization program, which supports facile manipulations of both tree layout and metadata. Click the first link to launch: https://achtman-lab.github.io/GrapeTree/MSTree_holder.html

databend

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

airavata-django-portal-sdk

Apache Airavata Django Portal SDK

quantms-utils

A python library with scripts and helpers classes for quantms workflow

visualpython

GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.

bigartm

Fast topic modeling platform

jupyterlab-visualpython

GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.

omni-json-db

KV JSON DB + Graph DB with Time-Travel + compression. NoSQL alternative to TinyDB.

django-mass-migration

Django app for managing long-running data operations on large and/or schemaless databases

hadeploy

An Hadoop Application Deployment tool

airavata-jupyter-magic

A general purpose Distributed Systems Framework