Bigdata Python Packages

fastwarc

A robust web archive analytics toolkit

resiliparse

A robust web archive analytics toolkit

uproot

ROOT I/O in pure Python and NumPy.

uproot3

ROOT I/O in pure Python and NumPy.

pybloom-live

Scalable Bloom Filter implemented in Python

cuallee

Possibly the fastest DataFrame-agnostic quality check library in town.

timehash

An algorithm for creating user configurable, variable-precision sliding windows of time. Useful for binning time values in large collections of data.

rayforce-py

High-performance and powerful Python DataFrame library built on top of RayforceDB

legend-pydataobj

LEGEND Python Data Objects

omni-json-db

A high-performance KV JSON database for Python with Time-Travel, concurrency and compression. The perfect NoSQL alternative to SQLite/TinyDB for effortlessly managing GB-scale data.

uproot4

ROOT I/O in pure Python and NumPy.

optimuspyspark

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

athenacli

AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

airavata-django-portal-sdk

Apache Airavata Django Portal SDK

databend

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

visualpython

GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.

jupyterlab-visualpython

GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.

pysparkgateway

Connect to remote Spark clusters seamlessly.

grapetree

GrapeTree is a fully interactive, tree visualization program, which supports facile manipulations of both tree layout and metadata. Click the first link to launch: https://achtman-lab.github.io/GrapeTree/MSTree_holder.html

quantms-utils

A python library with scripts and helpers classes for quantms workflow

datacatalog-custom-model-manager

Python package to load user-specified metadata models into Google Cloud Data Catalog, comprising Custom Entries, Tag Templates, and Tags

bigartm

Fast topic modeling platform

datacatalog-util

A Python package to centralize some Google Cloud Data Catalog scripts, this repo contains commands like bulk CSV operations that help leverage Data Catalog features.

django-mass-migration

Django app for managing long-running data operations on large and/or schemaless databases