Data Processing Pipelines Python Packages

convtools

convtools is a specialized Python library for dynamic, declarative data transformations with automatic code generation

wxdata

A Python package of end-to-end weather data clients & raw data clients with VPN/PROXY support, data processors that decode variable keys from GRIB format into a plain-language format & various tools for assisting Python automated workflows, querying meteorological datasets and filling gaps in meteorological data.

artifician

Artifician is an event driven framework developed to simplify the process of preparation of the dataset for Artificial Intelligence models.

filter-stations

A secure, unified Python interface for African climate data, integrating TAHMO station data and gridded datasets (IMERG, CHIRPS, ERA5, TAMSAT), and medium-to-seasonal weather models

thepipe

A lightweight, general purpose pipeline framework.

graphbook

Visual AI development framework for training and inference of ML models, scaling pipelines, and automating workflows with Python

graphbook-huggingface

Visual AI development framework for training and inference of ML models, scaling pipelines, and automating workflows with Python

invisible-rabbit

Scalable data pre processing and curation toolkit for LLMs

invisible-unicorn

Scalable data pre processing and curation toolkit for LLMs

lava-ray

Scalable data pre processing and curation toolkit for LLMs