Batch Processing Python Packages

optbinning

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

multidim-indexing

Batch multidimensional indexing for pytorch

fastuuid7

Fork-safe, CSPRNG-backed UUIDv7 for Python 3.9-3.14 with typed scalar and native batch APIs.

local-upscaler

Free, private photo editor that runs on your own machine: AI upscale, colorize, remove backgrounds and objects, color & light, film effects, sharpen, depth-of-field blur, crop, watermark, strip GPS metadata, size budgets, video, PDF, Steam tiles, and batch a folder with saved recipes.

ondine

Batch-process tabular data with any LLM. pandas & Polars in, structured columns out. Checkpointing, budget limits, and 100+ providers built in.

async-batcher

A service to batch the http requests.

calc-flow-python

Rust-native micro-batch and streaming calculation engine: typed Apache Arrow graphs compiled and executed with DataFusion, checkpoint/recovery, PyO3 Python bindings, and a local FastAPI/React Studio.

codarascan

Open-source Python barcode and QR code detector, localizer, and decoder for images and PDFs. Offline batch document scanning, multi-page processing, 40 selectable 1D/2D formats, ZXing-C++, and CLI.

async-batch-llm

Provider-agnostic framework for high-throughput LLM processing with async workers, automatic retries, rate limiting, and intelligent validation recovery.

reqstorm

Send thousands of HTTP requests from Python or the command line: per-host rate limits, retries with backoff, pagination, progress and logs, results straight to JSONL, CSV, SQLite, PostgreSQL or MySQL.

cicerone-recommender

Batch recommender (rectools + LightFM) with serve API, policies, and AutoML

sheetrender

Spreadsheet + HTML template in, a PDF per row out. The batch rendering engine behind sheetrender.com, as a standalone MIT library and CLI.

rubedo

Content-addressed caching and run history for Python batch pipelines — built for steps you can't afford to re-run

batch-img

Batch processing image files by utilizing **Pillow / PIL** library.

btx-skill-jev-judge

Claude Code plugin and Python CLI: hand a judgment repeated over many items to TypeSafe's Jev, with redaction, rate limiting, a pilot and a flat-question check.

pyffmpegcore

Safe, explainable FFmpeg automation for Python, CLI, Node.js, and CI: preview plans, verify outputs, keep privacy-conscious receipts.

cheminformant

⚗️ An all-in-one solution for chemical property retrieval from PubChem.

napari-tmidas

Automated batch processing of cell and tissue microscopy image data in Napari

batchstats

Python package for efficient, online statistical computations on streaming or large-scale data

coalestra

Dependency-free async Python library for building consistent, traceable snapshots from multiple read-only sources with caching, batching, single-flight deduplication, retries, circuit breakers, and sync support.

batchlane

Submit async batch jobs to any LLM provider's discount lane, or refuse honestly

pixopt

Fast Python image optimizer for the web. Resize, compress, convert formats, generate responsive srcset images, extract lazy-loading placeholders, and detect the optimal format automatically. CLI + library.

chiaroscuro-forge

Renaissance-inspired image enhancement and quality assessment for perceptual processing, color preservation, and reproducible pipelines