Stream Processing Python Packages

mediapipe

Cross-platform, customizable ML solutions for live and streaming media.

faststream

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

river

🌊 Online machine learning in Python

quixstreams

Python Streaming DataFrames for Kafka

sentry-arroyo

A library to build streaming applications that consume from and produce to Kafka.

hdx-python-utilities

A range of helpful utilities for Python developers including streaming tabular data, date parsing, JSON and YAML handling, dictionary and list utilities

distogram

A library to compute histograms on distributed environments, on streaming data

bytewax

Python Stream Processing

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

csp

csp is a high performance reactive stream processing library, written in C++ and Python

beavers

Python stream processing for analytics

mediapipe-nightly

Cross-platform, customizable ML solutions for live and streaming media.

dbt-materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

yasbd-lib

A high-accuracy, multilingual rule-based Sentence Boundary Detector (SBD) with a drop-in adapter for pysbd, delivering faster and more accurate sentence segmentation.

faust

Python Stream Processing

odctl

A curated collection of open source technologies and the odctl CLI for running a local Docker-based data platform to experiment with modern data architecture and MLOps.

gretel-client

The Gretel Python Client allows you to interact with the Gretel REST API.

scrna-matrix

High-performance C++ genomics library and Python extension for fast omics processing, sequence analysis, and stream profiling.

probstructs

Collection of probabilistic data structures - CM-Sketch, ECM-Sketch, exponential histogram

ralph-malph

:gear: Ralph, the ultimate Learning Record Store (and more!) for your learning analytics

csp-gateway

csp-gateway is a framework for building high-performance streaming applications

creme

🌊 Online machine learning in Python

slipstream-async

Slipstream provides a data-flow model to simplify development of stateful streaming applications.

flechtwerk

Truly async Python stream processing with real Kafka transactions for exactly-once delivery, and an MQTT→Kafka bridge that ACKs only after Kafka has the data.