Streaming Data Python Packages

smart-open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

river

🌊 Online machine learning in Python

optbinning

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

quixstreams

Python Streaming DataFrames for Kafka

shioaji

Shioaji all new cross platform api for trading ( 跨平台證券交易API )

streamz

Real-time stream processing for python

bytewax

Python Stream Processing

streaming-json-parser

Streaming JSON parser designed to process JSON data incrementally. The primary goal is to handle potentially incomplete JSON data streams, such as those produced by Large Language Models (LLMs), and return the current state of the parsed object at any time.

dbt-materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

scikit-multiflow

A machine learning package for streaming data in Python. The other ancestor of River.

creme

🌊 Online machine learning in Python

datafun-streaming

Shared Python utilities for Kafka, DuckDB, validation, stats, and visualization across streaming data analytics projects.

rdst

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

rxsci

ReactiveX for data science

pandas-streaming

Streaming API for pandas applied to big datasets

streamframe

streamframe is a lightweight engine for computing real-time features over streaming data, with constant-time updates and queries. Written in Rust and available as a Python library.

pysad

Streaming Anomaly Detection Framework in Python (Outlier Detection for Streaming Data)

screamer

Screamingly fast streaming indicators with C++ performance and Python simplicity.

uvatradier

wahoowa

streamdal-protos

Code-Native Data Privacy

slipstream-async

Slipstream provides a data-flow model to simplify development of stateful streaming applications.

typedkafka

A well-documented, fully type-hinted Kafka client for Python

sdtf

Exploring streaming options for decision trees and random forests. Based on scikit-learn fork.

cinje

A Pythonic and ultra fast template engine DSL.