Streaming Data Python Packages

smart-open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

72M 3K 389

optbinning

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

292K 529 116

river

🌊 Online machine learning in Python

227K 6K 641

shioaji

Shioaji all new cross platform api for trading ( 跨平台證券交易API )

92K 477 38

quixstreams

Python Streaming DataFrames for Kafka

77K 2K 107

streamz

Real-time stream processing for python

41K 1K 149

bytewax

Python Stream Processing

32K 2K 110

dbt-materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

6K 6K 513

streaming-json-parser

Streaming JSON parser designed to process JSON data incrementally. The primary goal is to handle potentially incomplete JSON data streams, such as those produced by Large Language Models (LLMs), and return the current state of the parsed object at any time.

5K 13 2

scikit-multiflow

A machine learning package for streaming data in Python. The other ancestor of River.

5K 798 188

creme

🌊 Online machine learning in Python

3K 6K 641

datafun-streaming

Utilities for streaming data analytics with Kafka and DuckDB.

3K 1 0

kafi

The Swiss Army Knife for Kafka

3K 33 6

rxsci

ReactiveX for data science

2K 14 2

pandas-streaming

Streaming API for pandas applied to big datasets

2K 31 9

rdst

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

1K 5K 160