Data Synchronization Python Packages

diffsync

A utility library for comparing and synchronizing different datasets.

crdt-merge

Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.

datalore

Data Interface Abstraction Library

aiosafeconsumer

Safely consume and process data.

dbmerge

A database-agnostic Python library for simple and fast UPSERT (Insert/Update/Delete) operations via SQLAlchemy.

django-kafka-streamer

Stream data to Apache Kafka.

simpledht

A Python-based Distributed Hash Table (DHT) implementation enabling cross-network key-value storage, automatic node discovery, and data replication with a simple CLI and library interface.

pupillib

A software package to perform trial extraction on Pupil Labs eye tracker data collected into XDF files through LabRecorder.