Data Integrity Python Packages

datajoint

Relational Workflows: where database schemas define executable data pipelines.

encypher-ai

Metadata encoding and extraction for AI-generated content

sqlbucket

Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.

calendar-smith

Lightweight fiscal, ISO week, and timezone utilities for Python. Includes CSV processing, date windows, and safety-first date parsing. Zero dependencies.

fancy-sa-filemodel

A sqlalchemy extension to store files to various storage and maintain the data integrity.

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.

dazzle-preserve

Preserve is a lightweight cross-platform file "preservation" tool that maintains directory structure during file transfers. Includes path normalization (relative/absolute/flat), SHA256 verification, metadata retention, and bidirectional operations. Perfect for archiving, backups, and transferring files while maintaining their original organization.

chkbit

Check your files for data corruption and run quick file deduplication

aldertech-asg

ASG (Aldertech Storage Governor): Health monitoring, capacity planning, and intelligent scrub scheduling for BTRFS RAID pools with mismatched drives

sisu-wrapper

A constraint-aware decision-support system for Aalto course planning, designed around deterministic ranking and explicit scheduling trade-offs

janus-validation

Janus is a Python library designed for robust data validation, serialization, and schema versioning. It offers a comprehensive toolkit to handle input validation, data transformation, and API schema evolution, making it ideal for modern Python applications that require data integrity and compatibility.

reprokit-ml

One-command determinism + manifest for ML projects.

lokryn-merkle-tree

Merkle tree and hash chain utilities for building tamper-evident audit logs in Python

vaultxfer

Secure SSH-based CLI file transfer & bidirectional sync tool with atomic operations and integrity verification.