Parquet Python Packages

pyarrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

influxdb3-python

Python module that provides a simple and convenient way to interact with InfluxDB 3.0.

petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

awkward0

Manipulate arrays of complex data structures as easily as Numpy.

parquet-metadata

Dump metadata about a Parquet file.

parquet-tools

easy install parquet-tools

lonboard

Fast, interactive geospatial data visualization in Jupyter.

dask-deltatable

A Delta Lake reader for Dask

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

quilt3

Quilt is a Scientific Data Management Platform on AWS that helps teams and AI find, trust, and reuse data through deeply versioned, context-rich data packages.

airflow-provider-xlsx

Airflow operators for converting XLSX files from/to Parquet/CSV/JSON

ematix-flow

Move data between databases, files, and streams from Python. 5.87× faster than PySpark.

polars-config-meta

A Polars plugin for persistent DataFrame-level metadata

koala-diff

Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.

daft-lts

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

sqllocks-spindle

Multi-domain, schema-aware synthetic data generator for Microsoft Fabric. 13 domains, billion-row scale, statistically calibrated. Lakehouse · Warehouse · SQL DB · Eventhouse writers.

parquet-py

A simple command-line interface & Python API for parquet

slothdb

SlothDB is an embedded SQL database that runs everywhere: on your laptop, on a server, and in the browser. Built from scratch. Up to 5x faster where it counts.

quilt

Quilt is a Scientific Data Management Platform on AWS that helps teams and AI find, trust, and reuse data through deeply versioned, context-rich data packages.

mat-vis-client

PBR texture data factory — ~3000 materials from ambientcg, polyhaven, gpuopen, physicallybased.info baked to Parquet on GitHub Releases. pip install mat-vis-client

cryo

cryo is the easiest way to extract blockchain data to parquet, csv, json, or python dataframes

imctermite

Enables extraction of measurement data from binary files with extension 'raw' used by proprietary software imcFAMOS/imcSTUDIO and facilitates its storage in open source file formats

rugo

📒 Python Parquet & JSON Lines reader