Pyarrow Python Packages

narwhals

Lightweight and extensible compatibility layer between dataframe libraries!

pyarrow-stubs

Type annotations for pyarrow

ibis-framework

the portable Python dataframe library

petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

datagrunt

Datagrunt is a Python library designed to simplify the way you work with CSV, Excel, and PDF files. It provides a streamlined approach to reading, processing, and transforming your data into various formats, making data manipulation efficient and intuitive.

vgi-python

Python SDK for the VGI (Vector Gateway Interface) protocol — host DuckDB scalar, table, aggregate, and table-in-out functions in an external worker over Arrow IPC.

keyedstablehash

Stable, keyed hashing for Python objects and columnar data. Think `stablehash`, but with SipHash-like keyed PRF semantics so hashes are deterministic for a given key and resistant to adversarial inputs.

daffy

Lightweight DataFrame validation decorators for Pandas, Polars, Modin, and PyArrow. No custom types required.

timedatamodel

A lightweight Pythonic data model for time series data, interoperable with NumPy, Pandas and Polars.

fast-xml-flattener

Rust-powered Python library for flattening nested XML into JSON, dict, CSV and Parquet. 4–7× faster than xmltodict.

jetxl

Blazingly fast Excel writer for Python, powered by Rust

schema2pyarrow

Converts AsyncApi and JsonSchema to PyArrow schema

pqviewer

View Apache Parquet Files In Your Terminal

fsspec-utils

Enhanced utilities and extensions for fsspec filesystems with multi-format I/O support

exness-data-preprocess

Professional forex tick data preprocessing with unified DuckDB storage, Phase7 OHLC schema, and sub-15ms query performance

pdf2dataset

Easily convert a subdirectory with big volume of PDF documents into a dataset, supports extracting text and images

db2ixf

db2ixf is a python package with a CLI that simplifies the parsing and processing of IBM Integration eXchange Format (IXF) files.

dremio-arrow

Dremio Arrow Flight Client

sql2arrow

This is a Python library that provides convenient and high-performance methods to parse INSERT SQL statements into Arrow arrays.

pysummaries

Generate beautiful summary tables from pandas, polars or pyarrow dataframes

biosets

A bioinformatics extension of 🤗 Datasets library, built for ML applications on biological and omics data, offering easy integration of metadata and low-code data management tools.

swemaps

Maps of Sweden in GeoParquet for easy usage.

turntable-spoonbill

the portable Python dataframe library

hops-petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.