Preprocessing Python Packages

jaconv

Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

nnaudio

Audio processing by using pytorch 1D convolution network

semhash

Fast Multimodal Semantic Deduplication & Filtering

kditransform

Kernel density integral transformation: feature preprocessing and univariate clustering (TMLR, 2023)

neologdn

Japanese text normalizer for mecab-neologd

contextualspellcheck

✔️Contextual word checker for better suggestions (not actively maintained)

pyprep

PyPREP: A Python implementation of the Preprocessing Pipeline (PREP) for EEG data

indic-num2words

Python library for converting numbers to words for all Indian Languages.

autots

Automated Time Series Forecasting

df2onehot

Convert a unstructured array into a stuctured dataframe.

autoreject

Automated rejection and repair of bad trials/sensors in M/EEG

eegprep

EEGPrep is an automated preprocessing tool for human EEG data built on a benchmarked EEGLAB pipeline

nvtabular

NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.

jionlp

中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com

nums-from-string

Extract numbers from a string

smashed

SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.

logprep

log data pre processing, generation and shipping in python

cmip6-preprocessing

Analysis ready CMIP6 data in python the easy way with pangeo tools.

pyhealth

A Deep Learning Python Toolkit for Healthcare Applications.

maldiamrkit

Comprehensive toolkit for MALDI-TOF mass spectrometry data preprocessing for antimicrobial resistance (AMR) prediction purposes

pretab

PreTab is a modular, scikit-learn-compatible representation and preprocessing library for tabular data. Its focus is feature representation and expansion: splines, neural basis maps, piecewise-linear encoding, kernel approximations, and language embeddings, each shipped as a standalone transformer that speaks the standard fit / transform.

xmip

Analysis ready CMIP6 data in python the easy way with pangeo tools.

phenofhy

phenofhy is a Python package for phenotype analysis in the Our Future Health trusted research environment