Data Filtering Python Packages

data-selection

DSIR large-scale data selection framework for language model training

scar-tool

[ACL 2025 main] SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models

sqljson

A powerful tool that allows users to query JSON data using SQL-like syntax. Effortlessly search, filter, and manipulate your JSON data with familiar SQL queries.

lumi-filter

A powerful and flexible data filtering library with unified interface for multiple data sources including Peewee ORM, Pydantic models, and Python iterables. Flask-friendly.