Data Selection Python Packages

transformers-domain-adaptation

:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains

data-selection

DSIR large-scale data selection framework for language model training

scar-tool

[ACL 2025 main] SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models

dataflex

A data-centric training system for Large Language Models

gen-dedup

Generative deduplication