Data Selection Python Packages

dataflex

[NeurIPS 2026] DataFlex: A Unified Benchmark and Evaluation Platform for Data-Centric Training of Large Language Models

transformers-domain-adaptation

:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains

scar-tool

[ACL 2025 main] SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models

data-selection

DSIR large-scale data selection framework for language model training

gen-dedup

Code for Generative Deduplication For Socia Media Data Selection (Findings of EMNLP 2024)