Preference Learning Python Packages

pairjudge

Pairwise LLM judges (A/B/tie): budget-aware multi-turn packing, position-bias correction, pseudo-label distillation. Generalized from the 4th-place (gold) solution to Kaggle LMSYS Chatbot Arena.

solidago

A toolbox for Solid Algorithmic Governance

magical-il

MAGICAL is a benchmark suite for robust imitation learning

pypbl

A python library for preference based learning using pairwise comparisons.

prl

[P]reference and [R]ule [L]earning algorithm implementation

flash-pref

Accelerate LLM preference finetuning with a single line of code

wiserl

PyTorch implementations for Offline Preference-Based RL (PbRL) algorithms