Preference Learning Python Packages

pairjudge

Pairwise LLM judges (A/B/tie): budget-aware multi-turn packing, position-bias correction, pseudo-label distillation. Generalized from the 4th-place (gold) solution to Kaggle LMSYS Chatbot Arena.

solidago

Free and open source code of the https://tournesol.app platform. Meet the community on Discord https://discord.gg/WvcSG55Bf3

rewardharness

Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward-model training. arXiv 2605.08703.

preorder4mlc

Pre-Order Based Multi-Label Classification via pairwise classifiers and ILP search (BOPOs). Reference implementation accompanying the paper.

magical-il

The MAGICAL benchmark suite for robust imitation learning (NeurIPS 2020)

pypbl

Python library for preference based learning

prl

[P]reference and [R]ule [L]earning algorithm implementation for Python 3 (https://arxiv.org/abs/1812.07895)

flash-pref

Accelerate LLM preference tuning via prefix sharing with a single line of code

wiserl

PyTorch implementations for Offline Preference-Based RL (PbRL) algorithms