Reward Hacking Python Packages

bohrin

Check the checker: an open standard for testing the graders of RL environments, with proof before every accusation.

checkwash

Flags known patterns of weakened tests and CI checks in a Git diff.

rldoctor

Diagnose why your GRPO/RLVR run is burning GPU-hours without learning. Reads the log you already have: W&B, TensorBoard, verl, TRL trainer_state.json.

rewardlint

Your RLVR reward function has a false-positive rate. Find out what it accepts that it shouldn't. Zero dependencies, no GPU, runs in a second.

ai-crucible

Diagnostic adversarial game for frontier LLMs — a policy-enforced kernel that mediates a Designer/Solver/Judge cycle, scores against a hidden oracle, and curates a Lab/Arena/Regression catalog.

goodhart

Catch reward traps before training. Static analysis for RL reward functions.

reward-lens

Measurement instruments for reward models and RL runs. Every reading is either evidence or a refusal that tells you what to fix.

tripwire-oracle

Correctness oracle for AI code-optimization loops: catches reward hacks with canonical, metamorphic, withheld-input, and timing checks.

rl-envdoctor

Adversarial QA for LLM-RL environments: find out what reward an empty answer earns. Model-free, zero API cost.

verifierlab

Local-first campaigns for stress-testing graders, reward functions, and policy checkers under optimization pressure.