Dpo Python Packages

whileai

Scientific RL and SFT post-training for AI agents: build evals that can fail, simulate situations, judge checked against people, pass@1 with a 95% interval, train with GRPO/SFT/DPO, prove every gain on a held-out set. uv add whileai

zeroproof

Scientific RL and SFT post-training for AI agents: build evals that can fail, simulate situations, judge checked against people, pass@1 with a 95% interval, train with GRPO/SFT/DPO, prove every gain on a held-out set. uv add whileai

dtflow

Inspect, clean, transform and ship LLM training data from one CLI; dt view is the interactive entry point.

soup-cli

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

mlx-lm-lora

Train Large Language Models on MLX.

oumi

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

trloom

YAML-driven end-to-end fine-tuning on Hugging Face TRL — one config for model, dataset, W&B, and Modal GPUs.

pushback

How often do you correct your coding agent, and on what kind of work? Measures it from your Claude Code transcripts, drafts CLAUDE.md rules from your recurring corrections, and exports them as preference pairs.

openpo

Synthetic data for fine tuning LLM

oat-llm

🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.

afterimage

Generate conversational, tool-calling, structured-output, and preference datasets — easily and at scale

forgelm

Config-driven LLM fine-tuning with safety evaluation, EU AI Act compliance, 6 alignment methods, and one-command bundled quickstart templates.

shadowlm

A fine-tuning SDK — any open model, any harness, any method. 12 training methods behind one argument; pure-stdlib core.

diffav

Physics-informed, RL-aligned evaluation engine for autonomous driving: JAX/Flax NNX trajectory-diffusion world models on the Waymo Open Dataset, DPO and reward-guided adversarial steering toward the scenarios that break AV stacks, physics feasibility, occupancy flow, WOSAC-style realism metrics and sensor simulation.

sillm-mlx

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.

gleamlm

GleamLM大语言模型全栈训练平台——核心代码用 PyTorch 从零实现，最低仅需 12GB 消费级显卡即可完整训练，另有一套工业代码与手写实现消费同一份数据逐层对照。模型训练支持CLI和可视化训练。

knovaryn

Open, MCP-native training-data foundry: turn permitted documents into traceable, quality-gated SFT and preference datasets.

oxrl

A lightweight post-training framework for LLMs and VLMs. 51 algorithms, 38 verified models. Scales with DeepSpeed, vLLM, and Ray.

zeroproof-simulations

Scientific RL and SFT post-training for AI agents: build evals that can fail, simulate situations, judge checked against people, pass@1 with a 95% interval, train with GRPO/SFT/DPO, prove every gain on a held-out set. uv add whileai

knowlyr-recorder

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

mlora-cli

An Efficient "Factory" to Build Multiple LoRA Adapters

knowlyr-hub

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-core

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-sandbox

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.