Dpo Python Packages

soup-cli

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

shadowlm

A fine-tuning SDK — any open model, any harness, any method. 12 training methods behind one argument; pure-stdlib core.

mlx-lm-lora

Train Large Language Models on MLX.

oumi

Easily fine-tune, evaluate and deploy Gemma 4, Qwen3.5, Qwen3.6, gpt-oss, DeepSeek-R1, or any open source LLM / VLM!

afterimage

Generate conversational, tool-calling, structured-output, and preference datasets — easily and at scale

forgelm

Config-driven LLM fine-tuning with safety evaluation, EU AI Act compliance, 6 alignment methods, and one-command bundled quickstart templates.

oat-llm

🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.

openpo

Synthetic data for fine tuning LLM

sillm-mlx

Running and training LLMs on Apple Silicon via MLX

oxrl

A lightweight post-training framework for LLMs and VLMs. 51 algorithms, 38 verified models. Scales with DeepSpeed, vLLM, and Ray.

provenir

Reproducible, evaluation-first orchestration for LLM fine-tuning. RLAIF, LLM-as-Judge, DataFlywheel, RAG generation, adapter merging, benchmark eval, REST API.

knowlyr-hub

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-recorder

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-core

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

toolbrain

A framework for training LLM-powered agents to use tools more effectively using Reinforcement Learning

knowlyr-sandbox

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-reward

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

mlora-cli

An Efficient "Factory" to Build Multiple LoRA Adapters

sygra

Graph-oriented Synthetic data generation Pipeline library

flash-pref

Accelerate LLM preference finetuning with a single line of code

knowlyr-trainer

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.