Dpo Python Packages

soup-cli

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

oumi

Easily fine-tune, evaluate and deploy Gemma 4, Qwen3.5, Qwen3.6, gpt-oss, DeepSeek-R1, or any open source LLM / VLM!

afterimage

Generate conversational, tool-calling, structured-output, and preference datasets — easily and at scale

mlx-lm-lora

Train LLMs on Apple silicon with MLX and the Hugging Face Hub

openpo

Build high quality synthetic datasets with AI feedback from 200+ LLMs

oat-llm

Online AlignmenT (OAT) for LLMs.

sillm-mlx

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.

knowlyr-sandbox

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-hub

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-recorder

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-reward

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

knowlyr-core

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

mlora-cli

An Efficient "Factory" to Build Multiple LoRA Adapters

toolbrain

A framework for agentic tool use training with reinforcement learning

oxrl

A lightweight post-training framework for LLMs and VLMs

sygra

Graph-oriented Synthetic data generation Pipeline library

knowlyr-trainer

Gymnasium-style RL framework for LLM agent training — MDP environments, three-layer process reward & SFT/DPO/GRPO policy optimization. CLI + MCP ready.

flash-pref

Accelerate LLM preference tuning via prefix sharing with a single line of code