Gym Python Packages

gymnasium

An API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)

stable-baselines3

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

pettingzoo

An API standard for multi-agent reinforcement learning environments, with popular reference environments and related utilities

sb3-contrib

Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

skrl

Modular Reinforcement Learning (RL) library (implemented in PyTorch, JAX, and NVIDIA Warp) with support for Gymnasium/Gym, NVIDIA Isaac Lab, MuJoCo Playground and other environments

minigrid

Simple and easily configurable grid world environments for reinforcement learning

envpool

C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

gem-llm

A Gym for Agentic LLMs

gym-minigrid

Simple and easily configurable grid world environments for reinforcement learning

mo-gymnasium

Multi-objective Gymnasium environments for reinforcement learning

rl-zoo3

A training framework for Stable Baselines3 reinforcement learning agents, with hyperparameter optimization and pre-trained agents included.

stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms

vmas

VMAS is a vectorized differentiable simulator designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface.

gym-sokoban

Sokoban environment for OpenAI Gym

lightzero

[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)

sumo-rl

Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries.

evox

Distributed GPU-Accelerated Framework for Evolutionary Computation. Comprehensive Library of Evolutionary Algorithms & Benchmark Problems.

rtgym

Easily implement custom Gymnasium environments for real-time applications

gym-ignition

Framework for developing OpenAI Gym robotics environments simulated with Ignition Gazebo

train

A library to build and train reinforcement learning agents in OpenAI Gym environments.

gym-trading-env

A simple, easy, customizable Gymnasium environment for trading.

scenario

Framework for developing OpenAI Gym robotics environments simulated with Ignition Gazebo

pyrddlgym

A toolkit for auto-generation of OpenAI Gym environments from RDDL description files.