Markov Decision Processes Python Packages

csle-collector

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-ryu

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-cli

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

gym-csle-stopping-game

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-common

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-attacker

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-defender

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-rest-api

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-cluster

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-agents

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-system-identification

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

gym-csle-intrusion-response-game

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

mdplib

Fast solvers for infinite-horizon discounted Markov Decision Processes (value iteration, Gauss-Seidel VI, policy iteration, Q-learning, SARSA) with a C++/pybind11 backend.

csle-base

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

auto-mdp-solver

Formalize a verbal dynamic-decision problem into an MDP, then train and interpret a deployable RL policy — as Claude Code skills with executable gates between stages

csle-tolerance

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

gym-csle-cyborg

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

gym-csle-apt-game

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

csle-attack-profiler

A research platform to develop automated security policies using quantitative methods, e.g., optimal control, computational game theory, reinforcement learning, optimization, evolutionary methods, and causal inference.

aniate

Approximate Numerics in Applied Transition Environments: finite MDPs and history-dependent decision problems in Python

pomdp-py

A framework to build and solve POMDP problems. Documentation: https://h2r.github.io/pomdp-py/

ns-gym

A framework for modeling non-stationary Markov decision processes and the key decision making problems in these environments

mdp-solver-ic

Fast solvers for infinite-horizon discounted Markov Decision Processes (value iteration, Gauss-Seidel VI, policy iteration, Q-learning, SARSA) with a C++/pybind11 backend.

jaxdp

A Dynamic Programming and RL package for discrete MDPs implemented in JAX