Mean field for Markov Decision Processes: from Discrete to Continuous OptimizationReportar como inadecuado

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.

1 MESCAL - Middleware efficiently scalable Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d-Informatique de Grenoble 2 EPFL - Ecole Polytechnique Fédérale de Lausanne

Abstract : We study the convergence of Markov Decision Processes made of a large number of objects to optimization problems on ordinary differential equations ODE. We show that the optimal reward of such a Markov Decision Process, satisfying a Bellman equation, converges to the solution of a continuous Hamilton-Jacobi-Bellman HJB equation based on the mean field approximation of the Markov Decision Process. We give bounds on the difference of the rewards, and a constructive algorithm for deriving an approximating solution to the Markov Decision Process from a solution of the HJB equations. We illustrate the method on three examples pertaining respectively to investment strategies, population dynamics control and scheduling in queues are developed. They are used to illustrate and justify the construction of the controlled ODE and to show the gain obtained by solving a continuous HJB equation rather than a large discrete Bellman equation.

Keywords : Mean Field Hamilton-Jacobi-Bellman Optimal Control Markov Decision Processes

Autor: Nicolas Gast - Bruno Gaujal - Jean-Yves Le Boudec -



Documentos relacionados