Abstract

Probabilistic max-plus schemes for solving Hamilton-Jacobi-Bellman equations

Marianne Akian
Institut National de Recherche en Informatique et Automatique (INRIA)

We consider Hamilton-Jacobi-Bellman equations
associated to diffusion control problems involving
a finite set-valued (or switching) control and possibly
a continuum-valued control.
We develop several lower complexity probabilistic numerical algorithms
for such equations by combining max-plus and numerical probabilistic approaches.
For deterministic optimal control problems,
the max-plus approach is using the max-plus linearity of
the Lax-Oleinik operator associated to the Hamilton-Jacobi equation.
A stochastic max-plus approach has been introduced by Z. Qu (2013) for
deterministic control problems.
We shall compare it with the stochastic dual dynamic programming
method which is used in the context of discrete time problems.

For stochastic control problems, we can use a max-plus approach
in the spirit of the one of
McEneaney, Kaise and Han (2011), which is based on the distributivity
of monotone operators with respect to suprema.
We show how to combine it with a numerical probabilistic approach
obtained by improving the one proposed by Fahim, Touzi and Warin (2011)
in such a way that it satisfies a monotonicity property.

This talk is based joint works with Jean-Philippe Chancelier, Benoît Tran
and Eric Fodjo.