Decomposition and parallel processing techniques for two-time scale controlled Markov chains
View/ Open
Date
2000Author
Filar, Jerzy A
Gondzio, Jacek
Haurie, Alain
Moresino, Francesco
Vial, Jean-Philippe
Metadata
Show full item recordAbstract
This paper deals with a class of ergodic control problems
for systems described by Markov chains with
strong and weak interactions. These systems are composed
of a set of m subchains that are weakly coupled.
Using results recently established by Abbad et
al. one formulates a limit control problem the solution
of which can be obtained via an associated non-differentiable
convex programming (NDCP) problem. The
technique used to solve the NDCP problem is the Analytic
Center Cutting Plane Method (ACCPM) which
implements a dialogue between, on one hand, a master
program computing the analytical center of a localization
set containing the solution and, on the other hand,
an oracle proposing cutting planes that reduce the size
of the localization set at each main iteration. The interesting
aspect of this implementation comes from two
characteristics: (i) the oracle proposes cutting planes
by solving reduced sized Markov Decision Problems
(MDP) via a linear program (LP) or a policy iteration
method; (ii) several cutting planes can be proposed simultaneously
through a parallel implementation on m
processors. The paper concentrates on these two aspects
and shows, on a large scale MDP obtained from
the numerical approximation "a la Kushner-Dupuis” of
a singularly perturbed hybrid stochastic control problem,
the important computational speed-up obtained.