Algorithms for singularly perturbed limiting average Markov Control Problems

Abstract

The authors consider a singularly perturbed Markov decision process (MDP) with the limiting average cost criterion. It is assumed that the underlying process is composed of n separate irreducible processes, and that the small perturbation is such that it 'unites' these processes into a single irreducible process. This structure corresponds to the Markov chains admitting strong and weak interactions. The authors introduce the formulation and some results given by Bielecki and Filar (1989) for the underlying control problem for the singularly perturbed MDP, the limit Markov control problem (limit MCP). It is demonstrated that the limit MCP can be solved by a suitably constructed linear program. An algorithm for solving the limit MCP based on the policy improvement method is constructed.