The completely mixed stochastic game

Abstract

We consider a zero-sum stochastic game with finitely many states and actions. Further we assume that the transition probabilities depend on the actions of only one player (player II, in our case), and that the game is completely mixed. That is, every optimal stationary strategy for either player assigns a positive probability to every action in every state. For these games, properties analogous to those derived by Kaplansky for the completely mixed matrix games, are established in this paper. These properties lead to the counter-intuitive conclusion that the controller need not know the law of motion in order to play optimally, but his opponent does not have this luxury.