New ways to solve games on any graph using efficient algorithms

New methods show that policy iteration and value iteration run in smoothed polynomial time for deterministic games.

Policy iteration and value iteration are two families of algorithms for turn-based stochastic games. These algorithms work well in practice but can run in exponential time on some input instances.

Researchers showed that policy iteration algorithms run in smoothed polynomial time for any game graph. This removes a previous requirement for an ergodic graph.

A condition number was defined to show that sufficiently-perturbed games are well-conditioned with high probability. The proof uses a new approach based on log-Lipschitz distributions.

Three new value iteration algorithms were proposed for solving deterministic mean-payoff games on any graph. These methods find fixed points of non-expansive operators applied to the Shapley operator.

The first two methods use Krasnoselskii-Mann iteration and Halpern iteration. The third method runs Zwick-Paterson and Halpern iterations in parallel and is quadratically faster.

All three new value iteration algorithms are correct and run in smoothed polynomial time. We still lack an algorithm that runs in polynomial time on all instances.

Key facts

  • Policy iteration algorithms run in smoothed polynomial time for any game graph.
  • Three new value iteration algorithms for deterministic mean-payoff games were proposed.
  • The third value iteration algorithm is quadratically faster than the others.
  • An algorithm that runs in polynomial time on all instances is still lacking.

All sections

Edition