Skip the sweep

Asynchronous dynamic programming backs up one state at a time in any order; values still settle, so you can back up only the states play reaches.

Part of Planning.

Read first

  • Improve as you go

    Value iteration gives every state the value of its best move, sweeps until the values stop changing, then reads off the optimal policy once.

Read next

  • Two pulls, one answer

    Generalized policy iteration: evaluation pulls values toward the policy, improvement pulls the policy toward the values, and both meet at the best policy.