Skip the sweep
Asynchronous dynamic programming backs up one state at a time in any order; values still settle, so you can back up only the states play reaches.
Part of Planning.
Read first
- Improve as you go
Value iteration gives every state the value of its best move, sweeps until the values stop changing, then reads off the optimal policy once.
Read next
- Two pulls, one answer
Generalized policy iteration: evaluation pulls values toward the policy, improvement pulls the policy toward the values, and both meet at the best policy.