Judging a way of playing

Iterative policy evaluation: a state is worth one step's reward plus the average worth of where you land (the Bellman equation), swept until values settle.

Part of Planning.

Read first

Read next

  • Better, then best

    Policy iteration alternates judging a policy and switching every state to its best-looking move; each round is never worse, and it reaches the optimal policy.

  • Play it out, then average

    Monte Carlo prediction estimates a state's value with no model by averaging the returns that follow each visit: unbiased, but it waits for the episode to end.