Judging a way of playing
Iterative policy evaluation: a state is worth one step's reward plus the average worth of where you land (the Bellman equation), swept until values settle.
Part of Planning.
Read first
- A square is worth its neighbours
A state is worth the average of reward plus the discounted worth of where you land: the Bellman equation.
Read next
- Better, then best
Policy iteration alternates judging a policy and switching every state to its best-looking move; each round is never worse, and it reaches the optimal policy.
- Play it out, then average
Monte Carlo prediction estimates a state's value with no model by averaging the returns that follow each visit: unbiased, but it waits for the episode to end.