Learning the way by walking it
Sarsa, on-policy TD control: nudge each state–action value toward the reward plus the value of the move you take next, while choosing moves by those values.
Part of Learning to act.
Read first
- Better, then best
Policy iteration alternates judging a policy and switching every state to its best-looking move; each round is never worse, and it reaches the optimal policy.
- Temporal-difference learning
Nudge each guess toward the next one, and learn during the game, not after it.
Read next
- Learning the best while still exploring
Q-learning backs up the best next move instead of the one taken, so it learns optimal values while still exploring (off-policy TD control).