Walk a few steps, then guess

Add up the next n rewards, then trust the value of where you land: a dial between TD(0) and Monte Carlo, where more steps mean less bias and more variance.

Part of Values.

Read first

  • Play it out, then average

    Monte Carlo prediction estimates a state's value with no model by averaging the returns that follow each visit: unbiased, but it waits for the episode to end.

  • Temporal-difference learning

    Nudge each guess toward the next one, and learn during the game, not after it.

Read next

  • Learning the way by walking it

    Sarsa, on-policy TD control: nudge each state–action value toward the reward plus the value of the move you take next, while choosing moves by those values.

Related