Sooner is worth more

Count each reward further off for less, and even a task that never ends has a finite return.

Part of Markov decision processes.

Read first

  • Reward versus value

    Reward is what’s good now; value is all the reward still to come. Choose by value.

Read next

  • A square is worth its neighbours

    A state is worth the average of reward plus the discounted worth of where you land: the Bellman equation.

  • Play it out, then average

    Monte Carlo prediction estimates a state's value with no model by averaging the returns that follow each visit: unbiased, but it waits for the episode to end.