Learning the policy itself

REINFORCE, the Monte Carlo policy gradient: make each move more likely in proportion to how much better than usual its return was, with a baseline to cut noise.

Part of Policy gradients.

Read first

  • Preferences, not values

    Learn how much to favour each option, and compare every reward with what you usually get.

  • Play it out, then average

    Monte Carlo prediction estimates a state's value with no model by averaging the returns that follow each visit: unbiased, but it waits for the episode to end.

Read next

  • A critic at every step

    Actor-critic methods judge each move as it is made with a learned critic and its TD error, instead of waiting for the return: less luck, some bias.

  • A centre and a spread

    Gaussian policies for continuous actions: draw each action from a bell curve and let the policy gradient move its centre and learn how widely to explore.

  • RLVR and GRPO: rewards you can check

    How reinforcement learning with verifiable rewards (RLVR) trains reasoning models, and how GRPO scores each answer against its own group's mean.