Continuous actions
DDPG (deep deterministic policy gradient) handles continuous actions: an actor climbs the critic's slope to the best action, with noise to explore nearby.
Part of Deep RL.
Read first
- A critic at every step
Actor-critic methods judge each move as it is made with a learned critic and its TD error, instead of waiting for the return: less luck, some bias.
- Deep Q-networks
Deep Q-networks (DQN) replace Q-learning's table with a neural network, steadied by experience replay and a frozen target network.
Related
- Continuous control with deep reinforcement learning
Lillicrap et al., 2016.
- A centre and a spread
Gaussian policies for continuous actions: draw each action from a bell curve and let the policy gradient move its centre and learn how widely to explore.