Top-k, top-p (nucleus) sampling and temperature

How language models pick the next word: sampling, top-k, top-p (nucleus) sampling and temperature, and why cutting the tail keeps text varied but sensible.

Part of Translation and generation.

Read next

Related