Smoothing: add-one and add-k for n-gram language models
Why one unseen word pair gives a sentence zero probability, and how add-one (Laplace) and add-k smoothing and the UNK token fix it.
Part of Predicting the next word.
Read next
- Perplexity: how language models are scored
Perplexity scores a language model by the per-word chance it gives held-out text, read as how many words it is choosing between at each step.