Dropout: why switching off neurons prevents overfitting
How dropout regularizes neural networks: random units are zeroed in training, kept units scaled by 1/(1−p), so the net acts like an average of thinned networks.
Part of Learning to see.
Read next
- Batch normalization: why networks normalize each batch
How batch normalization (BatchNorm) evens out each channel over a mini-batch, what the learned γ and β do, and why it uses running averages at test time.
Related
- Bias-variance tradeoff: why flexible models overfit
See why test error splits into bias², variance and noise, and why it is U-shaped as model complexity grows: the bias-variance tradeoff.