Logistic regression: sigmoid and log loss
How logistic regression turns a linear score into a probability with the sigmoid, and fits it by maximum likelihood: minimising log loss (cross-entropy).
Part of Linear classifiers.
Read first
- Gradient descent: step size and convergence
How gradient descent steps against the gradient, why the step size (learning rate) must stay under 2/L, and how the condition number sets its pace.
Read next
- The perceptron: mistake-driven learning and its bound
How the perceptron learns a linear classifier from its mistakes alone, and why on separable data it stops within (R/γ)² mistakes.
- Bag of words: turning text into word counts
How the bag-of-words model turns text into word counts for text classification, why learned weights beat word lists, and what losing word order costs.
Related
- GANs: generative adversarial networks explained
How generative adversarial networks (GANs) train a generator against a discriminator, why the generator's gradient vanishes, and what mode collapse is.