Skip-gram and negative sampling (word2vec)
How word2vec's skip-gram learns word vectors by guessing each word's neighbours, and why negative sampling makes it fast. Toy corpus, live training.
Part of Words as numbers.
Read next
- GloVe: word vectors from co-occurrence ratios
How GloVe (Global Vectors) turns ratios of word co-occurrence counts into vectors, why king − man + woman ≈ queen works, and how bias rides along.
Related
- CLIP: contrastive image–text pretraining
How CLIP (Contrastive Language-Image Pre-training) learns from captions alone: the similarity matrix, contrastive loss, temperature and zero-shot prompts.