CLIP: contrastive image–text pretraining
How CLIP (Contrastive Language-Image Pre-training) learns from captions alone: the similarity matrix, contrastive loss, temperature and zero-shot prompts.
Part of Images meet text.
Read first
- Vision Transformer (ViT): images as 16×16 patches
How the Vision Transformer (ViT) cuts an image into patches, turns each into a token with a [class] token, and why it needs far more data than a CNN.
Read next
- Vision-language models: how LLaVA sees images
How vision-language models (VLMs) like LLaVA turn image patches into visual tokens for a language model, and why Qwen2-VL's token count follows image size.
Related
- Skip-gram and negative sampling (word2vec)
How word2vec's skip-gram learns word vectors by guessing each word's neighbours, and why negative sampling makes it fast. Toy corpus, live training.