Vision-language models: how LLaVA sees images
How vision-language models (VLMs) like LLaVA turn image patches into visual tokens for a language model, and why Qwen2-VL's token count follows image size.
Part of Images meet text.
Read first
- CLIP: contrastive image–text pretraining
How CLIP (Contrastive Language-Image Pre-training) learns from captions alone: the similarity matrix, contrastive loss, temperature and zero-shot prompts.
Read next
- GANs: generative adversarial networks explained
How generative adversarial networks (GANs) train a generator against a discriminator, why the generator's gradient vanishes, and what mode collapse is.
Related
- Instruction tuning (SFT): teaching a model to answer
Instruction tuning, or supervised fine-tuning (SFT), trains a pretrained model on instruction–response pairs, scoring only the response tokens.