Instruction tuning (SFT): teaching a model to answer
Instruction tuning, or supervised fine-tuning (SFT), trains a pretrained model on instruction–response pairs, scoring only the response tokens.
Part of Post-training.
Read first
- Pretraining objectives: BERT, GPT and T5
How Transformers learn from raw text: next-word prediction (GPT), masked language modelling with the 80/10/10 rule (BERT), and span corruption (T5).
Read next
- DPO vs RLHF: learning from human preferences
How Direct Preference Optimization (DPO) trains a model on human preference pairs with one loss, reaching RLHF's result with no reward model or RL loop.
Related
- Vision-language models: how LLaVA sees images
How vision-language models (VLMs) like LLaVA turn image patches into visual tokens for a language model, and why Qwen2-VL's token count follows image size.