LoRA: low-rank fine-tuning for large language models
How Low-Rank Adaptation (LoRA) fine-tunes a large language model by freezing its weights and training a small rank-r change BA, merged in at the end.
Part of Post-training.
Read next
- Instruction tuning (SFT): teaching a model to answer
Instruction tuning, or supervised fine-tuning (SFT), trains a pretrained model on instruction–response pairs, scoring only the response tokens.