LoRA: low-rank fine-tuning for large language models

How Low-Rank Adaptation (LoRA) fine-tunes a large language model by freezing its weights and training a small rank-r change BA, merged in at the end.

Part of Post-training.

Read next