How AI describes OpenReview

LoRA (Low-Rank Adaptation) is a method that reduces the number of trainable parameters for fine-tuning large language models by learning pairs of rank decomposition matrices while freezing original weights. It achieves performance comparable or superior to full fine-tuning on benchmarks like GLUE and NLG tasks, with drastically reduced storage and efficient task switching.