Low-Rank Adaptation in AI Explainer
Low-rank adaptation, commonly called LoRA, adapts a pretrained model without directly updating every original weight. It freezes the base parameters and learns smaller matrices whose product represents a task-specific change to selected weight layers.
The low-rank structure assumes useful updates can be expressed in a much smaller space than the full weight matrix. That reduces trainable parameters and optimizer memory, and the learned update can often be stored separately from the base model.
LoRA is widely used to customize language and image models for domains, styles, or tasks. Different adapters can share one base model, and some implementations merge the update for inference so it adds little or no extra latency.
Rank, target layers, data quality, and training settings affect the result. A small adapter can still memorize sensitive material or introduce harmful behavior. Compatibility with the exact base-model version and careful evaluation are essential.
Low-rank adaptation, or LoRA, fine-tunes a model by training small low-rank weight updates while freezing the original weights.
Key terms include LoRA, low rank, frozen weights, adapter, weight update, rank decomposition, parameter efficiency and base model.
Two small trainable matrices approximate a larger task-specific change to selected model weights.
The pretrained weights remain fixed and can be shared among several adapters.
Adapter size does not measure the importance or safety of the behavior it introduces.
Teams must track base-model compatibility, training data, rank choices, and evaluation results.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.
