What Is Prompt Tuning?

Prompt Tuning Explainer

Prompt tuning adapts a pretrained language model by learning a short sequence of continuous vectors, often called a soft prompt. These vectors are placed with the input while the original model parameters remain frozen.

Unlike a written prompt, a soft prompt does not need to correspond to readable words. Gradient-based training adjusts its values for a downstream task, storing far fewer task-specific parameters than a full copy of the tuned model.

The method can make it practical to maintain many task variants of one large base model. Research has found that prompt tuning becomes more competitive with full fine-tuning as model scale increases, although outcomes vary by setting.

Soft prompts can be sensitive to initialization, data size, model choice, and distribution shift. Their compactness does not guarantee interpretability or safety. Each prompt-model combination still needs task-specific validation, monitoring, and version control.

Prompt tuning adapts a frozen language model by learning a small set of continuous prompt vectors for a task.

Key terms include soft prompt, continuous vector, frozen model, task parameters, initialization, fine-tuning, scale and transfer.

Only a small collection of learned input vectors changes during task adaptation.

The large pretrained model can remain shared across many separately stored soft prompts.

Performance depends on model scale, task structure, initialization, and available examples.

Compact task parameters still require careful testing and lifecycle management.

Explore more "Explainers"

Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.