Activation Steering in AI Explainer
Activation steering modifies a model’s internal activations during inference to influence its output. A steering direction may be estimated by comparing activations for contrasting prompts, examples, or labeled behaviors.
The direction is added or subtracted at selected layers and token positions, often with a tunable strength. This can shift properties such as sentiment, topic, style, refusal, or another measurable behavior without changing the model’s weights.
Steering is fast to experiment with and can reveal how behavior relates to internal representations. The same vector may behave differently across contexts, layers, models, or generation settings, and stronger intervention can damage unrelated capabilities.
Evaluation should measure target effects, off-target changes, robustness, reversibility, and adversarial responses. A controllable direction is not necessarily a stable or safe representation of the human concept used to create it.
Activation steering changes model behavior at inference time by adding, subtracting, or modifying directions in internal activation space.
Key terms include activation, steering vector, contrast pair, layer, token position, intervention strength, inference, off-target effect and reversibility.
Contrasting examples define a direction associated with a behavior.
Modifying activations along that direction shifts the model without weight updates.
Large interventions may reduce fluency, accuracy, or unrelated capabilities.
Target and off-target effects must be evaluated across diverse contexts.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.