Probing Classifier Explainer
A probing classifier is a secondary model trained on frozen internal representations from another model. Its job is to predict a selected property, such as grammatical number, object position, or topic, from those representations.
High probe accuracy suggests that information about the property is available to be decoded at that layer. Comparing layers, tasks, and controls can reveal where representations become more or less informative.
A powerful probe may learn the task even when the representation contains only weak clues. Conversely, a simple probe may miss information encoded in a nonlinear form. Probe capacity and dataset design therefore shape the conclusion.
Most importantly, decodability does not prove that the original model actually uses the information for its prediction. Causal interventions, control tasks, selectivity measures, and behavioral tests are needed for stronger claims.
A probing classifier tests whether information about a selected property can be decoded from a model's internal representations.
Key terms include probe, frozen representation, layer, decodability, probe capacity, control task, selectivity, causal intervention and behavior.
Internal activations become inputs to a separate supervised classifier.
Probe performance measures how readily a chosen property can be decoded.
A representation can contain a signal that the original model ignores.
Causal tests are needed before claiming that the feature drives behavior.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.