AI Model Backdoor Attack Explainer
A backdoor attack implants a hidden input-to-output rule in an AI model. The compromised model behaves normally on typical examples but switches to an attacker-chosen response when it detects a particular trigger.
The trigger might be a small visual pattern, an unusual phrase, a formatting detail, or a feature embedded during training. Backdoors can be introduced through poisoned data, altered training code, or a compromised model checkpoint.
Because clean-input accuracy can remain high, routine evaluation may not reveal the problem. The trigger can also be designed to be rare or subtle, making it difficult to discover without a relevant threat model.
Defenders inspect data and model provenance, test unusual input transformations, scan internal activations, and compare behavior before and after fine-tuning. Sensitive systems should treat third-party checkpoints and datasets as untrusted until they are evaluated.
A backdoor attack makes an AI model behave normally most of the time but produce an attacker-chosen result when a hidden trigger appears.
Key terms include backdoor, trigger, target label, poisoned data, checkpoint, clean accuracy, activation, provenance and model supply chain.
The model appears ordinary until a specific pattern or condition is present.
When triggered, it produces the response selected by the attacker.
A backdoored model may pass tests made only from clean examples.
Evaluation must include supply-chain checks and trigger-focused testing.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.