What Is Catastrophic Forgetting?

Catastrophic Forgetting Explainer

Catastrophic forgetting is the sharp loss of previously learned performance when a neural network is trained on new data or tasks. Updates that improve the new objective can overwrite parameters that were important to earlier behavior.

The problem is most visible in sequential training where old examples are no longer available. Standard shuffled training hides some interference by repeatedly mixing data from all tasks in each optimization cycle.

Mitigation methods replay stored or generated examples, penalize changes to important parameters, isolate parts of the network, or add new capacity. These approaches differ in memory requirements, privacy exposure, compute, and ability to transfer knowledge.

Forgetting is not measured by new-task accuracy alone. Evaluators record each task’s performance when learned and after later tasks, then calculate how much was lost. They should also distinguish harmful forgetting from useful adaptation to genuine change.

Catastrophic forgetting occurs when learning new information causes a neural network to lose previously acquired capabilities.

Key terms include interference, sequential training, replay, rehearsal, parameter importance, regularization, modular network, retention and transfer.

The same parameters often support several behaviors, so gradients for a new task can move them away from an older solution.

Without old examples or explicit constraints, the optimizer has no direct signal to preserve earlier performance.

Replay needs storage or generation, regularization can limit adaptation, and architecture growth consumes additional capacity.

Comparisons should report those costs along with retention and new-task learning.

Explore more "Explainers"

Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.