What Is a Residual Connection?

Residual Connection Explainer

A residual connection is a shortcut that adds an earlier representation to the output of one or more later layers. Instead of requiring a block to learn an entirely new mapping, the block can learn a residual change relative to its input.

The simplest form adds the input directly when dimensions match. If shapes differ, a projection or other adjustment may align them. The addition creates a short path through which information and gradients can travel across a deep network.

Residual networks showed that extremely deep image models could be optimized more effectively than comparable plain stacks. The idea now appears in transformers, speech systems, and many other architectures, often combined with normalization and attention or feedforward blocks.

A shortcut does not guarantee a better model. Its placement, scaling, normalization order, and dimension choices affect stability and computation. Designers still need controlled experiments because a residual path can preserve unhelpful information as easily as useful information.

A residual connection adds an earlier representation to a later layer, creating a shortcut through a neural network.

Key terms include shortcut connection, residual block, identity mapping, projection, gradient flow, deep network, skip path, dimension and normalization.

The block learns how its output should differ from the input, while the shortcut carries the original representation forward.

If the best transformation is close to identity, the residual branch can learn a small correction instead of rebuilding the full mapping.

Short paths help signals and gradients cross many layers, reducing the optimization difficulty of a plain deep stack.

Architecture details still matter, including when normalization occurs and how mismatched dimensions are projected.

Explore more "Explainers"

Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.