Convolution and Pooling

A convolution reuses one kernel across spatial locations. Its output records where a learned local pattern appears; pooling summarizes a neighborhood. Neither operation automatically understands objects.

Prerequisites

Complete Images as Tensors. Be able to read (N, C, H, W).

Learning goals

• calculate convolution output shapes from kernel, stride, dilation, and padding.
• distinguish input channels, output channels, and spatial axes.
• explain what max pooling keeps and discards.

Worked example

The input has one batch and one channel. A fixed 3-by-3 vertical-edge kernel slides over it with no padding and stride one. For one spatial axis,

$$H_{out}=\left\lfloor\frac{H_{in}+2p-d(k-1)-1}{s}+1\right\rfloor.$$

With $H_{in}=5$, $k=3$, $p=0$, $d=1$, and $s=1$, both output spatial axes equal 3.

import torch
from torch import nn

image = torch.tensor(
    [[[[0., 0., 1., 1., 1.],
       [0., 0., 1., 1., 1.],
       [0., 0., 1., 1., 1.],
       [0., 0., 1., 1., 1.],
       [0., 0., 1., 1., 1.]]]]
)
edge = nn.Conv2d(1, 1, kernel_size=3, bias=False)
with torch.no_grad():
    edge.weight.copy_(torch.tensor([[[[-1., 0., 1.], [-1., 0., 1.], [-1., 0., 1.]]]]))
response = edge(image)
pooled = nn.MaxPool2d(kernel_size=2, stride=1)(response)

print(f"input={tuple(image.shape)} kernel={tuple(edge.weight.shape)} response={tuple(response.shape)}")
print(f"response_rows={response[0, 0].tolist()}")
print(f"pooled={tuple(pooled.shape)} values={pooled[0, 0].tolist()}")

Output:

input=(1, 1, 5, 5) kernel=(1, 1, 3, 3) response=(1, 1, 3, 3)
response_rows=[[3.0, 3.0, 0.0], [3.0, 3.0, 0.0], [3.0, 3.0, 0.0]]
pooled=(1, 1, 2, 2) values=[[3.0, 3.0], [3.0, 3.0]]

PyTorch implements cross-correlation: it does not flip the stored kernel. Deep-learning literature usually still calls this convolution. Padding can preserve spatial size, stride downsamples, dilation spaces kernel taps, and groups change channel connectivity. A (C_out, C_in/groups, K_h, K_w) weight tensor couples channel and spatial decisions.

Common mistakes

• Forgetting the batch axis or swapping height and width.
• Assuming padding="same" is meaningful with every stride and expecting it to prevent boundary effects.
• Treating pooling as harmless invariance; it discards location and small-detail information.
• Interpreting a hand-designed edge response as a learned or semantically complete feature.

Practice

Set stride to 2 for the convolution and calculate the output shape before running. Then reverse the bright and dark halves and predict the response sign.

Next steps

Continue with Convolutional Neural Networks, where learned channels compose local features.

Primary sources

Checked 2026-10-16: PyTorch Conv2d, PyTorch MaxPool2d, and LeCun et al., gradient-based learning applied to document recognition.