Conv output shapes and parameter counts

Before you run a convolution, you need two numbers. The first is the output size, which decides the shape of everything after it. The second is the parameter count, which tells you how big the layer is. Three settings control the output size: kernel size k, stride s and padding p.

The output-size formula

For one spatial dimension:

out = floor((in + 2p − k) / s) + 1

Worked examples, all on a 28-pixel input:

• Kernel 3, padding 1, stride 1: floor((28 + 2 − 3) / 1) + 1 = 28. The output keeps the input size.
• Kernel 3, padding 1, stride 2: floor((28 + 2 − 3) / 2) + 1 = 14. The output is half the size.

A stack of 28 → 14 → 7 is common in small CNNs, where a stride-2 layer and a pooling layer each halve the grid.

Counting parameters

A convolution layer has C_out × C_in × k × k weights, plus one bias per output channel. Take Conv2d(1, 32, kernel_size=3, padding=1) on a grayscale 28 × 28 image:

weights = 32 × 1 × 3 × 3 = 288
biases  = 32
total   = 320
output  = (batch, 32, 28, 28)

Compare that with a dense layer on the flattened image. A 784 → 32 layer needs 784 × 32 + 32 = 25,120 parameters, about 78 times as many, and it ignores the local structure of the image.

The second convolution, Conv2d(32, 64, 3, padding=1), has 64 × 32 × 3 × 3 = 18,432 weights plus 64 biases. After two pooling layers the map is (batch, 64, 7, 7), which flattens to 3,136 features.

The classifier head, Linear(3136, 10), has 3,136 × 10 + 10 = 31,370 parameters. In this small network the head is the largest dense block. That is one reason modern CNNs use global average pooling before the head, which turns each channel into one number.

The compute cost

The number of multiply-adds for a convolution is roughly H' · W' · k · k · C_in · C_out, times the batch size. Spatial size and channel count both matter. If you double the channel count of every layer, each layer's cost goes up by four, because both C_in and C_out double.

Common mistakes

• Forgetting the biases. Each output channel has one bias. They are a small part of the count, but they belong in it.
• Mixing up padding and stride. Padding changes the border. Stride changes how far the kernel jumps.
• Assuming the head is small. A flatten followed by a large linear layer often holds most of the parameters.

Check yourself

1. A 28-pixel input has kernel 5, padding 2 and stride 1. What is the output size? floor((28 + 4 − 5) / 1) + 1 = 28, so the size is unchanged.
2. Why is the Linear(3136, 10) layer so large? It connects every flattened feature to every class, which is 3,136 × 10 weights.

Key takeaways

• Output size is floor((in + 2p − k) / s) + 1 per dimension.
• Padding of (k − 1) / 2 with stride 1 keeps the size for odd kernels.
• A convolution has C_out × C_in × k × k weights, and far fewer than a dense layer on the same image.
• Flattening a large map into a dense head can dominate the parameter count.

Further learning

Skip connections make very deep stacks of these layers trainable: residual connections.