1. Stack examples into a matrix
One example may be a feature vector [D]. Stack examples:
A dense layer then computes
for every example simultaneously.
X [B,D] @ W [D,H] → [B,H]
+ b [H]Broadcasting adds the same learned bias to each row.
2. Why batches are efficient
Modern CPUs/GPUs are optimized for large vectorized matrix operations. One matrix multiplication usually uses hardware far better than thousands of tiny Python loops.
Batches also create the gradient estimate used by stochastic optimization.
3. Per-example versus reduced loss
Suppose predictions and targets have shape [B,C]. Cross-entropy can first produce one loss per example [B], then reduce:
loss = losses.mean()If we average, parameter gradients represent the average contribution of the batch. If we sum, gradient magnitude grows roughly with batch size.
4. Shape Check: images
PyTorch convolution normally expects
[B,C,H,W]A batch of 32 RGB 224×224 images is
[32,3,224,224]Batch is an axis, not a neural-network layer.
5. Batches are also statistics
Batch size affects gradient noise, memory usage, throughput and sometimes normalization. We will study mini-batches explicitly in Chapter 92.
What to Remember
Batch computation stacks examples along a new axis and lets tensor operations process them together. Trace shapes before code.
Next → Build a Neural Network Without Autograd: can we now wire forward pass, loss and manual backward rules into a complete learner?