1. Shattering
A hypothesis class shatters a set of points if, for every possible assignment of binary labels to those points, some hypothesis in the class realizes that labeling.
For three points there are possible binary labelings.
VC dimension is the size of the largest set that the hypothesis class can shatter.
2. A line in 1-D
A single threshold classifier on a line cannot realize every arbitrary labeling of two ordered points. A richer interval classifier can realize more patterns. Capacity depends on what decision sets the class can express—not just how many numbers happen to be stored.
3. Why the idea matters
If a class can fit extremely many arbitrary label patterns, excellent training fit is weaker evidence that it discovered real structure. Statistical learning theory turns this intuition into generalization bounds involving capacity and sample size.
4. Do not misuse VC dimension
For modern deep networks, classical VC bounds are often too loose to explain practical generalization. The concept remains valuable because it teaches a deep principle:
we must reason about the richness of the set of functions a learner can choose from.
Optimization, norms, architecture and data structure further constrain what neural networks actually learn.
What to Remember
VC dimension formalizes a notion of expressive capacity through shattering. It is a theoretical lens, not a practical score to compute for every model.
Next → Random Seeds and Reproducibility: even if our theory is sound, can another run reproduce the result we report?