1. Let every example take a turn
In -fold cross validation, split the training data into folds. Train on folds and validate on the remaining fold. Repeat until every fold has served as validation once.
run 1: V T T T T
run 2: T V T T T
run 3: T T V T T
...Average the validation scores.
2. Why it helps
One train/validation split can be lucky or unlucky. Cross validation exposes the model to several different held-out subsets and gives us both a mean score and a sense of variability.
3. The split must respect the task
Ordinary random K-fold is wrong for some problems. Use stratified folds for imbalanced classification, group-aware folds when records from one entity must stay together, and time-aware validation for forecasting.
The validation design should mimic deployment.
4. Preprocessing belongs inside each fold
For every fold, scalers, imputers, feature selectors and PCA must be fitted only on that fold’s training portion. Otherwise information leaks from its validation portion.
5. Cross validation is not the final test
Use cross validation for model selection and uncertainty during development. Keep a final test set untouched when an unbiased final estimate matters.
What to Remember
Cross validation reuses scarce training data efficiently while repeatedly testing on unseen folds. Its power depends on choosing folds that respect real-world independence.
Next → Generalization: what do we really mean when we say performance on sampled data tells us something about unseen data?