TensorTau PathToAGI Doing Machine Learning Honestly Level 05

Level 05 · Doing Machine Learning Honestly · Chapter 070

Random Seeds and Reproducibility

If training contains randomness, what does it mean to reproduce an ML result?

Randomness enters data shuffling, train/validation splitting, parameter initialization, augmentation, dropout and sometimes low-level hardware kernels.

1. A seed controls a random-number sequence

import random, numpy as np, torch
random.seed(42)
np.random.seed(42)
torch.manual_seed(42)

This makes many sources of pseudo-randomness repeatable. It does not guarantee bit-for-bit reproducibility on every device/library/version.

2. A seed is not a scientific conclusion

If seed 42 produces 92% and seed 17 produces 88%, reporting only seed 42 hides instability. Fixing a seed is useful for debugging; evaluating multiple seeds is useful for evidence.

3. Reproducibility needs context

Record code commit, dataset/version, environment, dependencies, preprocessing, hyperparameters, random seeds and hardware-sensitive settings when relevant.

A metric without provenance is difficult to trust.

4. Determinism can cost performance

Some deterministic kernels may be slower. Decide whether debugging, auditing or exact repeatability justifies the trade-off.

What to Remember

A seed helps replay randomness. Reproducibility means preserving enough of the whole experiment that another run can meaningfully test the same claim.

Next → Why One Training Run Is Not Evidence: what if seed-to-seed variation is larger than the improvement we claim?

Six ways to work the chapter, climbing from explaining it in your own words to investigating it like a researcher. Reading is not enough — this is where the chapter lands.

  1. Explain: Reproducibility versus determinism.
  2. Calculate: Summarize mean/range of scores from five seeds.
  3. Predict: Name four operations whose randomness a seed may affect.
  4. Break: Remove seed control and measure run-to-run variation.
  5. Build: Write an experiment config that records seed, versions and hyperparameters.
  6. Investigate: Test reproducibility across CPU/GPU or library versions if available.