063 — Baselines · TensorTau

1. Start embarrassingly simple

Before a giant neural network, try a useful baseline:

mean/median prediction for regression;
majority or stratified class predictor;
logistic regression;
a small tree;
a heuristic already used by humans.

A baseline gives the result meaning.

2. Baselines are debugging tools

If a sophisticated model cannot beat logistic regression, possible explanations include weak features, implementation bugs, insufficient data, incorrect metric, optimization failure—or simply that the relationship is nearly linear.

That is valuable information.

3. Strong baselines matter

Beating a deliberately bad baseline proves little. Research compares against credible prior methods under fair settings.

The baseline should receive appropriate tuning and the same evaluation protocol whenever possible.

4. A hierarchy is better than one baseline

naive constant
   ↓
simple heuristic
   ↓
linear/classical model
   ↓
current strong method
   ↓
proposed method

Each rung tells us where complexity begins paying for itself.

What to Remember

A result without a meaningful comparison has no scale. Baselines turn “92%” into an interpretable claim.

Next → Error Analysis: once we know the model is imperfect, which mistakes should we fix first?