Inputs, Predictions, Errors and Parameters

Where We Are

In Chapter 001, four houses and four prices taught us the central idea of machine learning:

We do not write the rule. We write a process that can discover a rule from examples.

We ended with a tiny model:

and a tiny mystery.

Some numbers in this equation come from the world. Some are produced by the model. Some are adjusted during learning. And one number tells us how badly the model is doing.

If we mix these roles together, every later idea — gradients, backpropagation, neural networks, transformers — becomes confusing.

So before adding more mathematics, we need a clean map.

Today: we identify the four kinds of quantities that appear in almost every machine-learning system: inputs, predictions, errors/losses, and parameters.

Next: once we can name every moving piece, we can ask a sharper question: how should we measure wrongness?

1. The Problem: The Same Number Can Mean Very Different Things

Return to our house-price example.

Suppose we know:

A model tries to predict price using

Take one house: three rooms.

Let the model currently have

Then

But the true price is

On the page we now see the numbers

3, 1, 0, 3, 7, -4, 16

They are all just numbers.

But they do completely different jobs.

• one 3 describes the house,
• 1 and 0 describe the model,
• another 3 is the model’s answer,
• 7 comes from reality,
• -4 describes the mistake,
• 16 can describe the cost of that mistake.

If we do not distinguish these roles, learning looks like random arithmetic.

So let us build the vocabulary from the problem instead of memorizing definitions.

2. What Would a Useful Map Need?

We need a way to separate the quantities into roles that answer four questions:

1. What did we observe?
2. What did the model say?
3. How wrong was it?
4. What can the model change?

Those four questions will survive all the way from this tiny line to a billion-parameter neural network.

The names are:

Do not memorize the table yet. We will earn every row.

3. Input: The Information the Model Is Allowed to See

For our three-room house,

The number 3 is the input.

It is the information we give the model before asking for a prediction.

Intuition — The input is the question paper handed to the model.

If the model is pricing houses, the input might eventually include rooms, area, age and location.

If the model is recognizing a dog, the input might be image pixels.

If the model is predicting the next word, the input might be token IDs representing the previous words.

Different applications, same role:

WORLD → input → MODEL

A subtle point: input is not “everything we know”

Suppose you secretly know the true sale price while making the prediction.

Can you give that price to the model as an input?

Of course not. Then the task becomes trivial.

This leads to a rule that matters enormously later:

The input contains only information that would genuinely be available when the prediction must be made.

Otherwise we create data leakage — a topic we will revisit deeply in Level 5.

4. Target: The Answer the World Gives Us

For the three-room house,

This is the target, also called the label in many supervised-learning problems.

It is the answer we want the model to learn to predict.

Notice the timeline:

input x = 3
      ↓
model makes a guess
      ↓
prediction ŷ
      ↓
compare against target y = 7

The target does not help the model make this prediction. It is used afterward to judge the prediction during training.

Think — If the target were available before prediction, why would we need a model at all?

This distinction becomes especially important when we later split data into training, validation and test sets.

5. Parameters: The Model’s Adjustable Memory

Now look at

These numbers are not facts about the house.

They are numbers inside the model.

We call them parameters.

For our line,

there are two parameters:

• $w$ — the weight,
• $b$ — the bias.

Why call them parameters?

Because changing them changes the model’s behaviour.

Keep the same input, $x=3$.

The house did not change.

The target did not change.

Only the model’s internal dials changed.

And therefore the prediction changed.

Intuition — Parameters are the knobs the machine is allowed to turn while learning.

In a neural network there may be millions or billions of these knobs. The basic idea is unchanged.

6. First Tempting Wrong Idea: “The Input Is Also a Parameter”

A smart beginner may think:

“The input is a number in the formula. Why not call $x$ a parameter too?”

Because we do not learn $x$.

The house has three rooms whether the model likes it or not.

The model may change $w$ and $b$.

It may not change reality from three rooms to four rooms just to reduce its loss.

That gives us a clean distinction:

This one table prevents a surprising amount of confusion later.

7. Prediction: What the Current Model Believes

Given the current parameters and the input, the model computes

The symbol $\hat{y}$ is read “y-hat.”

The hat means:

this is our model’s estimate of $y$, not the true $y$ itself.

With

we get

The model has now made a prediction.

But nothing has learned yet.

This matters enough to say twice:

Prediction is not learning.

Prediction uses the current parameters.

Learning changes the parameters.

A calculator can make the same prediction a million times and never learn anything.

8. Error: The Direction and Size of One Mistake

The prediction is

The target is

A natural way to describe the mistake is

So

The error is negative.

What does the sign tell us?

The model predicted too low.

If instead

then

which says the model predicted too high.

So error contains two pieces of information:

1. magnitude — how far away,
2. sign — which side of the truth.

9. Error Is Not Yet a Loss Function

Here comes an important distinction.

Suppose we have four houses and four signed errors:

Average them:

Zero!

But every single prediction was wrong.

So signed error is useful for describing a mistake, but naive averaging lets opposite mistakes cancel.

That means we need another idea.

A loss function turns prediction and target into a number suitable for optimization.

For one example we might choose squared error:

For our example:

Now the mistake cannot disappear just because it points downward instead of upward.

A Tempting Wrong Idea — “Error and loss are the same thing.”

Not quite. Error is often the raw difference, such as $\hat{y}-y$. Loss is a designed objective that tells training how costly that mistake should be.

Different problems can use different losses even when the prediction error looks similar.

10. The Entire Training Loop in Seven Lines

We finally have all the pieces.

For one example:

1. Receive input x
2. Use parameters w, b
3. Compute prediction ŷ
4. Observe target y
5. Compute loss L
6. Decide how w and b should change
7. Update w and b

Or visually:

                 PARAMETERS
                  w      b
                   \    /
                    \  /
INPUT x ───────────→ MODEL ───────────→ PREDICTION ŷ
                                         │
                                         │ compare
                                         ▼
TARGET y ─────────────────────────────→ LOSS L
                                         │
                                         │ tells us how bad
                                         ▼
                                  update parameters

Everything later is an increasingly sophisticated version of this loop.

11. A Complete Hand Calculation

Let us run the loop once with actual numbers.

Given:

Step 1 — prediction

Step 2 — raw error

Step 3 — squared loss

So our state is:

The table is worth studying carefully. It is the miniature version of a deep-learning training step.

12. What Changes During Training?

Imagine the model updates itself to

Same house:

Same target:

New prediction:

New loss:

The loss fell from

What changed?

Only the parameter.

That is learning in its smallest visible form.

before: w = 1.0  → prediction 3.0 → loss 16

after:  w = 1.5  → prediction 4.5 → loss 6.25

The model became better because its adjustable internal state changed in a direction that reduced loss.

13. Parameters Versus Hyperparameters

There is another pair of terms beginners often confuse.

Parameters are learned by the model.

Examples:

• weights,
• biases.

Hyperparameters are choices we make about the learning process or architecture.

Examples:

• learning rate,
• batch size,
• number of layers,
• regularization strength.

For now, remember one sentence:

Parameters are learned inside training; hyperparameters configure training.

Later we will see that even this boundary can become subtle, but this distinction is excellent for building the foundation.

14. One Example Versus a Dataset

So far we used one house.

Real learning uses many examples.

Write the dataset as pairs:

For our four houses:

The model uses the same parameters for every example.

That is critical.

We do not learn a different $w$ for each house.

We try to find one rule that works across the dataset.

For every $i$:

Then each example produces a loss:

And we combine those losses into an overall objective such as the mean:

Now the learning problem becomes:

Find one set of parameters that makes the dataset’s overall loss small.

15. Why Shared Parameters Matter

Suppose instead we give each house its own private weight and bias.

House 1 gets $(w_1,b_1)$.

House 2 gets $(w_2,b_2)$.

And so on.

We could easily fit every training example perfectly.

But what happens when house 5 arrives?

There is no parameter set for it.

We memorized the training data instead of learning a transferable rule.

This gives us another deep principle:

Learning is powerful because the same parameters are forced to explain many examples.

This pressure is what can produce generalization.

We will return to that idea much later when studying overfitting and why neural networks generalize.

16. Geometry: Parameters Pick a Line

The equation

is not just arithmetic.

Every pair $(w,b)$ chooses a line.

Change $w$ and the line rotates.

Change $b$ and the line shifts up or down.

price
  ↑
  │                 /  w large
  │              /
  │           /
  │        /     w smaller
  │     /
  │  /
  └────────────────────────→ rooms

So parameters are not abstract bookkeeping.

They determine the shape of the function the model represents.

Training is therefore a search through a space of possible functions.

For this tiny model, the search space is all straight lines.

For a deep neural network, the search space is vastly richer — but parameters still choose the function.

17. The Same Roles Inside a Neural Network

Fast-forward to a future image classifier.

Suppose the input is a dog image.

The model contains millions of weights.

It outputs probabilities such as

dog: 0.20
cat: 0.70
bird: 0.10

The true target is dog.

Nothing fundamental has changed:

The scale changed.

The architecture changed.

The roles did not.

18. The Same Roles Inside a Language Model

Now jump even further.

Input:

"The sky is"

Target next token:

"blue"

Prediction:

P("blue") = 0.31
P("clear") = 0.12
P("falling") = 0.0004
...

Parameters:

billions of learned numbers inside embeddings, attention projections and feed-forward layers.

Loss:

how much probability the model failed to assign to the true next token.

Again:

input → parameters → prediction → compare with target → loss → update parameters

That loop is one of the deepest recurring patterns in modern AI.

19. Distinctions That Matter

20. What Each Symbol Means

21. One-Minute Explanation

A machine-learning system has four important roles.

The input is what the model is allowed to see.

The target is the answer reality gives us during training.

The model uses internal adjustable numbers called parameters to turn the input into a prediction.

We compare the prediction with the target and compute a loss.

Learning means changing the parameters so that future loss becomes smaller.

That is the basic training loop.

22. Common Mistakes

23. Socratic Questions

Do not look for answers immediately. Reason them out.

1. If $x=4$ and $y=9$, which quantity exists before the model runs?
2. If you change $w$ but keep $x$ fixed, what changes?
3. If you change $x$ but keep $w$ fixed, did the model learn?
4. Can two different parameter settings produce the same prediction for one input?
5. If they can, does one training example contain enough information to identify the correct parameters?
6. Why can signed errors cancel while squared errors cannot?
7. Why must the same parameters usually be used across many training examples?
8. Why is a test label not allowed to influence the model before evaluation?
9. In a language model, what is the target when predicting the next token?
10. What part of a neural network corresponds to the two tiny parameters $w$ and $b$?

24. What We Discovered

1. Inputs are information available to the model before prediction.
2. Targets are the answers used to judge predictions during supervised training.
3. Parameters are the model’s adjustable internal numbers.
4. Predictions are outputs produced by current parameters.
5. Errors describe differences between predictions and targets.
6. Loss functions convert mistakes into an optimization objective.
7. Learning means changing parameters so that loss tends to decrease.
8. The same parameters must explain many examples if we want a reusable rule rather than a lookup table.
9. These roles remain intact in neural networks and language models even when the scale becomes enormous.

25. Mathematics We Built

Model:

Raw error:

Squared loss for one example:

Prediction for example $i$:

Mean squared error across $n$ examples:

🔭 Bridge to Chapter 003 — Why Does a Machine Need a Loss Function?

We now know what every quantity does.

But we quietly made a major choice:

Why square the error?

Why not absolute error?

Why not signed error?

Why not something completely different?

And what properties must a useful loss function have if it is going to guide learning?

Next → We stop treating loss as “just a formula” and derive why a machine needs an objective at all.