<!-- NOTEBOOK-LAB-NAV -->
## 🧪 Interactive Lab
The matching notebook is the hands-on laboratory for this lesson. It should be run after reading the mathematical example below.
## 🧭 Where this lesson fitsPrevious lesson: Blog 20 — Build a Tiny Neural Network From Scratch.
Today: Blog 21 — Automatic Differentiation.
Next lesson: Blog 22 — Can a Neural Network Learn Any Shape?.
Student rule: understand the story first, then the equation, then the code.
## 1. The idea in one sentence
A computer can turn a chain of tiny operations into a chain of local derivatives.
## 2. Explain it like I am in Class 8
Imagine you are learning a new game. You first notice a pattern, then describe that pattern with a few numbers, and finally test whether your description still works when the example changes. Deep learning does the same thing, but the description can contain millions of learned numbers.
**Three questions:**
1. What information goes in?
2. What transformation turns it into a useful representation?
3. What evidence tells us whether the transformation worked?
## 3. The mathematics
For z = x^2 and L = 3z, dz/dx = 2x and dL/dx = 3(2x). The chain rule multiplies local slopes.
Do not memorize the symbols yet. Translate every symbol into a sentence. In the notebook, we use tiny arrays so every multiplication, average, similarity, or probability can be inspected.
## 4. Hand calculation
Before running Python, make a two- or three-number example yourself. Write every intermediate value. Then run the notebook and compare your answer with the program.
This is an important rule for this curriculum: **the computer checks your reasoning; it does not replace your reasoning.**
## 5. Build the smallest experiment
Use finite differences to check an analytical derivative, then compare the same derivative with PyTorch autograd.
The matching notebook implements this as a reproducible experiment with NumPy and, where appropriate, PyTorch. Change exactly one variable first. Predict the result before running the cell.
## 6. What can go wrong?
**Failure mode:** Change the finite-difference step size. Very large or very tiny steps can make the numerical estimate worse.
Also check:
- shapes and dimensions;
- numerical stability and finite values;
- whether the metric actually measures the intended behavior;
- whether the result changes when the random seed or data split changes.
## 7. From intuition to research
A research question is a question whose answer is not known from the lesson alone. Examples:
- Which assumption in this method matters most?
- What happens when that assumption is deliberately broken?
- Which metric changes first when the model fails?
- Does the result survive a different dataset, seed, or architecture?
For graduate work, turn one question into a falsifiable hypothesis, define a baseline, predefine the metric, run controlled ablations, and report uncertainty rather than only the best run.
## 8. Mini-project
Build a toy version using at least two competing methods. Keep the data fixed, change only the method, and record the result in a small table containing **hypothesis, setup, metric, result, interpretation**.
## 9. Mastery gate
You are ready to continue when you can:
- explain the idea without using jargon;
- calculate the tiny example by hand;
- run and modify the notebook;
- explain one failure mode;
- connect the lesson to the next chapter;
- propose one experiment that could prove your explanation wrong.
## What to remember
> **Deep learning is not a collection of APIs. It is mathematics, computation, experiments, and evidence connected into one learning process.**