Functions and Slopes

Where We Are

Up to this point, most of our mathematics described structure.

Vectors told us how to represent many numbers together.

Matrices told us how to transform vectors.

Eigenvectors and SVD told us which directions matter inside a transformation.

PCA told us which directions matter inside data.

But learning is not only about structure.

Learning is about change.

When a parameter moves, the prediction changes.

When the prediction changes, the loss changes.

If we want a machine to improve itself, we need a mathematical language for questions like:

If I increase this input a little, what happens to the output?

Is the output rising or falling?

Is it changing slowly or rapidly?

Does the rate of change stay constant, or does it itself change?

Those questions begin with two ideas that look simple but will power the entire rest of deep learning:

functions and slopes.

1. The Problem: A Table of Answers Is Not Yet a Rule

Return to the house-pricing story.

Suppose we observe:

We can store the table.

But in Chapter 001 we wanted something more powerful than memorization.

We wanted a rule:

This rule does something important.

It accepts an input and produces an output.

If

then

If

then

The rule is a machine that maps one number to another.

Mathematics gives such a machine a name:

a function.

2. A Function Is a Relationship, Not a Formula Shape

We often write

Read this as:

“$y$ is the output produced by the function $f$ when the input is $x$.”

The symbol $f$ is not multiplication.

It is a name for a rule.

For example,

Then

Another function could be

Then

Another could be

That one will later become ReLU.

The important idea is not the formula.

It is the mapping:

input  →  rule  →  output

3. Functions as Machines

A useful mental model is a box.

      ┌───────────────┐
 x ──▶│   function f  │──▶ f(x)
      └───────────────┘

For

the box performs:

input x
   ↓
multiply by 2
   ↓
add 1
   ↓
output

So for $x=4$:

4 → 8 → 9

That is already the seed of a computational graph.

Later, neural networks will simply be much larger function boxes made from many smaller operations.

4. Inputs and Outputs Have Meaning and Units

Suppose

models house price.

Here:

• $x$ = rooms,
• $f(x)$ = price in ₹ lakh.

The number 2 therefore has units:

This matters.

A slope is not merely “2.”

It tells us how much output changes per unit input.

Unit awareness is one of the easiest ways to catch nonsense mathematics.

If the input is seconds and the output is metres, slope has units metres per second.

If the input is weight and the output is loss, slope has units loss per unit weight.

5. A Graph Is a Picture of a Function

A function can be represented in several equivalent ways.

Rule

Table

Graph

f(x)
 ↑
 9|                •
 7|            •
 5|        •
 3|    •
 1|•
 0+----------------------→ x
    0   1   2   3   4

The graph is not decoration.

It lets us see change.

A steeply rising graph means output changes quickly as input increases.

A flat graph means output barely changes.

A falling graph means output decreases as input increases.

6. The First Question About Change

Suppose we move from

to

For

output moves from

to

Input change:

Output change:

The Greek letter $\Delta$ means “change in.”

So

means change in x,

and

means change in y.

Now ask:

How much output change occurred per unit input change?

Compute

That number is the slope between the two points.

7. The Slope Formula

For two points

and

slope is

or more compactly,

Read it literally:

change in output divided by change in input.

8. Rise Over Run

Geometry gives a second interpretation.

 y
 ↑
 9|                 • B
 8|                /|
 7|               / |
 6|              /  | rise = 6
 5|             /   |
 4|            /    |
 3|    • A----+     |
 2|      run = 3
 1|
 0+----------------------→ x

Moving from A to B:

• horizontal movement = run,
• vertical movement = rise.

So

This is the same mathematics as rate of change.

Geometry and units are telling the same story.

9. Positive, Negative, Zero and Steep Slopes

Positive slope

means output rises as input increases.

 /
/

Negative slope

means output falls as input increases.

\
 \

Zero slope

means output does not change.

──────

Larger magnitude

Compare

with

The second changes ten output units per input unit.

So slope carries two kinds of information:

• sign → direction of change,
• magnitude → speed/strength of change.

This exact interpretation will later become the meaning of a gradient.

10. Why Linear Functions Are Easy

Consider

Take any two points.

From $x=0$ to $x=1$:

From $x=1$ to $x=3$:

From $x=-5$ to $x=10$:

The slope is always 2.

That is what makes the graph a straight line.

For

$m$ is the constant slope.

$b$ shifts the line vertically but does not change steepness.

11. A Tempting Wrong Idea: “Slope Is Just the Weight”

For a simple linear model,

slope with respect to $x$ really is $w$.

That can create a dangerous intuition:

“Slope is always just the number multiplying x.”

Not true.

Consider

There is no single constant multiplying $x$.

The graph bends.

Its steepness changes depending on where we are.

A Tempting Wrong Idea

A straight line has one slope everywhere. A curve does not. For curves, “the slope” is incomplete unless we say where or over which interval.

That is the problem that will force derivatives.

12. Curves Change Their Rate of Change

Take

Some values:

Look at output changes for equal input steps of 1:

0 → 1 : +1
1 → 4 : +3
4 → 9 : +5
9 →16 : +7

The function is getting steeper.

So a curved function has a changing rate of change.

13. Average Slope on a Curve

For

find average slope from $x=1$ to $x=3$.

So

This number is correct—but what does it mean?

It summarizes the entire interval from 1 to 3.

It does not say the curve has slope 4 at every point.

The curve is less steep near 1 and more steep near 3.

14. The Secant Line

A line connecting two points on a curve is called a secant line.

 y
 ↑
 9|               • B
 8|             / |
 7|           /   |
 6|         /     |
 5|       /       |
 4|      curve
 3|    /
 2|  /
 1|• A
 0+----------------------→ x
    1           3

Its slope is

So average rate of change = slope of secant line.

This is our first bridge from algebra to local geometry.

15. Same Starting Point, Smaller Intervals

Now focus on the curve

at

We want to understand how steep it is near that point.

Start with a wide interval from 2 to 3:

Use a smaller interval from 2 to 2.5:

Smaller again, 2 to 2.1:

Smaller:

A pattern is emerging:

The secant slopes seem to be approaching 4.

We are not yet defining a derivative.

We are only observing a geometric phenomenon:

as the second point moves closer to the first, the average slope stabilizes toward a local steepness.

Chapter 018 will make that idea rigorous.

16. Why We Cannot Simply Use the Same Point Twice

A natural attempt is:

“If I want slope at $x=2$, just use both points at $x=2$.”

But then

And

So the slope formula becomes

Undefined.

This is not a small technical problem.

It is the central paradox that gives birth to calculus.

We need the slope at one point, but ordinary slope needs two distinct points.

Chapter 018 will solve exactly that problem.

17. Local Straightness

Curves often look straight when you zoom in far enough.

Imagine a huge circle.

From far away you see curvature.

Stand on a tiny patch and it looks almost flat.

The same happens with smooth function graphs.

far view:           zoomed view:

     ___                 /
   /     \               /
  /       \             /

This idea—local linearity—is one of the deepest reasons calculus works.

Near a point, many smooth nonlinear functions can be approximated by a line.

Later we will write this as a linear approximation.

Neural-network optimization relies on this constantly.

18. Why Slopes Matter for Learning

Suppose a model has one parameter $w$.

Its loss is

Evaluate a few values:

Graphically:

L(w)
 ↑
 9|•           •
 8|
 7|
 6|
 5|
 4|  •       •
 3|
 2|
 1|    •   •
 0|      •
  +----------------→ w
         3

We want the machine to move toward

What information would help?

If we are at $w=1$, the curve falls as $w$ increases.

If we are at $w=5$, the curve rises as $w$ increases.

So local slope tells us which way lowers the loss.

This is why derivatives will become the machine’s compass.

19. Sign of Slope as a Direction Signal

Suppose we move slightly to the right.

Positive slope

loss rises → moving right is bad

To reduce loss, move left.

Negative slope

loss falls → moving right is good

To reduce loss, move right.

Zero slope

Locally flat.

This may indicate a minimum, maximum, saddle, or plateau—we will learn the differences later.

The key optimization idea is already visible:

To go downhill, move opposite the slope.

In one dimension this is simple.

In millions of dimensions we will replace slope with gradient.

20. Magnitude of Slope as Sensitivity

Suppose two functions at a point have slopes:

and

A tiny input change affects the second output far more strongly.

So slope is also a measure of sensitivity.

This interpretation is useful across machine learning:

• how sensitive is loss to a weight?
• how sensitive is output to a pixel?
• how sensitive is prediction to a feature?
• how sensitive is a layer to its input?

Backpropagation will eventually compute these sensitivities throughout a network.

21. Units of Slope Are a Debugging Tool

Suppose

is loss as a function of weight.

Then slope has units

If prediction is price and input is square feet, slope may be

Keeping units in mind can reveal mistakes.

For example, adding a slope directly to a prediction is often dimensionally nonsensical.

The slope tells how one changes with respect to another.

22. Domain and Range

Functions do not always accept every imaginable input.

The set of allowed inputs is the domain.

The set of possible outputs is the range.

Example:

For real-valued arithmetic, $x$ must satisfy

So domain matters.

Another example:

$x=0$ is forbidden.

Why does this matter in ML?

Because functions such as logarithms appear in loss functions:

requires

Understanding domains prevents mysterious NaNs later.

23. One Input vs Many Inputs

So far we have used

One input number.

But a model usually has many parameters:

Or compactly,

where

collects all parameters.

Then “the slope” is no longer one number.

We will need to ask:

How does loss change if I move $w_1$ while holding everything else fixed?

What about $w_2$?

What about $b$?

Those questions lead to partial derivatives and gradients in Chapters 019–020.

But first we must master the one-variable case.

24. Composition of Functions

Functions can feed into one another.

Suppose

and

Then

means:

x
↓
f
↓
2x+1
↓
g
↓
(2x+1)^2

This is function composition.

Notice the parallel with matrices:

Deep neural networks are compositions of functions:

This is why understanding how change flows through composition will eventually force the chain rule.

25. A Tiny Neural-Network Preview

Consider one neuron:

followed by activation

Then a loss

This is a composition:

w
↓
z = wx+b
↓
a = ReLU(z)
↓
L = (a-y)^2

If $w$ changes slightly, $z$ changes.

That changes $a$.

That changes $L$.

Eventually we need a systematic way to trace these changes backward.

That will become backpropagation.

The whole story starts with slope.

26. Discrete Change vs Continuous Change

Not every function has a smooth graph.

Suppose

This function jumps.

There is no gradual transition at zero.

Another example is integer-valued room count.

A real house cannot have exactly 2.371 rooms in the usual sense.

Yet mathematical models often extend discrete variables into continuous space because calculus then becomes available.

That extension can be useful, but we should remember the modelling assumption.

27. Smoothness Preview

Chapter 001 discussed why squared error feels smoother than absolute error.

Now we can phrase that geometrically.

For

the slope changes gradually as x moves.

For

the graph has a corner at zero:

\ /
 V

The slope from the left is negative.

The slope from the right is positive.

There is no single obvious line that matches both sides at the corner.

Chapter 018 will give the precise language for this.

28. How to Read a Function Graph Like an Optimizer

When you see a graph, ask four questions.

1. Is it rising or falling?

That tells sign of slope.

2. How steep is it?

That tells magnitude of slope.

3. Is steepness changing?

That tells whether the function is curved.

4. Where does slope appear to become zero?

Those locations may be important optima or flat regions.

This is much more useful than memorizing graph shapes.

29. Shape Check

Even scalar calculus has shapes.

For

we have:

input x        scalar
output f(x)    scalar
Δx             scalar
Δf             scalar
Δf / Δx        scalar

Later:

will produce a gradient with $n$ components.

Thinking about input/output structure now will make multivariable calculus much easier.

30. Code: Sample a Function

def f(x):
    return x * x

for x in [0, 1, 2, 3, 4]:
    print(x, f(x))

The important idea is that code mirrors mathematical function notation:

f(3)

means “feed 3 into the rule named f.”

31. Code: Average Rate of Change

def average_slope(f, x1, x2):
    return (f(x2) - f(x1)) / (x2 - x1)


def square(x):
    return x ** 2

print(average_slope(square, 1, 3))  # 4.0

Now shrink the interval near 2:

for h in [1, 0.5, 0.1, 0.01]:
    print(h, average_slope(square, 2, 2 + h))

Before running, predict the pattern.

You should expect values:

5
4.5
4.1
4.01

Do not yet call 4 “the derivative.”

For now, say:

the nearby average slopes are converging toward 4.

Chapter 018 will explain why that statement can become an exact definition.

32. Break It: Vertical Change With Zero Horizontal Change

Try to compute slope between

and

Then

So

Undefined.

Geometrically, the line is vertical.

This reminds us:

slope is fundamentally “output change per input change.” If there is no input change to divide by, ordinary slope is undefined.

33. Break It: Large Average Interval Hides Local Behaviour

Consider

From $x=-1$ to $x=1$:

Average slope is

Does that mean the function is flat throughout the interval?

No.

It rises and falls inside.

A single average slope can hide rich local behaviour.

This is exactly why optimization needs local rates of change.

34. Break It: Correlation Is Not Slope

Suppose two variables tend to increase together.

That is a statement about association across a dataset.

Slope describes the change of an output with respect to an input inside a function or fitted relationship.

They can be connected, but they are not interchangeable.

Later, statistics will make this distinction precise.

35. History Lens — From Motion to Calculus

Ancient mathematicians studied tangents and changing quantities long before modern notation existed.

By the seventeenth century, problems in astronomy, mechanics and geometry made the need urgent:

• how fast is a planet moving at an instant?
• what is the tangent to a curve at one point?
• where does a curve reach a maximum or minimum?

Newton and Leibniz independently developed systematic calculus methods in the late seventeenth century, building on substantial earlier work by mathematicians such as Fermat, Barrow, Descartes, Cavalieri and others.

The historical pressure was the same one we now face in machine learning:

average change is not enough when the behaviour changes continuously from point to point.

36. Distinctions That Matter

37. What We Discovered

1. A function maps inputs to outputs.
2. A graph visualizes that mapping.
3. Change is represented with differences such as $\Delta x$ and $\Delta y$.
4. Average slope is

5. Slope sign gives direction of change.
6. Slope magnitude gives sensitivity or steepness.
7. Linear functions have constant slope.
8. Nonlinear functions have changing slope.
9. Average slope on a curve is the slope of a secant line.
10. Shrinking the interval reveals a stable local steepness for smooth functions.
11. Using the exact same point twice creates the undefined expression $0/0$.
12. Local slope is precisely the information optimization needs to know which way loss changes.
13. Function composition prepares us for the chain rule and backpropagation.

38. Mathematics We Built

Change in input

Change in output

Average rate of change

Linear function

where $m$ is constant slope.

Function composition

No derivative formula has been assumed yet.

We have earned the need for one.

39. What Each Symbol Means

40. One-Minute Explanation

A function is a machine that maps an input to an output. To understand how the output changes, compare two input points. The change in output divided by the change in input is the average slope, or rate of change, between them. Straight lines have the same slope everywhere. Curves do not: their steepness changes from place to place. If we want the local steepness near one point, we can move a second point closer and closer and watch the secant slopes stabilize. We cannot simply use the exact same point because that would require dividing $0$ by $0$. Solving that paradox rigorously is the job of the derivative.

41. Mastery Check

You are ready for Chapter 018 only if you can answer these without looking back:

1. What is a function in plain English?
2. What is the difference between $f$ and $f(x)$?
3. What does $\Delta$ mean?
4. Why does slope have units?
5. What do the sign and magnitude of slope tell us?
6. Why does a straight line have one slope everywhere?
7. Why does $x^2$ not have one global slope?
8. What is a secant line?
9. Why can average slope hide local behaviour?
10. What happens to secant slopes of $x^2$ near $x=2$ as the interval shrinks?
11. Why can we not compute slope at a single point by plugging the same point twice into the ordinary slope formula?
12. Why does optimization care about local slope?
13. How does function composition foreshadow neural networks and the chain rule?

🔭 Bridge to Chapter 018

We now know exactly what we want.

For a curve such as

we want the exact local slope at one point.

Ordinary slope needs two points.

If we collapse them into one, we get

Yet shrinking the gap suggests a perfectly stable answer is hiding there.

How can we obtain the slope at a single instant without ever dividing by zero?

That question forces the invention of the derivative.