Matrix-Matrix Multiplication as Composition

Where We Are

Chapter 009 showed that a matrix can act on a vector:

A deep network does not stop after one transformation.

It repeatedly applies transformations:

Writing nested expressions works for two layers. It becomes painful for twenty.

We need a way to combine transformations themselves.

Today: we discover matrix–matrix multiplication as function composition, many dot products, and “apply the right matrix first.”

Next: once matrices can represent transformations, we will study what makes a transformation linear and what geometry it preserves.

1. The Problem: Two Transformations in Sequence

Suppose we first stretch x-coordinates by 2:

Then shear horizontally:

Take

First apply $A$:

Then apply $B$:

Can one matrix $C$ do the whole job?

We want

for every vector $\mathbf{x}$, not only this one.

2. What Would the Combined Matrix Need to Do?

From Chapter 009, a matrix is determined by where it sends the basis vectors.

So to discover the combined matrix, ask what the two-step process does to

and

Whatever those final vectors are, they must become the columns of $C$.

This gives us a route to matrix multiplication without memorizing a rule.

3. Transform the First Basis Vector

Start with

Apply $A$:

Now apply $B$:

So the first column of the combined matrix is

4. Transform the Second Basis Vector

Now

Apply $A$:

Apply $B$:

So the second column is

Therefore

Check our original vector:

It matches the two-step process.

5. The Discovery: Matrix Multiplication

We write the combined transformation as

so that

This notation encodes order.

The matrix closest to the vector acts first.

That is exactly like function composition:

Matrix multiplication is transformation composition.

6. Why the Order Looks Backward

Suppose you read

Human language may tempt you to say “B then A.”

But parentheses reveal the truth:

$A$ touches $\mathbf{x}$ first.

Then $B$ acts on the result.

This is not an arbitrary convention. It follows from how function composition works.

7. Derive the Entry Formula

Let

The first column of $BA$ is $B$ applied to the first column of $A$:

The second column is

Therefore

Each entry is a row–column dot product.

8. The Row–Column Rule

To find entry $(i,j)$ of $BA$:

1. take row $i$ of $B$,
2. take column $j$ of $A$,
3. dot them.

Symbolically:

The repeated index $k$ is the inner dimension being summed over.

This formula looks abstract only until you remember:

one output entry = one dot product.

9. Shape Rule

Suppose

and

Then

Shape mnemonic:

(p × m)(m × n) = (p × n)
      ↑  ↑
      must match

The inner dimensions disappear because they are summed over in dot products.

The outer dimensions survive.

10. Why Matrix Multiplication Is Not Commutative

For ordinary numbers,

For matrices, usually

Why?

Because order of transformations matters.

Stretch then rotate is generally not the same as rotate then stretch.

That is a geometric fact, not a symbolic annoyance.

11. Concrete Example: Order Matters

Let

and a 90° rotation

Take

Stretch then rotate

then

Rotate then stretch

then

Different answers.

Therefore

12. A Tempting Wrong Idea: “Just Multiply Matching Entries”

Element-wise multiplication would give

But composition requires that each output coordinate of one transformation be fed into every relevant input of the next.

That mixing is exactly what row–column dot products achieve.

A Tempting Wrong Idea

Element-wise multiplication combines corresponding entries. Matrix multiplication composes transformations. They solve different problems.

13. Associativity: Why Deep Composition Is Manageable

Matrix multiplication satisfies

So parentheses can move without changing the final transformation.

This matters computationally.

Suppose shapes are:

Different parenthesizations can require very different amounts of work.

The mathematics is the same; the computational cost may not be.

This becomes important in optimized numerical computing.

14. Identity Matrix

The identity matrix does nothing:

For every vector,

And for compatible matrices,

It plays the same role as the number 1 in ordinary multiplication.

15. Inverse Matrix as “Undo”

If a matrix $A$ has an inverse $A^{-1}$, then

So if

then

The inverse undoes the transformation.

Not every matrix has one. Later, rank will explain why.

16. Neural-Network Connection

Ignoring nonlinearities for a moment, imagine two dense layers:

and

Substitute:

Therefore

Two purely linear layers collapse into one linear layer.

That observation will become crucial when we ask why neural networks need nonlinear activation functions.

17. Batch Computation Preview

Suppose each column of $X$ is one input vector.

Then

applies the same transformation to many examples at once.

Matrix–matrix multiplication therefore appears in two ways:

1. compose transformations,
2. process batches of vectors efficiently.

Modern deep-learning workloads are dominated by this operation.

18. Code From Scratch

def matmul(A, B):
    rows_a = len(A)
    cols_a = len(A[0])
    rows_b = len(B)
    cols_b = len(B[0])

    if cols_a != rows_b:
        raise ValueError("inner dimensions must match")

    out = [[0.0 for _ in range(cols_b)] for _ in range(rows_a)]

    for i in range(rows_a):
        for j in range(cols_b):
            total = 0.0
            for k in range(cols_a):
                total += A[i][k] * B[k][j]
            out[i][j] = total

    return out

The three loops have clear meanings:

• i: output row,
• j: output column,
• k: dot-product accumulation.

NumPy compresses all of that into

C = A @ B

but the mathematics underneath is unchanged.

19. Break It

Shape mismatch

A (3×4) matrix cannot multiply a (5×2) matrix because 4 ≠ 5.

Wrong order

Even when both $AB$ and $BA$ exist, they usually differ.

Assuming every matrix has an inverse

A transformation that squashes a plane onto a line loses information. No inverse can recover what was destroyed.

Forgetting nonlinearities

Two linear neural-network layers collapse into one. Adding nonlinear activations prevents this collapse and gives deep networks their expressive power.

20. History Lens — Composition as Algebra

One of linear algebra’s great achievements is turning sequences of geometric operations into algebraic objects that can themselves be multiplied.

Instead of separately describing “rotate, then stretch, then shear,” we can multiply matrices and obtain a single transformation.

That ability to compose operations is why matrices became fundamental across geometry, mechanics, graphics, control theory and neural networks.

21. Distinctions That Matter

22. What We Discovered

1. Matrix multiplication composes transformations.
2. In $BA\mathbf{x}$, $A$ acts first.
3. Each matrix-product entry is a row–column dot product.
4. Shape rule: $(p\times m)(m\times n)=(p\times n)$.
5. Matrix multiplication is associative but generally not commutative.
6. The identity matrix leaves vectors unchanged.
7. Inverse matrices, when they exist, undo transformations.
8. Multiple purely linear neural-network layers collapse into one matrix multiplication.
9. Matrix–matrix multiplication also powers batch computation.

23. One-Minute Explanation

A matrix transforms vectors. If one matrix acts and then another acts, the two transformations can be combined by multiplying the matrices. The rightmost matrix acts first, just like nested functions. Each entry of the product is a dot product between a row from the left matrix and a column from the right matrix. Order matters because transformations do not generally commute: rotate-then-stretch is not the same as stretch-then-rotate. In neural networks, matrix multiplication composes layers and processes large batches efficiently.

24. Mastery Check

1. Why does $BA\mathbf{x}$ mean apply $A$ first?
2. Derive a 2×2 matrix product by hand.
3. Why are matrix-product entries row–column dot products?
4. What shape results from (7×3) @ (3×5)?
5. Why is (7×3) @ (4×5) invalid?
6. Give a geometric explanation for $AB\ne BA$.
7. What does the identity matrix do?
8. What does an inverse matrix mean geometrically?
9. Why can two linear neural-network layers collapse into one?
10. Why does adding a nonlinearity change that conclusion?

🔭 Bridge to Chapter 011

We can now represent transformations and compose them.

But we have been using the word linear without fully asking what it permits and forbids.

What exactly makes a transformation linear, and what geometric structure must it preserve?

That is the next question.