3.5: Matrix multiplication - Math 124 Course Notes

from pathlib import Path
import sys
_notes_root = next(p for p in [Path.cwd(), *Path.cwd().parents] if (p / 'myst.yml').exists())
if str(_notes_root / 'ch03') not in sys.path:
    sys.path.insert(0, str(_notes_root / 'ch03'))
from friendly_visuals import mapping, transform, projection_line, projection_plane
from chapter3_visuals import save_figure, widget, show_3d

The golden rule¶

Before we figure out how to multiply two matrices, let’s check when we can.

For example, a $2\times3$ matrix can multiply a $3\times4$ matrix, and the answer has shape $2\times4$ . A $2\times3$ matrix cannot multiply a $2\times4$ matrix: the inner numbers, 3 and 2, don’t match. This means that matrix multiplication is not commutative: just because $AB$ is defined (meaning it is legal, by the golden rule, to multiply $A$ on the right by $B$ ) does not mean $BA$ is defined.

This is the same rule we used for matrix-vector multiplication. A vector is just a matrix with one column. Now the second matrix can have several columns.

One column at a time¶

Multiply the first matrix by each column of the second matrix, then put the answers next to each other. Consider

A=\begin{bmatrix}4&-3&7\\-2&6&0\end{bmatrix},\qquad B=\begin{bmatrix}{\color{#3d81f6}5}&{\color{orange}-4}\\{\color{#3d81f6}-2}&{\color{orange}8}\\{\color{#3d81f6}3}&{\color{orange}-6}\end{bmatrix}.

The shapes are $(2\times3)(3\times2)$ , so $AB$ has shape $2\times2$ . For the first column,

A{\color{#3d81f6}\begin{bmatrix}5\\-2\\3\end{bmatrix}} =\begin{bmatrix}4({\color{#3d81f6}5})-3({\color{#3d81f6}-2})+7({\color{#3d81f6}3})\\-2({\color{#3d81f6}5})+6({\color{#3d81f6}-2})+0({\color{#3d81f6}3})\end{bmatrix} ={\color{#3d81f6}\begin{bmatrix}47\\-22\end{bmatrix}}.

For the second column,

A{\color{orange}\begin{bmatrix}-4\\8\\-6\end{bmatrix}} =\begin{bmatrix}4({\color{orange}-4})-3({\color{orange}8})+7({\color{orange}-6})\\-2({\color{orange}-4})+6({\color{orange}8})+0({\color{orange}-6})\end{bmatrix} ={\color{orange}\begin{bmatrix}-82\\56\end{bmatrix}}.

Place the two output vectors next to each other, keeping their order:

AB=\begin{bmatrix}{\color{#3d81f6}47}&{\color{orange}-82}\\{\color{#3d81f6}-22}&{\color{orange}56}\end{bmatrix}.

We used matrix-vector multiplication twice. Use the interactive below to follow either column.

Loading...

Square example¶

Let

C=\begin{bmatrix}2&-1\\1&3\end{bmatrix},\qquad D=\begin{bmatrix}{\color{#3d81f6}1}&{\color{orange}4}\\{\color{#3d81f6}2}&{\color{orange}-1}\end{bmatrix}.

Multiplying one column at a time gives

C{\color{#3d81f6}\begin{bmatrix}1\\2\end{bmatrix}}={\color{#3d81f6}\begin{bmatrix}0\\7\end{bmatrix}},\qquad C{\color{orange}\begin{bmatrix}4\\-1\end{bmatrix}}={\color{orange}\begin{bmatrix}9\\1\end{bmatrix}},

so

CD=\begin{bmatrix}{\color{#3d81f6}0}&{\color{orange}9}\\{\color{#3d81f6}7}&{\color{orange}1}\end{bmatrix}.

Activity: Find one entry

Let

E=\begin{bmatrix}2&-1&4\\0&3&5\\-2&6&1\\7&0&-3\\4&-2&1\end{bmatrix},\qquad F=\begin{bmatrix}3&-5\\2&4\\-1&6\end{bmatrix}.

What is the entry in position $(5,2)$ of $EF$ ? Compute only that entry.

Two transformations¶

Suppose we first project onto the line spanned by $\begin{bmatrix}1\\2\end{bmatrix}$ , then rotate counterclockwise by $30^\circ$ . The two matrices are

P=\frac15\begin{bmatrix}1&2\\2&4\end{bmatrix},\qquad R=\begin{bmatrix}\frac{\sqrt3}{2}&-\frac12\\\frac12&\frac{\sqrt3}{2}\end{bmatrix}.

For $\vec v=\begin{bmatrix}5\\0\end{bmatrix}$ ,

\vec v\xrightarrow{\ P\ }\begin{bmatrix}1\\2\end{bmatrix} \xrightarrow{\ R\ }\begin{bmatrix}\frac{\sqrt3}{2}-1\\\frac12+\sqrt3\end{bmatrix}.

One matrix performs both steps:

RP=\frac1{10}\begin{bmatrix}\sqrt3-2&2\sqrt3-4\\1+2\sqrt3&2+4\sqrt3\end{bmatrix}.

Multiplying $(RP)\vec v$ gives the same final vector. Because the projection happens first, its matrix appears closest to $\vec v$ .

Stretch, then rotate¶

Now suppose we stretch horizontally by a factor of 2 and then rotate counterclockwise by $90^\circ$ :

H=\begin{bmatrix}2&0\\0&1\end{bmatrix},\qquad R=\begin{bmatrix}0&-1\\1&0\end{bmatrix}.

For an input $\begin{bmatrix}x\\y\end{bmatrix}$ , the two steps are

\begin{bmatrix}x\\y\end{bmatrix} \xrightarrow{\ H\ }\begin{bmatrix}2x\\y\end{bmatrix} \xrightarrow{\ R\ }\begin{bmatrix}-y\\2x\end{bmatrix}.

The final rule has matrix $\begin{bmatrix}0&-1\\2&0\end{bmatrix}$ . And that’s exactly what we get by multiplying:

RH=\begin{bmatrix}0&-1\\1&0\end{bmatrix}\begin{bmatrix}2&0\\0&1\end{bmatrix} =\begin{bmatrix}0&-1\\2&0\end{bmatrix}.

That’s why stretching first and rotating second gives $RH$ . For example,

\begin{bmatrix}2\\1\end{bmatrix}\xrightarrow{\ H\ }\begin{bmatrix}4\\1\end{bmatrix} \xrightarrow{\ R\ }\begin{bmatrix}-1\\4\end{bmatrix},\qquad (RH)\begin{bmatrix}2\\1\end{bmatrix}=\begin{bmatrix}-1\\4\end{bmatrix}.

Order matters¶

Would rotating first and stretching second give the same result? Let’s check:

HR=\begin{bmatrix}2&0\\0&1\end{bmatrix}\begin{bmatrix}0&-1\\1&0\end{bmatrix} =\begin{bmatrix}0&-2\\1&0\end{bmatrix}.

This is different from $RH$ . Our same input now follows

\begin{bmatrix}2\\1\end{bmatrix}\xrightarrow{\ R\ }\begin{bmatrix}-1\\2\end{bmatrix} \xrightarrow{\ H\ }\begin{bmatrix}-2\\2\end{bmatrix}.

Why did the order matter? The stretch always doubles the horizontal coordinate. Rotating changes which part of the original vector points horizontally. Try both orders below.

Loading...

In general, $AB\ne BA$ . Unlike multiplication of real numbers, matrix multiplication is not commutative.

Sometimes reversing the order changes even the shape of the answer. For example, let

A=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix},\qquad B=\begin{bmatrix}1&0&0\\0&1&0\end{bmatrix}.

These are the two transformations at the end of Chapter 3.4. Multiplying gives

BA=\begin{bmatrix}1&0\\0&1\end{bmatrix},\qquad AB=\begin{bmatrix}1&0&0\\0&1&0\\1&1&0\end{bmatrix}.

The first product has shape $2\times2$ : append the sum of two entries, then discard it, and you’re back where you started. The second has shape $3\times3$ : discard the third entry, then replace it with the sum of the first two. You won’t generally get the original third entry back.

Sometimes a product is defined in only one order. For example, let

C=\begin{bmatrix}1&2&0\\-1&0&3\end{bmatrix},\qquad D=\begin{bmatrix}1&0&2&-1\\0&1&3&2\\2&-1&0&1\end{bmatrix}.

Here, $C$ has shape $2\times3$ and $D$ has shape $3\times4$ . The product $CD$ is defined and has shape $2\times4$ :

CD=\begin{bmatrix}1&2&0\\-1&0&3\end{bmatrix}\begin{bmatrix}1&0&2&-1\\0&1&3&2\\2&-1&0&1\end{bmatrix} =\begin{bmatrix}1&2&8&3\\5&-3&-2&4\end{bmatrix}.

But $DC$ is not defined: its shapes would be $(3\times4)(2\times3)$ , and the inner sizes, 4 and 2, do not match.

It is possible for $AB=BA$ in special cases. For instance, doubling every coordinate and then rotating has the same effect as rotating and then doubling every coordinate. But this is not always true: we cannot assume that $AB=BA$ for arbitrary matrices.

Properties¶

We can still use several familiar rules. Whenever the shapes allow the operations,

\begin{aligned} (AB)C&=A(BC),\\ A(B+C)&=AB+AC,\\ (A+B)C&=AC+BC,\\ c(AB)&=(cA)B=A(cB). \end{aligned}

The first property lets us change the parentheses, keeping the matrices in the same order. Both sides apply $C$ first, then $B$ , then $A$ . Changing the parentheses is different from reversing the order!

The identity matrix leaves vectors unchanged, so for a matrix $A$ of shape $n\times d$ ,

I_nA=A=AI_d.

Projecting twice¶

Here’s one last example. If $P=\begin{bmatrix}1&0\\0&0\end{bmatrix}$ projects onto the $x$ -axis, then

P^2=\begin{bmatrix}1&0\\0&0\end{bmatrix}\begin{bmatrix}1&0\\0&0\end{bmatrix} =\begin{bmatrix}1&0\\0&0\end{bmatrix}=P.

Does that make sense geometrically? After the first projection, our vector is already on the $x$ -axis. Projecting it again leaves it there.

Activity: Practice multiplying

Let $A=\begin{bmatrix}1&1\\1&0\end{bmatrix}$ and $D=\begin{bmatrix}2&0\\0&1\end{bmatrix}$ .

Compute $AD$ and $DA$ , one column at a time. Are they equal?
For $\vec v=\begin{bmatrix}1\\2\end{bmatrix}$ , compute $A(D\vec v)$ and $(AD)\vec v$ . Do they agree?
Which product describes applying $A$ first and then stretching horizontally by a factor of 2?

Expanding matrix expressions¶

The distributive properties let us expand expressions much as we do with numbers. We just need to keep the factors in their original order. Let’s start with

A=\begin{bmatrix}1&2\\0&-1\end{bmatrix},\qquad B=\begin{bmatrix}2&0\\1&1\end{bmatrix},\qquad C=\begin{bmatrix}-1&3\\0&2\end{bmatrix}.

Adding first gives

B+C=\begin{bmatrix}1&3\\1&3\end{bmatrix},\qquad A(B+C)=\begin{bmatrix}3&9\\-1&-3\end{bmatrix}.

Multiplying first gives

AB=\begin{bmatrix}4&2\\-1&-1\end{bmatrix},\qquad AC=\begin{bmatrix}-1&7\\0&-2\end{bmatrix}.

Their sum is the same matrix! This is an example of $A(B+C)=AB+AC$ . Notice that $A$ stays on the left of both terms. Similarly, in $(B+C)A=BA+CA$ , it stays on the right.

Squaring a sum¶

Now suppose $A$ and $B$ are square matrices of the same shape. Recall that $(A+B)^2$ means $(A+B)(A+B)$ . Expanding gives

\begin{aligned} (A+B)^2&=A(A+B)+B(A+B)\\ &=A^2+AB+BA+B^2. \end{aligned}

With real numbers, we would combine the middle two terms. Can we do that here? Only if $AB=BA$ . In general, they are different matrices, so $AB+BA$ cannot be replaced by $2AB$ .

The horizontal stretch and rotation from earlier give a concrete example where they don’t commute. Doubling every coordinate and rotating gives an example where they do. The distributive property works in both cases; whether we can combine terms depends on the particular matrices.

Activity: Expand before simplifying

Let $A$ and $B$ be square matrices of the same shape.

Expand $(A-B)^2$ , keeping every product in order.
Expand $(A-B)(A+B)$ . Is it always equal to $A^2-B^2$ ?
What additional condition on $A$ and $B$ makes both familiar scalar formulas work?