5.3: Subspaces, bases, dimension, and rank

Subspaces¶

Lines and planes are both “flat” objects that extend infinitely, without edges, in their directions. A line, whether it is in $\mathbb R^2$, $\mathbb R^3$, or $\mathbb R^{17}$ for that matter, is 1-dimensional in some sense and can be thought of as a “slice” of space. Similarly, a plane in $\mathbb R^3$ is a 2-dimensional “slice.”

The general notion of a “flat slice” through the origin is more precisely referred to as a subspace.

Imagine a subspace as a gated community of vectors, where linear combinations cannot escape the community. These conditions must hold for every vector in the set and every real scalar. One counterexample is enough to show that a set is not a subspace.

Examples and non-examples¶

Below, we’ve shown 4 examples of subsets of $\mathbb R^2$ in blue. Orange vectors belong to the set, while pink vectors (for the sets that are not subspaces) show a violation of one of the subspace properties. The pink cross marks the missing zero vector.

• The line $y=2x$ is a subspace. Its vectors have the form $\begin{bmatrix}a\\2a\end{bmatrix}$. It contains $\vec0$, and

  $$\begin{bmatrix}a\\2a\end{bmatrix}+\begin{bmatrix}b\\2b\end{bmatrix}=\begin{bmatrix}a+b\\2(a+b)\end{bmatrix},\qquad c\begin{bmatrix}a\\2a\end{bmatrix}=\begin{bmatrix}ca\\2ca\end{bmatrix}$$

  are still on the line.

• The first quadrant, where $x\ge0$ and $y\ge0$, is not a subspace. It contains $\begin{bmatrix}2\\3\end{bmatrix}$ but not $-\begin{bmatrix}2\\3\end{bmatrix}=\begin{bmatrix}-2\\-3\end{bmatrix}$. It fails closure under scalar multiplication.

• The line $y=3x-4$ is not a subspace. It misses the origin: $0\ne3(0)-4$. It is an affine subspace—a shifted subspace. Being flat is not enough.

• The parabola $y=x^2$, written in set-builder notation as

  $$S=\left\{\begin{bmatrix}x\\y\end{bmatrix}\in\mathbb R^2:y=x^2\right\},$$

  is not a subspace. It contains $\begin{bmatrix}1\\1\end{bmatrix}$, but doubling this vector gives $\begin{bmatrix}2\\2\end{bmatrix}$, which is not on the parabola. This fails closure under both scalar multiplication and addition. Passing through the origin is not enough either.

The possible subspaces in $\mathbb R^2$ and $\mathbb R^3$ are:

Spans and column spaces¶

Subspaces generalize lines and planes through the origin, which we already know how to describe as spans of one or two vectors. Here’s the key connection: Every span of vectors in $\mathbb R^n$ is a subspace of $\mathbb R^n$. In particular, $\operatorname{colsp}(A)$ for an $n\times d$ matrix $A$ is a subspace of $\mathbb R^n$.

For example, revisit the matrices from Chapter 5.1:

• For $A=\begin{bmatrix}1&0\\1&1\\0&1\end{bmatrix}$, $\operatorname{colsp}(A)$ is a subspace of $\mathbb R^3$: the plane $x-y+z=0$.

• Adding the redundant column in $B$ leaves that plane unchanged, so $\operatorname{colsp}(B)$ is the same subspace.

• The three independent columns of $C$ span all of $\mathbb R^3$, which is itself a subspace of $\mathbb R^3$.

Bases¶

Consider the $4\times3$ matrix

Its column space is a subspace of $\mathbb R^4$, since its columns have four entries. But we don’t need all three columns to describe $\operatorname{colsp}(A)$, because its third column is redundant:

Removing $\vec a_3$ leaves the column space unchanged. Indeed,

and every combination of $\vec a_1,\vec a_2$ is an output by choosing $r=0$. These two columns span the column space and are independent, since neither is a scalar multiple of the other.

We say $\{\vec a_1,\vec a_2\}$ form a basis for $\operatorname{colsp}(A)$.

The plural of basis is bases. Every nonzero subspace has infinitely many bases. Here are some examples and non-examples of bases for $\operatorname{colsp}(A)$, where

The last collection shows that a basis need not use the original matrix’s columns.

Dimension¶

Different bases of a subspace can use different vectors, but they always contain the same number of vectors.

For our $4\times3$ matrix

the basis $\{\vec a_1,\vec a_2\}$ contains two vectors. Thus $\dim(\operatorname{colsp}(A))=2$: its column space is a plane through the origin in $\mathbb R^4$.

These are three different counts: 4 rows tell us that outputs have four entries; 3 columns tell us that inputs have three entries; dimension 2 tells us how many vectors a basis contains.

A line has dimension 1, regardless of how many entries its vectors have. For example,

is a line in $\mathbb R^5$, with the displayed nonzero vector as a basis.

The columns of $I_n$ form the standard basis of $\mathbb R^n$: they are independent, and any vector is their linear combination with its entries as coefficients. Hence $\dim(\mathbb R^n)=n$. By convention, the empty collection is a basis of $\{\vec0\}$, which has dimension zero.

For example, 3 linearly independent vectors in $\mathbb R^3$ must form a basis of $\mathbb R^3$ and span all of $\mathbb R^3$. Since $\mathbb R^3$ has dimension 3, an independent collection of 3 vectors already has as many vectors as a basis; it cannot span a smaller subspace.

Example: A basis from equations¶

Consider the set

What does this set look like? The equations force $x_2=-x_1$ and $x_3=2x_4$. Choose $x_1=s$ and $x_4=t$. Then

Every vector in $S$ has this form, and every choice of $s,t$ satisfies both equations. Thus $S$ is the span of the two displayed vectors, so it is a subspace. Neither vector is a scalar multiple of the other, so they form a basis. Therefore $\dim(S)=2$: it is a plane through the origin in $\mathbb R^4$.

Rank and solvability¶

Equivalently, rank is the maximum number of linearly independent columns. Our $4\times3$ matrix $A$ has rank 2.

Here are several examples of matrices and their ranks; we have already seen many of these matrices in this chapter.

As a thought experiment, consider a $7\times10$ matrix. Its smallest possible rank is 0, achieved by the zero matrix. The largest is 7: its column space is a subspace of $\mathbb R^7$, so it cannot have dimension larger than 7.

More generally, a basis chosen from the columns of an $n\times d$ matrix cannot contain more than $d$ vectors, and a subspace of $\mathbb R^n$ cannot have dimension larger than $n$. Therefore,

Invertibility¶

An $n\times n$ matrix is invertible exactly when its $n$ columns are independent, or equivalently, when its rank is $n$. Those columns then form a basis of $\mathbb R^n$.

Solvability¶

The equation $A\vec x=\vec b$ has a solution exactly when $\vec b\in\operatorname{colsp}(A)$. If we want a solution for every $\vec b\in\mathbb R^n$, the column space must be all of $\mathbb R^n$:

We compare rank to the number of rows when asking whether every output is possible. A rectangular matrix can produce every output without being invertible: for example, $\begin{bmatrix}1&0&0\\0&1&0\end{bmatrix}$ has rank 2, but its third input entry can be anything.

As another example, consider

The first two columns are independent. The third is their sum, and the fourth is three times the second minus the first. Thus the first two columns form a basis of $\operatorname{colsp}(F)$, and $\operatorname{rank}(F)=2$.

A general combination of the basis vectors is

So $\operatorname{colsp}(F)$ is the plane $z=x+y$. This tells us exactly which outputs are possible. For instance,

but $\begin{bmatrix}2\\3\\6\end{bmatrix}$ is impossible. The columns are dependent, so every possible output has infinitely many inputs. Rank 2 describes the dimension of the output set; the plane equation tells us whether a particular vector belongs to it.