2.7: Orthogonal projections and orthonormal bases in ℝ³

import numpy as np
import plotly.graph_objects as go
from pathlib import Path
import sys

_notes_root = next(p for p in [Path.cwd(), *Path.cwd().parents] if (p / 'myst.yml').exists())
if str(_notes_root) not in sys.path:
    sys.path.insert(0, str(_notes_root))
from plot_style import style_plotly

BLUE, ORANGE, PINK = '#3d81f6', 'orange', '#d81a60'


def base3():
    fig=style_plotly(go.Figure(),renderer='plotly_mimetype')
    axis=dict(range=[-7,10],dtick=2,tickfont=dict(size=11),showbackground=True,showspikes=False,
              backgroundcolor='white',gridcolor='#e5e7eb',zerolinecolor='#9ca3af')
    fig.update_layout(autosize=True,height=520,showlegend=False,font=dict(size=16),
                      margin=dict(l=0,r=0,t=15,b=0),
                      scene=dict(bgcolor='white',xaxis=dict(title=dict(text='x',font=dict(size=13)),**axis),yaxis=dict(title=dict(text='y',font=dict(size=13)),**axis),
                                 zaxis=dict(title=dict(text='z',font=dict(size=13)),**axis),aspectmode='cube',
                                 camera=dict(eye=dict(x=1.6,y=-2.1,z=1.3))))
    for direction in np.eye(3):
        line3(fig,-6*direction,8*direction,'#9ca3af',width=2)
    return fig


def line3(fig,start,end,color=BLUE,width=5,dash='solid'):
    fig.add_trace(go.Scatter3d(x=[start[0],end[0]],y=[start[1],end[1]],z=[start[2],end[2]],
                              mode='lines',line=dict(color=color,width=width,dash=dash),
                              hoverinfo='skip',showlegend=False))


def vec3(fig,end,label,color=BLUE,start=(0,0,0),offset=(0.25,0.25,0.35)):
    start,end=np.asarray(start,float),np.asarray(end,float)
    direction=(end-start)/np.linalg.norm(end-start)
    line3(fig,start,end-0.15*direction,color,width=7)
    fig.add_trace(go.Cone(x=[end[0]],y=[end[1]],z=[end[2]],u=[direction[0]],v=[direction[1]],w=[direction[2]],
                         anchor='tip',sizemode='absolute',sizeref=0.45,colorscale=[[0,color],[1,color]],
                         showscale=False,hoverinfo='skip'))
    pos=end+offset
    fig.add_trace(go.Scatter3d(x=[pos[0]],y=[pos[1]],z=[pos[2]],mode='text',text=[label],
                              textfont=dict(family='Palatino',color=color,size=20),hoverinfo='skip'))


def plane3(fig,normal,color=BLUE,opacity=0.24,d=0,bounds=(-5,7)):
    # Draw a complete rectangle in the plane, with room around every edge.
    n=np.asarray(normal,float)
    unit=n/np.linalg.norm(n)
    reference=np.eye(3)[np.argmin(np.abs(unit))]
    u=np.cross(unit,reference)
    u=u/np.linalg.norm(u)
    v=np.cross(unit,u)
    center=d*n/np.dot(n,n)
    lo,hi=bounds
    corners=np.array([center+a*u+b*v for a,b in
                      [(lo,lo),(hi,lo),(hi,hi),(lo,hi)]])
    fig.add_trace(go.Mesh3d(x=corners[:,0].tolist(),y=corners[:,1].tolist(),z=corners[:,2].tolist(),
                            i=[0,0],j=[1,2],k=[2,3],
                            color=color,opacity=opacity,flatshading=True,
                            lighting=dict(ambient=1,diffuse=0,specular=0,fresnel=0,roughness=1),
                            showscale=False,hoverinfo='skip'))
    for i in range(4):
        line3(fig,corners[i],corners[(i+1)%4],color,width=2)
    lower=min(fig.layout.scene.xaxis.range[0],float(corners.min())-1)
    upper=max(fig.layout.scene.xaxis.range[1],float(corners.max())+1)
    fig.update_scenes(xaxis_range=[lower,upper],yaxis_range=[lower,upper],
                      zaxis_range=[lower,upper])

In Chapter 2.3, we broke a vector in $\mathbb R^2$ into two perpendicular components. Let’s extend that idea to $\mathbb R^3$ . First, we’ll break a vector into a component along a line and a component in a plane perpendicular to that line. Then, we’ll use three perpendicular directions to describe the vector using an orthonormal basis.

Recalling projections in ℝ²¶

Let $\ell=\operatorname{span}(\vec w)$ , where $\vec w$ is nonzero. Given a vector $\vec v$ in $\mathbb R^2$ , we want to write

\vec v=\vec v_1+\vec v_2,

where $\vec v_1$ lies on $\ell$ and $\vec v_2$ lies on $\ell^\perp$ , the line perpendicular to $\ell$ through the origin.

Since $\vec v_1$ lies on $\ell$ , we can write $\vec v_1=c\vec w$ for some scalar $c$ . The remaining vector is $\vec v_2=\vec v-c\vec w$ . To make it perpendicular to $\ell$ , we need

\begin{aligned} (\vec v-c\vec w)\cdot\vec w&=0,\\ \vec v\cdot\vec w-c(\vec w\cdot\vec w)&=0. \end{aligned}

Solving for $c$ gives

c=\frac{\vec v\cdot\vec w}{\vec w\cdot\vec w}.

Therefore,

\vec v_1=\operatorname{proj}_{\ell}(\vec v) =\frac{\vec v\cdot\vec w}{\vec w\cdot\vec w}\vec w, \qquad \vec v_2=\operatorname{proj}_{\ell^\perp}(\vec v)=\vec v-\vec v_1.

Notice that $\vec w$ does not need to have length one. We only need $\vec w\ne\vec0$ , so that the denominator is nonzero.

Projecting onto a line and a plane in ℝ³¶

In $\mathbb R^3$ , the vectors perpendicular to a line through the origin form a plane through the origin. Let $\ell=\operatorname{span}(\vec w)$ and let $P=\ell^\perp$ . Equivalently, $\ell=P^\perp$ : the vectors perpendicular to every vector in $P$ form the line $\ell$ .

We again want to write $\vec v=\vec v_1+\vec v_2$ , but now $\vec v_1$ lies on $\ell$ and $\vec v_2$ lies in $P$ . The derivation above still works! It only uses dot products, so adding a third coordinate does not change the argument.

Loading...

The blue component $\vec v_1$ lies on $\ell=P^\perp$ , and the orange component $\vec v_2$ lies in $P$ . Together, they add to $\vec v$ . Drag to rotate the picture.

Definition: Orthogonal projections onto a line and a plane

Let $\ell$ be a line through the origin in $\mathbb R^3$ , and let $P=\ell^\perp$ . Write $\vec v=\vec v_1+\vec v_2$ , where $\vec v_1$ lies on $\ell$ and $\vec v_2$ lies in $P$ .

The orthogonal projection of $\vec v$ onto $\ell$ is

\vec v_1=\operatorname{proj}_{\ell}(\vec v).

It is the unique vector on $\ell$ for which $\vec v-\vec v_1$ is perpendicular to $\ell$ .

The orthogonal projection of $\vec v$ onto $P$ is

\vec v_2=\operatorname{proj}_P(\vec v)=\vec v-\vec v_1.

It is the unique vector in $P$ for which $\vec v-\vec v_2$ is perpendicular to $P$ .

If $\vec w$ is a nonzero normal vector to $P$ , we can first find the component along $\ell=\operatorname{span}(\vec w)$ , then subtract it from $\vec v$ .

Why does the second component lie in $P$ ? Recall from Chapter 2.5 that $P$ consists of the vectors whose dot product with its normal $\vec w$ is zero. Our choice of $c$ ensures exactly that:

\vec w\cdot(\vec v-c\vec w)=0.

Example: Projecting onto a plane¶

Let

\vec v=\begin{bmatrix}5\\1\\2\end{bmatrix},\qquad P:\ 2x-3y+4z=0.

Find the projections of $\vec v$ onto $\ell=P^\perp$ and onto $P$ .

The coefficients in the equation of $P$ give us a normal vector:

\vec w=\begin{bmatrix}2\\-3\\4\end{bmatrix},\qquad \ell=\operatorname{span}(\vec w).

We compute

\vec v\cdot\vec w=5(2)+1(-3)+2(4)=15,

\vec w\cdot\vec w=2^2+(-3)^2+4^2=29.

So the projection onto the line is

\vec v_1=\operatorname{proj}_{\ell}(\vec v) =\frac{15}{29}\begin{bmatrix}2\\-3\\4\end{bmatrix} =\begin{bmatrix}30/29\\-45/29\\60/29\end{bmatrix}.

Subtracting this from $\vec v$ gives the projection onto the plane:

\begin{aligned} \vec v_2=\operatorname{proj}_P(\vec v) &=\begin{bmatrix}5\\1\\2\end{bmatrix} -\begin{bmatrix}30/29\\-45/29\\60/29\end{bmatrix}\\ &=\begin{bmatrix}115/29\\74/29\\-2/29\end{bmatrix}. \end{aligned}

Let’s check that $\vec v_2$ lies in $P$ by substituting its coordinates into the plane’s equation:

2\left(\frac{115}{29}\right)-3\left(\frac{74}{29}\right) +4\left(-\frac{2}{29}\right) =\frac{230-222-8}{29}=0.

Also, $\vec v-\vec v_2=\vec v_1$ is a multiple of the normal vector, so it is perpendicular to $P$ .

v=np.array([5.,1.,2.])
w=np.array([2.,-3.,4.])
v1=(v@w)/(w@w)*w
v2=v-v1
fig=base3()
plane3(fig,w,ORANGE,opacity=0.18,bounds=(-4,6))
line3(fig,-0.9*w,1.2*w,BLUE,width=3)
vec3(fig,v1,'v₁',BLUE,offset=(-0.4,-0.3,0.4))
vec3(fig,v2,'v₂',ORANGE,offset=(0.3,0.3,-0.2))
vec3(fig,v,'v',PINK,offset=(0.3,0.1,0.4))
line3(fig,v1,v,ORANGE,width=4,dash='dash')
line3(fig,v2,v,BLUE,width=4,dash='dash')
# A right-angle marker in the plane containing the two components.
a=v1/np.linalg.norm(v1)*0.4
b=v2/np.linalg.norm(v2)*0.4
line3(fig,a,a+b,'#6b7280',width=3)
line3(fig,a+b,b,'#6b7280',width=3)
fig.add_trace(go.Scatter3d(x=[2.8,-2.4],y=[-4.2,3.2],z=[5.6,3.6],
                         mode='text',text=['ℓ','P'],textfont=dict(family='Palatino',size=22),
                         hoverinfo='skip'))
fig.update_scenes(xaxis_range=[-7,7],yaxis_range=[-7,7],zaxis_range=[-7,7],
                  camera=dict(eye=dict(x=1.5,y=-2.0,z=1.2)))
fig.show()

Loading...

Activity 1: Orthogonal projections in ℝ³

Let

\vec v=\begin{bmatrix}4\\3\\1\end{bmatrix},\qquad P:\ 2x+y-2z=0.

Find the orthogonal projection of $\vec v$ onto $P$ .

Find a normal vector $\vec n$ to $P$ .
Find the projection of $\vec v$ onto $\ell=\operatorname{span}(\vec n)$ .
Use your answer to find $\operatorname{proj}_P(\vec v)$ .
Verify that your answer lies in $P$ .

Solution

The coefficients in the plane’s equation give the normal vector

\vec n=\begin{bmatrix}2\\1\\-2\end{bmatrix}.

Since $\vec v\cdot\vec n=8+3-2=9$ and $\vec n\cdot\vec n=4+1+4=9$ ,

\operatorname{proj}_{\ell}(\vec v) =\frac99\begin{bmatrix}2\\1\\-2\end{bmatrix} =\begin{bmatrix}2\\1\\-2\end{bmatrix}.

Therefore,

\operatorname{proj}_P(\vec v) =\begin{bmatrix}4\\3\\1\end{bmatrix} -\begin{bmatrix}2\\1\\-2\end{bmatrix} =\begin{bmatrix}2\\2\\3\end{bmatrix}.

This vector lies in $P$ because $2(2)+2-2(3)=0$ .

Orthonormal bases in ℝ³¶

So far, we have broken a vector into a component along a line and a component in a plane. Can we break the plane component into two perpendicular components as well?

Choose two perpendicular lines through the origin in $P$ . Both are perpendicular to $\ell=P^\perp$ , so we now have three mutually perpendicular lines. Choosing a unit vector along each line gives three perpendicular unit directions. These play the same role as the two vectors in an orthonormal basis of $\mathbb R^2$ from Chapter 2.2.

“Pairwise orthogonal” means that each pair of distinct vectors is orthogonal. We need to check all three dot products.

Every vector in $\mathbb R^3$ can be written uniquely as a linear combination of these three directions. Geometrically, we can first project onto the line along $\vec u_1$ and its perpendicular plane, then split the plane component along $\vec u_2$ and $\vec u_3$ .

Example: The standard basis¶

The standard basis vectors

\vec e_1=\begin{bmatrix}1\\0\\0\end{bmatrix},\qquad \vec e_2=\begin{bmatrix}0\\1\\0\end{bmatrix},\qquad \vec e_3=\begin{bmatrix}0\\0\\1\end{bmatrix}

form an orthonormal basis. Each has length one, and every pair has dot product zero. For example,

\begin{bmatrix}5\\1\\2\end{bmatrix}=5\vec e_1+\vec e_2+2\vec e_3.

Example: A different orthonormal basis¶

Consider

\vec u_1=\begin{bmatrix}1/3\\2/3\\2/3\end{bmatrix},\qquad \vec u_2=\begin{bmatrix}2/3\\1/3\\-2/3\end{bmatrix},\qquad \vec u_3=\begin{bmatrix}-2/3\\2/3\\-1/3\end{bmatrix}.

First, check their lengths:

\begin{aligned} \lVert\vec u_1\rVert&=\sqrt{\frac{1+4+4}{9}}=1,\\ \lVert\vec u_2\rVert&=\sqrt{\frac{4+1+4}{9}}=1,\\ \lVert\vec u_3\rVert&=\sqrt{\frac{4+4+1}{9}}=1. \end{aligned}

Next, check all three dot products:

\begin{aligned} \vec u_1\cdot\vec u_2&=\frac{2+2-4}{9}=0,\\ \vec u_1\cdot\vec u_3&=\frac{-2+4-2}{9}=0,\\ \vec u_2\cdot\vec u_3&=\frac{-4+2+2}{9}=0. \end{aligned}

All three vectors have length one and are pairwise orthogonal, so they form an orthonormal basis of $\mathbb R^3$ .

Finding coefficients using dot products¶

Let $(\vec u_1,\vec u_2,\vec u_3)$ be an orthonormal basis. We can write a vector $\vec v$ as

\vec v=c_1\vec u_1+c_2\vec u_2+c_3\vec u_3.

How do we find $c_1$ , $c_2$ , and $c_3$ ? As in $\mathbb R^2$ , taking a dot product isolates one coefficient at a time. Taking the dot product with $\vec u_1$ gives

\begin{aligned} \vec v\cdot\vec u_1 &=c_1(\vec u_1\cdot\vec u_1)+c_2(\vec u_2\cdot\vec u_1)+c_3(\vec u_3\cdot\vec u_1)\\ &=c_1(1)+c_2(0)+c_3(0)\\ &=c_1. \end{aligned}

The first dot product is 1 because $\vec u_1$ is a unit vector. The other two are 0 because the vectors are orthogonal. Taking dot products with $\vec u_2$ and $\vec u_3$ similarly gives $c_2=\vec v\cdot\vec u_2$ and $c_3=\vec v\cdot\vec u_3$ .

Each term $(\vec v\cdot\vec u_i)\vec u_i$ is the projection of $\vec v$ onto the line $\operatorname{span}(\vec u_i)$ . The denominator in the projection formula is $\vec u_i\cdot\vec u_i=1$ . The coefficient is a scalar; multiplying it by $\vec u_i$ gives the corresponding vector component.

Example: Writing a vector in the new basis¶

Use the orthonormal basis from the previous example:

\vec u_1=\begin{bmatrix}1/3\\2/3\\2/3\end{bmatrix},\qquad \vec u_2=\begin{bmatrix}2/3\\1/3\\-2/3\end{bmatrix},\qquad \vec u_3=\begin{bmatrix}-2/3\\2/3\\-1/3\end{bmatrix}.

Let $\vec v=\begin{bmatrix}-5\\2\\5\end{bmatrix}$ . Its coefficients are

\begin{aligned} c_1=\vec v\cdot\vec u_1&=\frac{-5+4+10}{3}=3,\\ c_2=\vec v\cdot\vec u_2&=\frac{-10+2-10}{3}=-6,\\ c_3=\vec v\cdot\vec u_3&=\frac{10+4-5}{3}=3. \end{aligned}

Therefore,

\vec v=3\vec u_1-6\vec u_2+3\vec u_3.

We can check this by carrying out the linear combination in standard coordinates:

\begin{aligned} 3\vec u_1-6\vec u_2+3\vec u_3 &=\begin{bmatrix}1\\2\\2\end{bmatrix} +\begin{bmatrix}-4\\-2\\4\end{bmatrix} +\begin{bmatrix}-2\\2\\-1\end{bmatrix}\\ &=\begin{bmatrix}-5\\2\\5\end{bmatrix}=\vec v. \end{aligned}

The coordinates in this basis are $3,-6,3$ , while the standard coordinates are $-5,2,5$ . These are two descriptions of the same vector. The negative coefficient -6 means that the second component points opposite to $\vec u_2$ .

u1=np.array([1.,2.,2.])/3
u2=np.array([2.,1.,-2.])/3
u3=np.array([-2.,2.,-1.])/3
components=[3*u1,-6*u2,3*u3]
fig=base3()
for u,component,label,color in zip([u1,u2,u3],components,['3u₁','−6u₂','3u₃'],[BLUE,ORANGE,PINK]):
    line3(fig,-6.7*u,4*u,color,width=2)
    vec3(fig,component,label,color)
line3(fig,components[0],components[0]+components[1],ORANGE,width=4,dash='dash')
line3(fig,components[0]+components[1],sum(components),PINK,width=4,dash='dash')
vec3(fig,sum(components),'v','#333333',offset=(-0.5,0.2,0.3))
fig.update_scenes(xaxis_range=[-7,5],yaxis_range=[-5,7],zaxis_range=[-5,7],
                  camera=dict(eye=dict(x=1.6,y=-2.1,z=1.3)))
fig.show()

Loading...

The components $3\vec u_1$ , $-6\vec u_2$ , and $3\vec u_3$ lie on three mutually perpendicular lines. The dashed copies show how they add to $\vec v$ . Drag to rotate the picture.

Activity 2: Coordinates in an orthonormal basis

Consider the vectors

\vec u_1=\begin{bmatrix}3/7\\-2/7\\6/7\end{bmatrix},\qquad \vec u_2=\begin{bmatrix}6/7\\3/7\\-2/7\end{bmatrix},\qquad \vec u_3=\begin{bmatrix}-2/7\\6/7\\3/7\end{bmatrix}.

Verify that each vector has length 1.
Verify that the vectors are pairwise orthogonal by computing $\vec u_1\cdot\vec u_2$ , $\vec u_1\cdot\vec u_3$ , and $\vec u_2\cdot\vec u_3$ .
Explain why $(\vec u_1,\vec u_2,\vec u_3)$ is an orthonormal basis of $\mathbb R^3$ .
Let $\vec w=\begin{bmatrix}4\\-1\\5\end{bmatrix}$ . Compute $\vec w\cdot\vec u_1$ , $\vec w\cdot\vec u_2$ , and $\vec w\cdot\vec u_3$ .
Use these dot products to write $\vec w$ as a linear combination of $\vec u_1$ , $\vec u_2$ , and $\vec u_3$ .
Check your answer by carrying out the linear combination in standard coordinates.

Solution

The lengths are

\begin{aligned} \lVert\vec u_1\rVert&=\sqrt{\frac{9+4+36}{49}}=1,\\ \lVert\vec u_2\rVert&=\sqrt{\frac{36+9+4}{49}}=1,\\ \lVert\vec u_3\rVert&=\sqrt{\frac{4+36+9}{49}}=1. \end{aligned}

The dot products are

\begin{aligned} \vec u_1\cdot\vec u_2&=\frac{18-6-12}{49}=0,\\ \vec u_1\cdot\vec u_3&=\frac{-6-12+18}{49}=0,\\ \vec u_2\cdot\vec u_3&=\frac{-12+18-6}{49}=0. \end{aligned}

Thus, the three vectors have length one and are pairwise orthogonal, so they form an orthonormal basis of $\mathbb R^3$ .

Taking dot products with $\vec w$ gives

\begin{aligned} \vec w\cdot\vec u_1&=\frac{12+2+30}{7}=\frac{44}{7},\\ \vec w\cdot\vec u_2&=\frac{24-3-10}{7}=\frac{11}{7},\\ \vec w\cdot\vec u_3&=\frac{-8-6+15}{7}=\frac17. \end{aligned}

Therefore,

\vec w=\frac{44}{7}\vec u_1+\frac{11}{7}\vec u_2+\frac17\vec u_3.

To check,

\begin{aligned} \frac{44}{7}\vec u_1+\frac{11}{7}\vec u_2+\frac17\vec u_3 &=\frac{1}{49}\left( \begin{bmatrix}132\\-88\\264\end{bmatrix} +\begin{bmatrix}66\\33\\-22\end{bmatrix} +\begin{bmatrix}-2\\6\\3\end{bmatrix}\right)\\ &=\frac{1}{49}\begin{bmatrix}196\\-49\\245\end{bmatrix} =\begin{bmatrix}4\\-1\\5\end{bmatrix}=\vec w. \end{aligned}

Application: Forces on a drone¶

The usual $x$ -, $y$ -, and $z$ -directions describe a force using fixed, global coordinates. For a moving drone, it can be more useful to describe the same force using the drone’s forward, rightward, and upward directions. If these directions form an orthonormal basis, three dot products tell us how much force acts along each one.

Activity 3: Forces on a drone

A drone’s orientation is described by three local directions:

\vec u_1=\begin{bmatrix}3/7\\-2/7\\6/7\end{bmatrix},\qquad \vec u_2=\begin{bmatrix}6/7\\3/7\\-2/7\end{bmatrix},\qquad \vec u_3=\begin{bmatrix}-2/7\\6/7\\3/7\end{bmatrix}.

These vectors point in the drone’s forward, rightward, and upward directions, respectively. They form an orthonormal basis of $\mathbb R^3$ .

A force acting on the drone has global coordinates

\vec F=\begin{bmatrix}4\\-1\\5\end{bmatrix}\ \mathrm N.

Find the component of $\vec F$ in the drone’s forward direction, $\vec F\cdot\vec u_1$ .
Find the component of $\vec F$ in the drone’s rightward direction, $\vec F\cdot\vec u_2$ .
Find the component of $\vec F$ in the drone’s upward direction, $\vec F\cdot\vec u_3$ .
Use your answers to write $\vec F$ as a linear combination of the drone’s local directions.
Check your answer by carrying out the linear combination in standard coordinates.
Interpret the signs and relative sizes of the three components. In which local direction does most of the force act?

Solution

The basis and vector have the same entries as in Activity 2, so the same dot products give the force components:

\vec F\cdot\vec u_1=\frac{44}{7}\ \mathrm N,\qquad \vec F\cdot\vec u_2=\frac{11}{7}\ \mathrm N,\qquad \vec F\cdot\vec u_3=\frac17\ \mathrm N.

Thus,

\vec F=\left(\frac{44}{7}\vec u_1+\frac{11}{7}\vec u_2+\frac17\vec u_3\right)\mathrm N.

In standard coordinates, this is

\begin{aligned} \vec F &=\frac{1}{49}\left( \begin{bmatrix}132\\-88\\264\end{bmatrix} +\begin{bmatrix}66\\33\\-22\end{bmatrix} +\begin{bmatrix}-2\\6\\3\end{bmatrix}\right)\mathrm N\\ &=\begin{bmatrix}4\\-1\\5\end{bmatrix}\ \mathrm N. \end{aligned}

All three coefficients are positive, so the force has forward, rightward, and upward components relative to the drone. The forward component is largest: $44/7\approx6.29$ N, compared with $11/7\approx1.57$ N rightward and $1/7\approx0.14$ N upward. Most of the force acts in the drone’s forward direction.

The negative second entry in the global coordinates does not imply a negative rightward component. The global directions and the drone’s local directions are different.