10 Euclidean spaces
10.1 Inner products
A symmetric bilinear form \(\langle\cdot{,}\cdot\rangle\) on an \(\mathbb{R}\)-vector space \(V\) allows to talk about vectors being orthogonal, but so far we have not defined the length of a vector or the distance between two vectors. In \(\mathbb{R}^n\) equipped with the standard scalar product \(\langle\cdot{,}\cdot\rangle,\) the length of a vector \(\vec{x}=(x_i)_{1\leqslant i\leqslant n} \in \mathbb{R}^n\) is denoted by \(\Vert\vec{x}\Vert\) and defined as \[\Vert\vec{x}\Vert=\sqrt{\langle \vec{x},\vec{x}\rangle}=\sqrt{\sum_{i=1}^n (x_i)^2}.\] Over the real numbers we can only take square roots of non-negative numbers. Hence if we want to analogously define the length of vectors in an abstract vectors space \(V\) that is equipped with a bilinear form \(\langle\cdot{,}\cdot\rangle,\) then we need that \(\langle v,v\rangle\geqslant 0\) for all vectors \(v \in V.\) This is known as positivity. Clearly, having a positive symmetric bilinear form on an \(\mathbb{R}\)-vector space, we can define the length of vectors as in the case of \(\mathbb{R}^n\) equipped with the standard scalar product. It might however still happen that there are vectors \(v \in V\) different from the zero vector \(0_V\) that satisfy \(\langle v,v\rangle=0.\) Naturally, one might ask that the zero vector is the only vector with length zero. This leads to the notion of definiteness.
Definition 10.1 • Properties of bilinear forms
A bilinear form \(\langle\cdot{,}\cdot\rangle\) on an \(\mathbb{R}\)-vector space \(V\) is called
positive if \(\langle v,v\rangle\geqslant 0\) for all vectors \(v \in V\);
definite if \(\langle v,v\rangle=0\) if and only if \(v=0_V.\)
Combining positivity, definiteness and symmetry, we arrive at the notion of an inner product:
Definition 10.2 • Inner product
Let \(V\) be an \(\mathbb{R}\)-vector space. A bilinear form \(\langle\cdot{,}\cdot\rangle\) on \(V\) that is positive definite and symmetric is called an inner product.
Remark 10.3
Notice that an inner product \(\langle\cdot{,}\cdot\rangle\) on an \(\mathbb{R}\)-vector space \(V\) is always a non-degenerate bilinear form. Indeed, if \(v_0 \in V\) satisfies \(\langle v,v_0\rangle=0\) for all vectors \(v \in V,\) then we also have \(\langle v_0,v_0\rangle=0\) and hence \(v_0=0_V,\) since \(\langle\cdot{,}\cdot\rangle\) is positive definite.
Example 10.4 • Inner products
The standard scalar product on \(\mathbb{R}^n\) defined by the rule (9.1) is indeed an inner product. Clearly, \(\langle\cdot{,}\cdot\rangle\) is symmetric and from the Analysis I module we know that \(y^2\geqslant 0\) for all real numbers \(y\) and that \(y^2=0\) if and only if \(y=0.\) Since \[\langle \vec{x},\vec{x}\rangle=\sum_{i=1}^n (x_i)^2,\] we conclude that \(\langle\cdot{,}\cdot\rangle\) is positive definite and hence an inner product. The vector space \(\mathbb{R}^n\) equipped with the standard scalar product is sometimes denoted by \(\mathbb{E}^n\) (the letter E is to remind of the Greek Geometer Euclid).
We consider \(V=M_{3,3}(\mathbb{R})\) and let \(U\subset V\) be the subspace consisting of anti-symmetric matrices. On \(U\) we define a symmetric bilinear form (notice the minus sign) \[\langle\cdot{,}\cdot\rangle: U\times U \to \mathbb{R}, \quad (\mathbf{A},\mathbf{B})\mapsto \langle \mathbf{A},\mathbf{B}\rangle=-\operatorname{Tr}(\mathbf{A}\mathbf{B}).\] An element \(\mathbf{A}\) of \(U\) satisfies \(\mathbf{A}^T=-\mathbf{A}\) and hence can be written as \[\mathbf{A}=\begin{pmatrix} 0 & x & y \\ -x & 0 & z \\ -y & -z & 0 \end{pmatrix}\] for real numbers \(x,y,z.\) We obtain \[\langle \mathbf{A},\mathbf{A}\rangle=-\operatorname{Tr}\begin{pmatrix}-x^2-y^2 & -yz & xz\\ -yz & -x^2-z^2 & -xy \\ xz & -xy & -y^2-z^2\end{pmatrix}=2x^2+2y^2+2z^2.\] We conclude that \(\langle \mathbf{A},\mathbf{A}\rangle\geqslant 0\) and \(\langle \mathbf{A},\mathbf{A}\rangle=0\) if and only if \(\mathbf{A}=\mathbf{0}_3.\) Therefore, \(\langle\cdot{,}\cdot\rangle\) is an inner product on \(U.\)
Let \(a<b\) be real numbers and consider \(V=\mathsf{C}([a,b],\mathbb{R}),\) the \(\mathbb{R}\)-vector space of continuous real-valued functions on the interval \([a,b].\) As in Example 9.2, (vi) we obtain a symmetric bilinear form on \(V\) via the definition \[\langle\cdot{,}\cdot\rangle: V \times V \to \mathbb{R}, \qquad (f,g) \mapsto \langle f,g\rangle=\int_{a}^bf(x)g(x)\mathrm{d}x.\] The properties of integration from the Analysis module imply that \(\langle\cdot{,}\cdot\rangle\) is also positive definite and hence an inner product.
Remark 10.5 • Naming convention
As we have seen, the standard scalar product on \(\mathbb{R}^n\) is an example of an inner product. It is common to refer to inner products as scalar products as well. In these notes we will reserve the term scalar product for the standard scalar product on \(\mathbb{R}^n\) and use inner product for a general positive definite symmetric bilinear form.
Notice that a symmetric bilinear form can be positive, but not positive definite:
Example 10.6
For any \(x_0 \in \mathbb{R},\) the symmetric bilinear form on \(V=\mathsf{P}(\mathbb{R})\) defined by \[\langle\cdot{,}\cdot\rangle: V \times V \to \mathbb{R}, \qquad (p,q)\mapsto p(x_0)q(x_0)\] satisfies \(\langle p,p\rangle=p(x_0)^2\geqslant 0\) and hence is positive. It is however not an inner product. The polynomial \(f\) defined by the rule \(x \mapsto f(x)=(x-x_0)\) for all \(x \in \mathbb{R}\) is different from the zero polynomial \(o: x \mapsto 0\; \forall x \in \mathbb{R},\) but also satisfies \(\langle f,f\rangle=0.\)
An inner product \(\langle\cdot{,}\cdot\rangle\) on an abstract \(\mathbb{R}\)-vector space \(V\) allows to define geometric notions like length and distance on \(V.\)
Definition 10.7 • Euclidean space
A pair \((V,\langle\cdot{,}\cdot\rangle)\) consisting of an \(\mathbb{R}\)-vector space \(V\) and an inner product \(\langle\cdot{,}\cdot\rangle\) on \(V\) is called a Euclidean space.
The mapping \(\Vert \cdot \Vert : V \to \mathbb{R}\) defined by the rule \[v\mapsto \Vert v \Vert=\sqrt{\langle v,v\rangle}\] for all \(v \in V\) is called the norm induced by \(\langle\cdot{,}\cdot\rangle\). Moreover, for any vector \(v \in V,\) the real number \(\Vert v \Vert\) is called the length of the vector \(v\).
The mapping \(d : V \times V \to \mathbb{R}\) defined by the rule \[(v_1,v_2) \mapsto d(v_1,v_2)=\Vert v_1-v_2\Vert\] for all \(v_1,v_2 \in V\) is called the metric induced by \(\langle\cdot{,}\cdot\rangle\) (or also metric induced by the norm \(\Vert\cdot\Vert\)). Furthermore, for any vectors \(v_1,v_2 \in V,\) the real number \(d(v_1,v_2)\) is called the distance from the vector \(v_1\) to the vector \(v_2\).
Recall that the angle between two non-zero vectors \(\vec{v}_1,\vec{v}_2 \in \mathbb{E}^2\) is defined as follows. The half lines spanned by \(\vec{v}_1\) and \(\vec{v}_2\) will each intersect the circle of radius \(1\) centred at the origin in exactly one point. Consequently, the circle of radius \(1\) is divided into two segments, depicted in red and blue in Figure 10.1. The minimum of the lengths of the two circle segments is the angle \(\theta\) between \(\vec{v}_1\) and \(\vec{v}_2.\)
It is tempting to use (10.3) as a definition of the angle between two vectors \(v_1,v_2\) in an abstract Euclidean space \((V,\langle\cdot{,}\cdot\rangle).\) That is, for any non-zero vectors \(v_1,v_2 \in V\) define the angle between \(v_1,v_2\) to be the unique real-number \(\theta \in [0,\pi]\) such that \[\cos\theta=\frac{\langle v_1,v_2\rangle}{\Vert v_1\Vert \Vert v_2\Vert}.\] Since the cosine is a bijective mapping from \([0,\pi]\) into \([-1,1],\) this definition only makes sense if the quotient \(\langle v_1,v_2\rangle/(\Vert v_1\Vert \Vert v_2\Vert)\) lies in the interval \([-1,1]\) for all pairs \(v_1,v_2 \in V\) of non-zero vectors. That this is indeed the case follows from one of the most important inequalities in mathematics (recall that for \(x \in \mathbb{R}\) we write \(|x|\) for the absolute value of \(x\)):
Proposition 10.8 • Cauchy–Schwarz inequality
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a Euclidean space. Then, for any two vectors \(v_1,v_2 \in V,\) we have \[\tag{10.1} |\langle v_1,v_2\rangle|\leqslant \Vert v_1\Vert \Vert v_2\Vert.\] Furthermore, \(|\langle v_1,v_2\rangle|=\Vert v_1\Vert \Vert v_2\Vert\) if and only if \(\{v_1,v_2\}\) are linearly dependent.
By the Cauchy–Schwarz inequality we thus have for all non-zero vectors \(v_1,v_2 \in V\) \[0\leqslant \frac{|\langle v_1,v_2\rangle|}{\Vert v_1\Vert \Vert v_2\Vert}\leqslant 1,\] so that \(\langle v_1,v_2\rangle/(\Vert v_1\Vert \Vert v_2\Vert) \in [-1,1].\) This allows to define:
Definition 10.9 • Angle between two vectors
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a Euclidean space and \(v_1,v_2 \in V\) two non-zero vectors. The angle between the vectors \(v_1\) and \(v_2\) is the unique real number \(\theta \in [0,\pi]\) such that \[\cos\theta=\frac{\langle v_1,v_2\rangle}{\Vert v_1\Vert \Vert v_2\Vert}.\]
Remark 10.10
Notice that two non-zero vectors \(v_1,v_2\) in \(\mathbb{E}^2\) are orthogonal in the sense that \(\langle v_1,v_2\rangle=0\) if and only if they are perpendicular, that is, the angle between \(v_1\) and \(v_2\) is \(\pi/2.\)
Proof of Proposition 10.8. First consider the case where \(v_2=0_V.\) Then both sides of (10.1) are \(0,\) hence the inequality holds and, moreover, \(v_1\) and \(v_2\) are linearly dependent.
Let therefore be \(v_1,v_2 \in V\) with \(v_2\neq 0_V\) and consider the function \(p : \mathbb{R}\to \mathbb{R}\) defined by the rule \[p(t)=\langle v_1+tv_2,v_1+tv_2\rangle\] for all \(t \in \mathbb{R}.\) Using the bilinearity and the symmetry of \(\langle\cdot{,}\cdot\rangle,\) we expand \[p(t)=\langle v_1,v_1\rangle+2t\langle v_1,v_2\rangle+t^2\langle v_2,v_2\rangle=\Vert v_1\Vert^2+2t\langle v_1,v_2\rangle+t^2\Vert v_2\Vert^2.\] Since \(v_2\neq 0_V,\) the function \(p\) is a polynomial of degree \(2\) in the variable \(t.\) If the discriminant of \(p\) is positive, then \(p\) has two distinct zeros and attains both positive and negative values. The bilinear form \(\langle\cdot{,}\cdot\rangle\) is positive definite, hence we have \(p(t)\geqslant 0\) for all \(t \in \mathbb{R}\) and the discriminant \(\Delta\) of \(p\) must be non-positive \[\Delta=4\left(\langle v_1,v_2\rangle^2-\Vert v_1\Vert^2 \Vert v_2\Vert^2\right) \leqslant 0.\] Taking the square root implies (10.1).
If \(v_1,v_2\) are linearly dependent and since \(v_2\neq 0_V,\) there exists a scalar \(s\) such that \(v_1=sv_2.\) Hence \(\Vert v_2\Vert \Vert v_1\Vert =|s|\Vert v_2\Vert^2=|\langle sv_2,v_2\rangle|\) and equality holds in (10.1).
Conversely, suppose that \(\langle v_1,v_2\rangle=\pm \Vert v_1\Vert \Vert v_2\Vert.\) Then we obtain \[p(t)=\Vert v_1\Vert^2\pm 2t \Vert v_1\Vert \Vert v_2\Vert+t^2\Vert v_2\Vert^2=(\Vert v_1\Vert \pm t \Vert v_2\Vert)^2.\] Taking \(t_0=\mp \Vert v_1\Vert/\Vert v_2\Vert\) gives \(p(t_0)=\langle v_1+t_0 v_2,v_1+t_0v_2\rangle=0.\) Since \(\langle\cdot{,}\cdot\rangle\) is positive definite, this implies that \(v_1+t_0v_2=0_V\) and hence \(v_1,v_2\) are linearly dependent.
Example 10.11 • Cauchy–Schwarz inequality
Consider \(V=\mathbb{R}^n\) equipped with the standard scalar product \(\langle\cdot{,}\cdot\rangle.\) The Cauchy–Schwarz inequality translates to the statement that for all \(\vec{x}=(x_i)_{1\leqslant i\leqslant n}\) and \(\vec{y}=(y_i)_{1\leqslant i\leqslant n} \in \mathbb{R}^n,\) we have \[\Big|\sum_{i=1}^n x_iy_i\Big|\leqslant \sqrt{\sum_{i=1}^n(x_i)^2}\sqrt{\sum_{i=1}^n(y_i)^2}.\]
For \(V=\mathsf{C}([a,b],\mathbb{R})\) and inner product defined as in Example 10.4 (iii) above, taking the square of the Cauchy–Schwarz inequality, we obtain that for all \(f,g \in V\) \[\Big|\int_{a}^bf(x)g(x)\mathrm{d}x\Big|^2\leqslant \int_a^b f(x)^2\mathrm{d}x\int_a^b g(x)^2\mathrm{d}x.\]
The norm induced by an inner product satisfies a few elementary properties:
Proposition 10.12 • Properties of the norm
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a Euclidean space with induced norm \(\Vert \cdot \Vert : V \to \mathbb{R}.\) Then
for all \(v \in V\) we have \(\Vert v \Vert \geqslant 0\) and \(\Vert v \Vert=0\) if and only if \(v=0_V\);
for all \(s\in \mathbb{R}\) and all \(v \in V\) we have \(\Vert sv\Vert=|s|\Vert v\Vert\);
for all vectors \(v_1,v_2 \in V,\) we have the so-called triangle inequality \[\Vert v_1+v_2\Vert\leqslant \Vert v_1\Vert+\Vert v_2\Vert.\]
Proof. The first two properties follow immediately from the definition of \(\Vert \cdot \Vert\) and the positive definiteness of \(\langle\cdot{,}\cdot\rangle.\) Using the Cauchy–Schwarz inequality (10.1), we obtain for all \(v_1,v_2 \in V\) \[\begin{aligned} \Vert v_1+v_2\Vert^2&=\langle v_1+v_2,v_1+v_2\rangle=\langle v_1,v_1\rangle+2\langle v_1,v_2\rangle+\langle v_2,v_2\rangle \\ &\leqslant \Vert v_1\Vert^2+2|\langle v_1,v_2\rangle|+\Vert v_2\Vert^2 \leqslant \Vert v_1 \Vert^2+2\Vert v_1\Vert \Vert v_2\Vert+\Vert v_2\Vert^2\\ &=\left(\Vert v_1\Vert+\Vert v_2\Vert\right)^2, \end{aligned}\] and where we also used that \(\langle v_1,v_2\rangle\leqslant |\langle v_1,v_2\rangle|.\) Since both \(\Vert v_1+v_2\Vert\geqslant 0\) and \(\Vert v_1\Vert+\Vert v_2\Vert\geqslant 0,\) taking the square root implies \[\Vert v_1+v_2\Vert\leqslant \Vert v_1\Vert+\Vert v_2\Vert,\] as claimed.
Likewise, we obtain:
Proposition 10.13 • Properties of the metric
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a Euclidean space with induced metric \(d : V \times V \to \mathbb{R}.\) Then for all \(v_1,v_2,v_3 \in V\) we have
\(d(v_1,v_2)=0\) if and only if \(v_1=v_2\);
\(d(v_1,v_2)=d(v_2,v_1)\) (symmetry);
\(d(v_1,v_3)\leqslant d(v_1,v_2)+d(v_2,v_3)\) (triangle inequality).
Proof. Exercise.
10.2 The orthogonal projection
In the Euclidean setting, the restriction of an inner product to a subspace is again an inner product:
Lemma 10.14
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a Euclidean space and \(U\subset V\) a subspace. Then the restriction \(\langle\cdot{,}\cdot\rangle|_U\) of \(\langle\cdot{,}\cdot\rangle\) to \(U\) is an inner product and hence \((U,\langle\cdot{,}\cdot\rangle|_U)\) is a Euclidean space as well.
Proof. Symmetry and positive definiteness holds for all vectors or pairs of vectors in \(V,\) hence also for all vectors or pairs of vectors in \(U\subset V.\)
Remark 10.15
Since an inner product is a map \(\langle\cdot{,}\cdot\rangle: V\times V \to \mathbb{R},\) it would be more precise to write \(\langle\cdot{,}\cdot\rangle|_{U\times U}\) and speak of the restriction of \(\langle\cdot{,}\cdot\rangle\) to \(U\times U.\) For simplicity, we well use the terminology of Lemma 10.14.
Recall that a projection is an endomorphism \(\Pi : V \to V\) which satisfies \(\Pi \circ \Pi=\Pi\) and that for a projection \(\Pi : V \to V\) we have \(V=\operatorname{Ker}\Pi\oplus\operatorname{Im}\Pi.\) Given two subspaces \(U_1\) and \(U_2\) of \(V\) such that \(V=U_{1}\oplus U_2,\) we can write every vector \(v \in V\) uniquely as a sum \(v=u_1+u_2\) where \(u_i\in U_i\) for \(i=1,2.\) The mapping \(\Pi : V \to V\) defined by the rule \(\Pi(v)=u_1\) for all \(v \in V\) thus is a projection with \(\operatorname{Im}\Pi=U_1\) and \(\operatorname{Ker}\Pi=U_2.\) Notice that \(\Pi\) is the unique projection whose image is \(U_1\) and whose kernel is \(U_2.\) If \(\hat{\Pi} : V \to V\) is another projection with this property, then we have for all \(v\in V\) \[\hat{\Pi}(v)=\hat{\Pi}(u_1+u_2)=\hat{\Pi}(u_1)\] Since \(u_1 \in U_1=\operatorname{Im}\hat{\Pi},\) we can write \(u_1=\hat{\Pi}(w)\) for some vector \(w \in V,\) hence \(\hat{\Pi}(u_1)=\hat{\Pi}(\hat{\Pi}(w))=\hat{\Pi}(w)=u_1.\) We thus have \[\hat{\Pi}(v)=\hat{\Pi}(u_1)=u_1=\Pi(v)\] so that \(\hat{\Pi}=\Pi.\) This shows that there is precisely one projection with \(\operatorname{Im}\Pi=U_1\) and \(\operatorname{Ker}\Pi=U_2.\)
Remark 10.16
By Lemma 10.14 and Remark 10.3, the restriction of an inner product \(\langle\cdot{,}\cdot\rangle\) on a finite dimensional vector space \(V\) to a subspace \(U\subset V\) is always non-degenerate. Therefore, by Corollary 9.24, the orthogonal subspace \(U^{\perp}\) is always a complement to \(U,\) so that \(V=U\oplus U^{\perp}\) and \[\dim U^{\perp}=\dim V-\dim U\] by Remark 6.7 and Proposition 6.12.
This allows to define:
Definition 10.17 • Orthogonal projection
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a finite dimensional Euclidean space and \(U\subset V\) a subspace. The projection whose image is \(U\) and whose kernel is \(U^{\perp}\) is called the orthogonal projection onto the subspace \(U\) and will be denoted by \(\Pi_U^{\perp}.\)
While the existence of the orthogonal projection onto a subspace \(U\) of a Euclidean space \((V,\langle\cdot{,}\cdot\rangle)\) follows abstractly from the above considerations, it is illustrative to give an explicit geometric construction. We first consider the case where \(U\) is spanned by a non-zero vector \(u \in V.\) We define a linear map \(\Pi_U^{\perp} : V \to V\) by the rule \[\Pi^{\perp}_U(v)=\frac{\langle v,u\rangle}{\langle u,u\rangle}u\] for all \(v \in V.\) Then \(\Pi_U^{\perp}(u)=u\) and \(\operatorname{Ker}\Pi_U^{\perp}=\left\{v \in V\,|\, \langle v,u\rangle=0\right\}=U^{\perp}.\) Since \(\Pi_U^{\perp}(v)=su\) for some scalar \(s\in \mathbb{K},\) we conclude that \(\Pi_U^{\perp}\circ \Pi_U^{\perp}=\Pi_U^{\perp},\) hence \(\Pi_U^{\perp}\) is the orthogonal projection onto \(U.\)
In general, we have:
Proposition 10.18
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a finite dimensional Euclidean space and \(U\subset V\) a subspace of dimension \(k \in \mathbb{N}.\) Let \(\{u_1,\ldots,u_k\}\) be an orthogonal basis of \(U,\) then the map \(\Pi_U^{\perp} : V \to V\) defined by the rule \[\tag{10.2} \Pi_U^{\perp}(v)=\sum_{i=1}^k \frac{\langle v,u_i\rangle}{\langle u_i,u_i\rangle}u_i\] for all \(v \in V\) is the orthogonal projection onto \(U.\)
Proof. Let \(n=\dim V.\) Notice that since \(\langle\cdot{,}\cdot\rangle\) is positive definite, we must have \(\langle u_i,u_i\rangle>0\) for \(1\leqslant i\leqslant k,\) hence the map \(\Pi_U^{\perp}\) is well defined. For \(1\leqslant j\leqslant k\) we obtain \[\Pi_U^{\perp}(u_j)=\sum_{i=1}^k \frac{\langle u_j,u_i\rangle}{\langle u_i,u_i\rangle}u_i=\frac{\langle u_j,u_j\rangle}{\langle u_j,u_j\rangle}u_j=u_j,\] where we use the orthogonality of the basis \(\{u_1,\ldots,u_k\}.\) By definition, for all \(v \in V\) we have \(\Pi_U^{\perp}(v)=\sum_{i=1}^ks_i u_i\) for scalars \(s_i=\frac{\langle v,u_i\rangle}{\langle u_i,u_i\rangle}.\) Since \(\Pi_U^{\perp}(u_i)=u_i,\) we obtain \[\Pi_U^{\perp}(\Pi_U^{\perp}(v))=\Pi_U^{\perp}\left(\sum_{i=1}^k s_i u_i\right)=\sum_{i=1}^k s_i\Pi_U^{\perp}(u_i)=\sum_{i=1}^k s_iu_i=\Pi_U^{\perp}(v).\] Hence we have \(\Pi_U^{\perp}\circ \Pi_U^{\perp}=\Pi_U^{\perp}\) and \(\Pi_U^{\perp}\) is a projection.
By Remark 10.16 we can write \(V=U\oplus U^{\perp}\) and by Theorem 3.64 we can find a basis \(\{u_{k+1},\ldots,u_n\}\) of \(U^{\perp}\) so that \(\{u_1,\ldots,u_k,u_{k+1},\ldots,u_n\}\) is a basis of \(V.\) Let \(v \in V.\) We write \(v=\sum_{j=1}^n t_j u_j\) for scalars \(t_j,\) \(1\leqslant j\leqslant n.\) Then \(v\) lies in the kernel of \(\Pi_U^{\perp}\) if and only if we have \[0_V=\Pi_U^{\perp}(v)=\sum_{i=1}^k \frac{\Big\langle \sum_{j=1}^nt_ju_j,u_i\Big\rangle}{\langle u_i,u_i\rangle}u_i=\sum_{i=1}^k\sum_{j=1}^n t_j\frac{\langle u_j,u_i\rangle}{\langle u_i,u_i\rangle}u_i=\sum_{i=1}^k t_iu_i,\] where we use that the vectors \(\{u_1,\ldots,u_k\}\) are orthogonal and that \(\{u_{k+1},\ldots,u_n\} \in U^{\perp}.\) The vector \(v\) thus lies in the kernel of \(\Pi_U^{\perp}\) if and only if \(v=\sum_{i=k+1}^n t_iu_i,\) that is, if and only if \(v \in U^{\perp}.\) The map \(\Pi_U^{\perp}\) thus is the orthogonal projection on \(U.\)
Remark 10.19
Let \((V,\langle\cdot{,}\cdot\rangle)\) be a finite-dimensional Euclidean space and \(U\subset V\) a subspace. Then for all \(v \in V\) we can write \(v=v-\Pi_U^{\perp}(v)+\Pi_U^{\perp}(v).\) Since \(\Pi_U^{\perp}(v) \in U\) and \(V=U\oplus U^{\perp},\) it follows that the vector \[v_{\perp_U}=v-\Pi_U^{\perp}(v) \in U^{\perp}\] and moreover, \(v_{\perp_U}=0_V\) if and only if \(v \in U.\)
Exercises
Exercise 10.20
Let \(\vec{v}_1,\vec{v}_2 \in \mathbb{E}^2\) be two non-zero vectors. Show that the angle \(\theta\) between \(\vec{v}_1\) and \(\vec{v}_2\) satisfies \[\tag{10.3} \langle \vec{v}_1,\vec{v}_2\rangle=\Vert \vec{v}_1\Vert\Vert \vec{v}_2\Vert \cos\theta.\]
Solution
First note that a rotation \(f_{\mathbf{R}_\varphi}:\mathbb{R}^2\to\mathbb{R}^2\) about any angle \(\varphi\in \mathbb{R}\) leaves the angle between \(\vec v_1\) and \(\vec v_2\) and \(\langle\cdot{,}\cdot\rangle\) invariant, since \[\langle \mathbf{R}_\varphi \vec v_1,\mathbf{R}_\varphi \vec v_2\rangle = \langle \vec v_1,\mathbf{R}_\varphi^T\mathbf{R}_\varphi \vec v_2\rangle = \langle \vec v_1,\vec v_2\rangle.\] We might therefore pick \(\varphi\) in such a way that \(\mathbf{R}_\varphi\vec v_1\) is a multiple of \(\vec e_1\) so that \[\begin{aligned} \langle \vec v_1,\vec v_2\rangle & = \|\vec v_1\|\|\vec v_2\|\left \langle \frac{\vec v_1}{\| \vec v_1\|}, \frac{\vec v_2 }{\|\vec v_2\|}\right\rangle= \|\vec v_1\|\|\vec v_2\|\left \langle \mathbf{R}_\varphi \frac{\vec v_1}{\| \vec v_1\|}, \mathbf{R}_\varphi \frac{\vec v_2 }{\|\vec v_2\|}\right\rangle \\ & = \|\vec v_1\|\|\vec v_2\|\left \langle \vec e_1,\begin{pmatrix}{\cos (\pm\theta)}\\{\sin (\pm\theta)}\end{pmatrix}\right\rangle = \|\vec v_1\|\|\vec v_2\| \cos\theta. \end{aligned}\]