Let’s push the boundary of what we know about the inner product (dot product) to involve vector spaces and dive headfirst into the properties and definitions of Inner Product Spaces.
As seen in our previous lessons, an inner product is merely a generalization of the dot product. And we know that a vector space is generalized as a linear structure closed under addition and scalar multiplication.
Intuition Behind Inner Product Spaces
But, when we combine these two ideas, we gain some important features.
- Length (norm)
- Angle
- Orthogonality
Recall that the inner product of \(\mathbb{R}^{n}\) is a function \(\langle\vec{a}, \vec{b}\rangle\) defined by:
\begin{equation}
\langle\vec{a}, \vec{b}\rangle=(\vec{a})^T \cdot \vec{b}=a_1 b_1+a_2 b_2+\cdots+a_n b_n
\end{equation}
Understanding Inner Product Space Properties
An inner product on a vector space \(\mathrm{V}\) is a function or assignment that for any two vectors \(\mathrm{u}\) and \(\mathrm{v}\) in \(\mathrm{V}\), there is a real number \(\langle\vec{a}, \vec{b}\rangle\) satisfying the following properties for vectors a, b, w in \(\mathrm{V}\) and all scalars c:
- Symmetry: \(\langle\vec{a}, \vec{b}\rangle=\langle\vec{b}, \vec{a}\rangle\)
- Linearity: \(\langle\vec{a}+\vec{b}, \vec{w}\rangle=\langle\vec{a}, \vec{w}\rangle+\langle\vec{b}, \vec{w}\rangle\)
- Homogeneity: \(\langle\vec{c}, \vec{b}\rangle=c\langle\vec{a}, \vec{b}\rangle\)
- Positivity: \(\langle\vec{a}, \vec{a}\rangle \geq 0\) and \(\langle\vec{a}, \vec{a}\rangle=0\) if and only if \(\vec{a}=\overrightarrow{0}\)
Which mean, a vector space \(\mathrm{V}\) with an inner product is called an inner product space.
This seems easy enough, right?
And what’s amazing is that these axioms are similar to the ones we saw for the dot product, so it makes it easier to understand and use.
Let’s work through an example to help make sense of things.
Example of Inner Product Spaces
Let \(\mathbb{R}^{2}\) have the inner product \(\langle\vec{a}, \vec{b}\rangle=4 a_{1} b_{1}+5 a_{2} b_{2}\) with \(\vec{a}=(1,1)\) and \(\vec{b}=(5,-1)\). Find \(\|\vec{a}\|,\|\vec{b}\|\), and \(\|\langle\vec{a}, \vec{b}\rangle\|\).
First, we will recognize that we \(\vec{a}=(\underbrace{1}_{a_{1}}, \underbrace{1}_{a_{2}})\) and \(\vec{b}=(\underbrace{5}_{b_{1}}, \underbrace{-1}_{b_{2}})\)
Next, we will find \(\|\vec{a}\|\) knowing that \(\|\vec{a}\|=\sqrt{\langle\vec{a}, \vec{a}\rangle}\) and that if \(\langle\vec{a}, \vec{b}\rangle=4 a_{1} b_{1}+5 a_{2} b_{2}\) then \(\langle\vec{a}, \vec{a}\rangle=4 a_{1} a_{1}+5 a_{2} a_{2}\), so.
\begin{align*}
\|\vec{a}\| &= \sqrt{\langle\vec{a}, \vec{a}\rangle} \\
&= \sqrt{4 a_{1} a_{1}+5 a_{2} a_{2}} \\
&= \sqrt{4(1)(1)+5(1)(1)} \\
&= \sqrt{9} \\
&= 3
\end{align*}
Now we will find \(\|\vec{b}\|\) knowing that \(\| \vec{b}||=\sqrt{\langle\vec{b}, \vec{b}\rangle}\) and that if \(\langle\vec{a}, \vec{b}\rangle=4 a_{1} b_{1}+5 a_{2} b_{2}\) then \(\langle\vec{b}, \vec{b}\rangle=4 b_{1} b_{1}+5 b_{2} b_{2}\), so
\begin{align*}
\|\vec{b}\| &= \sqrt{\langle\vec{b}, \vec{b}\rangle} \\
&= \sqrt{4 b_{1} b_{1}+5 b_{2} b_{2}} \\
&= \sqrt{4(5)(5)+5(-1)(-1)} \\
&= \sqrt{105}
\end{align*}
And lastly, we will calculate \(\|\langle\vec{a}, \vec{b}\rangle\|\) knowing that \(\langle\vec{a}, \vec{b}\rangle=4 a_{1} b_{1}+5 a_{2} b_{2}\).
\begin{equation}
\begin{aligned}
\|\langle\vec{a}, \vec{b}\rangle\| & =\sqrt{\langle\vec{a}, \vec{b}\rangle} \\
& =\sqrt{4 a_1 b_1+5 a_2 b_2} \\
& =\sqrt{4(1)(5)+5(1)(-1)} \\
& =\sqrt{15}
\end{aligned}
\end{equation}
Easy!
And now, let’s look at inner product spaces involving polynomials.
Inner Product Spaces and Polynomials
Let \(\mathrm{V}\) be \(P_{2}\) with inner product given by evaluation at \(-1,0\), and 1 and where \(p(t)=4+t\) and \(q(t)=5-4 t^{2}\).
Compute \(\langle p, q\rangle\).
All this means we are tasked with finding the inner product of \(\mathrm{p}\) and \(\mathrm{q}\) for the three values given. So, let’s first find \(\mathrm{p}\) and \(\mathrm{q}\) for our three values.
\begin{equation}
\begin{gathered}
p(-1)=4+(-1)=3 \\
p(0)=4+(0)=4 \\
p(1)=4+(1)=5
\end{gathered}
\end{equation}
\begin{equation}
\begin{gathered}
q(-1)=5-4(-1)^2=1 \\
q(0)=5-4(0)^2=5 \\
q(1)=5-4(1)^2=1
\end{gathered}
\end{equation}
Now all we have to do is substitute our values into our formula for the inner product.
\begin{align*}
\langle p, q\rangle &= p(-1)q(-1) + p(0)q(0) + p(1)q(1) \\
&= (3)(1) + (4)(5) + (5)(1) \\
&= 3 + 20 + 5 \\
&= 28
\end{align*}
Super easy!
Inner Product Spaces and Orthogonality
Alright, now let’s dig deeper and extend these ideas to orthogonality by noticing the existence of orthogonal bases for a finite-dimensional subspace of an inner product space can be established using the Gram-Schmidt process.
What does this mean?
We can compute the orthogonal projection of a vector onto a subspace.
Let’s take a closer look.
Example of Orthogonal Projection in Inner Product Spaces
For example, let’s use our example above where \(\mathrm{V}\) be \(P_{2}\) with inner product given by evaluation at \(-1,0\), and 1 and where \(p(t)=4+t\) and \(q(t)=5-4 t^{2}\), and let’s compute the orthogonal projection of \(\mathrm{q}\) onto the subspace spanned by \(\mathrm{p}\), or \(\mathrm{p}\) and \(\mathrm{q}\).
So, from our previous lesson, we know that the formula for finding the orthogonal projection is:
\(\hat{y}=\operatorname{Proj}_{u} y\) where:
\begin{equation}
\hat{y}=\frac{\vec{y} \cdot \vec{u}_1}{\vec{u}_1 \cdot \vec{u}_1} \vec{u}_1+\frac{\vec{y} \cdot \vec{u}_2}{\vec{u}_2 \cdot \vec{u}_2} \vec{u}_2+\cdots+\frac{\vec{y} \cdot \vec{u}_p}{\vec{u}_p \cdot \vec{u}_p} \vec{u}_p
\end{equation}
But we’re going to make a minor adjustment to the notation for inner product spaces so that our new formula is:
\begin{equation}
\hat{q}=\operatorname{Proj}_p q \text { where } \hat{q}=\frac{\langle q, p\rangle}{\langle p, p\rangle} p
\end{equation}
So, all we have to do is find our inner products (dot products) and plug them into our handy-dandy formula!
\begin{align*}
\hat{q} &= \frac{\langle q, p\rangle}{\langle p, p\rangle} p \\
&= \frac{\langle 5-4t^2, 4+t \rangle}{\langle 4+t, 4+t \rangle} (4+t)
\end{align*}
Nice.
Inner Product Spaces and the Cauchy-Schwarz Inequality
But here’s where things get even better. We can apply the Pythagorean Theorem to the orthogonal decomposition of a vector to create one of the most widely used inequalities in all mathematics, the Cauchy-Schwarz Inequality.
The Cauchy-Schwarz Inequality applies to vector spaces that have an inner product space and states that for all \(\vec{a}, \vec{b}\) in \(\mathrm{V}\): \(\mid\langle\vec{a}, \vec{b}\rangle\mid \leq\|\vec{a}\|\|\vec{b}\|\)
And this can be extended to the triangle inequality where \(\|\vec{a}+\vec{b}\| \leq\|\vec{a}\|+\|\vec{b}\|\).
Inner Product Spaces for Continuous Real-Valued Functions
This also leads us naturally to a widely used inner product space for continuous real-valued functions.
If \(C[a, b]\) be the vector space of continuous functions \(\mathrm{f}\) and \(\mathrm{g}\) on the closed interval \([a, b]\), then the inner product is given by:
\begin{equation}
\langle f, g\rangle=\int_a^b f(x) g(x) d x
\end{equation}
Which means we will multiply two vectors and then sum them over the interval from a to b.
Let’s look at an example.
Cauchy-Schwarz Inequality Example
Suppose \(C[0,1]\) is the vector space for continuous real-valued functions with an inner product space defined by:
\begin{equation}
\langle f, g\rangle=\int_0^1 f(x) g(x) d x
\end{equation}
Verify the Cauchy-Schwarz inequality for \(f(x)=1\) and \(g(x)=x\).
Alright, so the Cauchy-Schwarz inequality states that:
\begin{equation}
\vec{a}, \vec{b} \text { in } \mathrm{V}:|\langle\vec{a}, \vec{b}\rangle| \leq\|\vec{a}\|\|\vec{b}\|
\end{equation}
Therefore for our given situation, we must use our integration skills and show that:
\begin{equation}
\|\vec{a}+\vec{b}\| \leq\|\vec{a}\|+\|\vec{b}\|
\end{equation}
\begin{align*}
& \langle f, g\rangle=\langle 1, x\rangle \\\\
& =\int_{0}^{1}(1)(x) d x \\\\
& =\int_{0}^{1} x d x \\\\
& =\left.\frac{1}{2} x^{2}\right|_{0} ^{1} \\\\
& =\frac{1}{2}(1)^{2}-\frac{1}{2}(0)^{2}\\\\
& =\frac{1}{2} \\\\
\end{align*}
\begin{align*}
& \|f\|=\sqrt{\langle f, f\rangle} \\\\
& =\sqrt{\langle 1,1\rangle} \\\\
& =\sqrt{\int_{0}^{1}(1)(1) d x} \\\\
& =\sqrt{\int_{0}^{1} 1 d x} \\\\
& =\left.\sqrt{x}\right|_{0} ^{1} \\\\
& =\sqrt{1}-\sqrt{0} \\\\
& =1 \\\\
\end{align*}
\begin{align*}
& \|g\|=\sqrt{\langle g, g\rangle} \\\\
& =\sqrt{\langle x, x\rangle} \\\\
& =\sqrt{\int_{0}^{1}(x)(x) d x} \\\\
& =\sqrt{\int_{0}^{1} x^{2} d x} \\\\
& =\left.\sqrt{\frac{1}{3} x^{3}}\right|_{0} ^{1}\\\\
& =\sqrt{\frac{1}{3}(1)^{3}}-\sqrt{\frac{1}{3}(0)^{3}} \\\\
& =\sqrt{\frac{1}{3}} \\\\
& =\frac{\sqrt{3}}{3}
\end{align*}
Now, all we have to do is substitute our values into our inequality and verify its veracity.
\begin{equation}
|\langle f, g\rangle| \leq\|f\|\|g\| \frac{1}{2} \leq(1)\left(\frac{\sqrt{3}}{3}\right)
\end{equation}
Next Steps
Awesome, right? Together in this video, you will:
- Explore definitions and properties of inner product spaces
- Examine length, distances, and orthogonality in these spaces
- Discover the connection between inner products, the Gram-Schmidt process, and the best approximation theorem
- Study the Cauchy-Schwarz inequality and the Triangle Inequality
- Learn how to use integral calculus for vector spaces of continuous functions
It’s going to be great, so let’s get to it.
Video Tutorial w/ Full Lesson & Detailed Examples
Get access to all the courses and over 450 HD videos with your subscription
Monthly and Yearly Plans Available
Still wondering if CalcWorkshop is right for you?
Take a Tour and find out how a membership can take the struggle out of learning math.