Chapter 5 — Inner products and orthogonality

Summary

This important chapter introduces the concept of an inner product and the structures that follows from it, notabily, the concept of orthogonality and orthogonal projections. In particular, working with a basis in an infinite-dimensional vector space becomes more easy with an inner product (and associated norm) instead of “just a norm”, when using orthogonality. In a Hilbert space (=metric complete inner product space), a set of vectors that is complete (the linear span defines a dense subspace) can be turned into an orthonormal set (using Gram-Schmidt) which then defines a basis (expansion theorem). Linear maps and linear operators between Hilbert spaces can also have more structure. Bounded linear functionals (elements from the dual space) are one-to-one associated with vectors in the primal space via the inner product. Bounded linear maps have adjoints. Linear operators can satisfy relations with their adjoint, e.g. they can be equal (self-adjoint), the adjoint can be the inverse (unitary) or they can commute with the adjoint (normal), which then imposes particular constraints on the spectrum and the eigenvectors. On the practical side, the orthogonal projection allows to construct least square solutions to overdetermined systems.

Not covered in class

Chapter 5 is one of the most important chapters of the course and has been covered completely.

Important concepts

• Bilinear form, sesquilinear form, quadratic form, symmetric and Hermitian, degenerate, positive (semi)definite versus indefinite, matrix congruence (= basis transform for bilinear forms)
• Inner product, standard/Euclidean inner product (in \(\mathbb{C}^n\), in \(\ell^2\), in \(L^2\)), metric/Gram matrix, inner product norm, continuity of the inner product, Hilbert space (=metric complete inner product space)
• Orthogonality, orthonormal set, orthogonal complement, orthogonal projection, orthogonal direct sum decomposition, orthogonal projector
• Orthonormal basis, Plancherel/Parseval identity, Gram-Schmidt orthonormalisation, QR decomposition
• Riesz representation theorem: (Anti)-isomorphism between dual space (= bounded linear functionals) and original Hilbert space
• Bounded linear maps, operator norm expressed using inner product, bounded linear maps have closed kernels
• Adjoint of a linear map, self-adjoint operators, isometric and unitary maps, normal operator
• Least squares solution, Moore-Penrose pseudoinverse

For applications / exercises

• Computing inner products, applying Gram-Schmidt (e.g. with custom inner products or between functions in a function space)
• Knowing and using the properties of self-adjoint and normal operators