Hilbert Spaces Explained: Inner Products to Fourier

A Hilbert space is a complete inner product space: a vector space with an inner product ⟨x, y⟩ that is linear in the first argument, conjugate-symmetric, and positive definite, complete in the norm ||x|| = sqrt(⟨x, x⟩). It is the infinite-dimensional analogue of Euclidean space, keeping angles, lengths, and orthogonality.

Hilbert spaces carry quantum mechanics, Fourier analysis, and signal processing. The inner product gives orthogonal projections, best approximations, and orthonormal bases, while completeness guarantees infinite orthogonal expansions converge. Every Hilbert space looks like l^2 in the right coordinates.

Key Concepts

• Inner product axioms: linearity in the first slot, ⟨y, x⟩ = conjugate of ⟨x, y⟩, and ⟨x, x⟩ >= 0 with equality only for x = 0.
• Induced norm: ||x|| = sqrt(⟨x, x⟩); the inner product can be recovered from the norm by the polarization identity.
• Orthogonality: x is orthogonal to y when ⟨x, y⟩ = 0; orthogonal sets of nonzero vectors are automatically linearly independent.
• Orthonormal basis: a maximal orthonormal set; every vector expands as x = sum ⟨x, e_i⟩ e_i.
• Examples: l^2 (square-summable sequences); L^2 (square-integrable functions); R^n and C^n with the dot product.
• Projection theorem: for a closed subspace M, every x decomposes uniquely as m + m^perp, and m is the closest point in M to x.

Formulas

• Cauchy-Schwarz inequality: |⟨x, y⟩| <= ||x|| ||y||, with equality exactly when x and y are linearly dependent; units are the product of the units of x and y.
• Parallelogram law: ||x + y||^2 + ||x - y||^2 = 2||x||^2 + 2||y||^2; a norm comes from an inner product exactly when this holds.
• Pythagorean theorem: if ⟨x, y⟩ = 0 then ||x + y||^2 = ||x||^2 + ||y||^2.
• Bessel's inequality: for any orthonormal set {e_i}, sum |⟨x, e_i⟩|^2 <= ||x||^2.
• Parseval's identity: for an orthonormal basis, ||x||^2 = sum |⟨x, e_i⟩|^2, so the expansion x = sum ⟨x, e_i⟩ e_i converges and preserves energy.

Worked Example

In R^3 with the dot product, expand x = (3, 4, 0) in the standard orthonormal basis e_1, e_2, e_3. The coefficients are c_1 = ⟨x, e_1⟩ = 3, c_2 = 4, c_3 = 0, so x = 3 e_1 + 4 e_2. Parseval's identity predicts ||x||^2 = 9 + 16 + 0 = 25, and directly ||x||^2 = 25. The projection onto span{e_1} is (3, 0, 0), and the error (0, 4, 0) is orthogonal to e_1, illustrating the projection theorem: the coefficient gives the closest point in the subspace.

Common Misconceptions

• "Every Banach space is a Hilbert space." Only those whose norm satisfies the parallelogram law; l^1 and l^infinity are Banach but not Hilbert.
• "Orthonormal bases are like finite bases." In infinite dimensions the expansion is an infinite series that converges in norm; it is a basis only in the Hilbert sense, not a Hamel basis.
• "Orthogonal means perpendicular in 3D." Orthogonality ⟨x, y⟩ = 0 generalizes perpendicularity to functions and sequences, where no picture in 3D exists.

Real-World Applications

• Quantum mechanics: states are unit vectors in a Hilbert space, observables are operators, and probabilities come from inner products via the Born rule.
• Signal processing: Fourier and wavelet expansions are orthonormal expansions in L^2; Parseval's identity says energy is preserved.
• Machine learning: kernel methods implicitly map data into a Hilbert space (the RKHS) where linear methods apply.
• Partial differential equations: weak solutions are found by orthogonal projection (Galerkin methods) in Sobolev Hilbert spaces.

A Hilbert space is Euclidean geometry with infinitely many dimensions. Cauchy-Schwarz and Parseval are the workhorse estimates, orthonormal bases turn vectors into coefficient sequences, and the projection theorem gives best approximations. If a norm satisfies the parallelogram law, you are in Hilbert territory.