Inner Products and Orthonormal Bases: Gram-Schmidt Guide

An inner product ⟨x, y⟩ pairs two vectors to produce a scalar, generalizing the dot product. It is linear in the first argument, satisfies ⟨y, x⟩ = conjugate of ⟨x, y⟩, and is positive definite: ⟨x, x⟩ >= 0 with equality only for zero. From it come length, angle, and orthogonality (⟨x, y⟩ = 0).

Inner products matter because orthogonality simplifies everything. In an orthonormal basis, coordinates are computed by inner products, projections become term-by-term, and the norm is the square root of summed squared coefficients. The Gram-Schmidt process builds such a basis from any independent set and powers QR factorization and Fourier series.

Key Concepts

• Inner product axioms linearity in the first slot, conjugate symmetry, positive definiteness.
• Orthogonal set distinct vectors pair to 0; nonzero orthogonal vectors are linearly independent.
• Orthonormal set: orthogonal plus every vector has norm 1, so ⟨e_i, e_j⟩ is 1 when i = j and 0 otherwise.
• Orthogonal projection: onto span{e_i}, the projection of x is sum ⟨x, e_i⟩ e_i, the closest point in the span to x.
• Gram-Schmidt process: subtract from each vector its projections onto previous orthonormal vectors, then normalize.
• Fourier coefficients c_i = ⟨x, e_i⟩ are the coordinates of x.

Formulas

• Gram-Schmidt step: w_k = v_k - the sum of ⟨v_k, e_i⟩ e_i over earlier i, then e_k = w_k / ||w_k||; each subtraction removes exactly the components along earlier directions.
• Expansion: x = sum ⟨x, e_i⟩ e_i for an orthonormal basis; coefficients come from inner products, no linear system needed.
• Cauchy-Schwarz |⟨x, y⟩| <= ||x|| ||y||; used to bound each Gram-Schmidt projection step.

Worked Example

Apply Gram-Schmidt to v_1 = (1,1,0), v_2 = (1,0,1), v_3 = (0,1,1) in R^3. First e_1 = v_1/sqrt(2) = (1/sqrt(2), 1/sqrt(2), 0). Next ⟨v_2, e_1⟩ = 1/sqrt(2), so w_2 = (1,0,1) - (1/2, 1/2, 0) = (1/2, -1/2, 1), with norm sqrt(3/2) = sqrt(6)/2, giving e_2 = (1/sqrt(6), -1/sqrt(6), 2/sqrt(6)). Then ⟨v_3, e_1⟩ = 1/sqrt(2) and ⟨v_3, e_2⟩ = 1/sqrt(6), so w_3 = (0,1,1) - (1/2, 1/2, 0) - (1/6, -1/6, 1/3) = (-2/3, 2/3, 2/3), with norm 2/sqrt(3), giving e_3 = (-1/sqrt(3), 1/sqrt(3), 1/sqrt(3)). Check: ⟨e_2, e_3⟩ = (-1 - 1 + 2)/sqrt(18) = 0.

Common Misconceptions

• "Gram-Schmidt changes the span." At each step the span of {e_1, …, e_k} equals the span of {v_1, …, v_k}; only the basis vectors change, never the space they span.
• "You must normalize at the end." Normalizing each w_k immediately keeps numbers small; postponing it works but invites arithmetic errors.
• "Orthogonal vectors are enough for easy coordinates." Without unit length, the expansion needs x = sum (⟨x, e_i⟩ / ||e_i||^2) e_i; orthonormality is what makes coefficients pure inner products.

Real-World Applications

• QR factorization Gram-Schmidt on matrix columns gives A = QR, behind least-squares solvers.
• Fourier series: sines and cosines are orthogonal in L^2; coefficients are inner products, computed without solving systems.
• Signal compression: projecting onto a few orthonormal basis vectors (DCT in JPEG) keeps most of the energy with few coefficients.
• Quantum computing: qubit states live in C^2 with the standard inner product, and orthonormal bases are measurement bases.

Inner products give length and angle, orthonormal bases give effortless coordinates, and Gram-Schmidt builds them from anything independent: subtract old directions, normalize, verify.