Gram Schmidt Orthogonalisation: Explained with Examples

Gram-Schmidt turns independent vectors into an orthogonal set spanning the same space, one at a time. Keep the first vector, subtract from the second its shadow along the first, subtract from the third its shadows along the first two, and continue, so each new vector is scrubbed of all previous directions.

Orthogonal bases are worth it: coordinates become simple inner products, projections become sums of one-dimensional pieces, and algorithms become stable. Gram-Schmidt underlies QR decomposition, least-squares solvers and the orthogonal polynomials used across physics and engineering.

Key Concepts

• Projection: proj_u(v) = ((v,u)/(u,u)) u, the component of v along u.
• Gram-Schmidt step: u1 = v1; u2 = v2 minus proj_u1(v2); u3 = v3 minus proj_u1(v3) minus proj_u2(v3); and so on.
• Output: u1 … uk pairwise orthogonal, same span as v1 … vk.
• Orthonormal version: divide each uj by its norm.
• QR decomposition: on matrix columns, Gram-Schmidt gives A = QR with orthonormal Q and upper-triangular R.

Formulas

• General step: uj = vj minus sum of proj_ui(vj) for i below j.
• Orthonormal coordinates: v = sum of (v,ei) ei, so each coordinate is one inner product.
• Span preserved: span(u1,…,uj) = span(v1,…,vj) at every step.

Worked Example

Orthogonalise v1 = (1,1,0), v2 = (1,0,1), v3 = (0,1,1). u1 = (1,1,0). For u2: (v2,u1) = 1, (u1,u1) = 2, so subtract (1/2)(1,1,0): u2 = (1/2,minus 1/2,1). For u3: (v3,u1) = 1 gives proj (1/2,1/2,0); (v3,u2) = 1/2 and (u2,u2) = 3/2 give coefficient 1/3, proj (1/6,minus 1/6,1/3). Then u3 = (0,1,1) minus (1/2,1/2,0) minus (1/6,minus 1/6,1/3) = (minus 2/3, 2/3, 2/3). Verify: (u1,u3) = minus 2/3 + 2/3 = 0; (u2,u3) = minus 1/3 minus 1/3 + 2/3 = 0. All pairs orthogonal, same span as the originals.

Common Misconceptions

• Order does not matter. It does: each step removes only earlier directions, so reordering gives a different valid set.
• It fixes dependent vectors. Dependence yields uj = 0, a division-by-zero warning; drop it and continue.
• Normalising breaks orthogonality. Rescaling keeps inner products zero; the span is unchanged.

Real-World Applications

• Least squares: QR via Gram-Schmidt stably solves overdetermined systems behind regression software.
• Signal processing: orthogonal filter banks separate signals into independent bands.
• Graphics: building an orthonormal camera frame from a view direction is one Gram-Schmidt step.

Gram-Schmidt orthogonalises incrementally: each vector keeps only the part outside previous directions. Subtract every earlier shadow, verify zero inner products, normalise at the end for an orthonormal basis.