Spectral Theorem for Symmetric Matrices: Statement and

The spectral theorem says that a real symmetric matrix can be completely understood through its eigenvalues and eigenvectors: all its eigenvalues are real, and it has a full set of eigenvectors that are mutually perpendicular. This turns a dense matrix into a simple diagonal one, viewed from a rotated coordinate system.

Symmetric matrices appear everywhere: covariance matrices, stiffness matrices, undirected graph adjacencies, and Hessians in optimization. The spectral theorem is what makes principal component analysis and vibration analysis work.

Key Concepts

• Symmetric matrix. A square matrix A with A^T = A, meaning entry (i, j) equals entry (j, i). Only square matrices can be symmetric.
• Eigenvalue and eigenvector. A number λ and nonzero vector x with Ax = λx; for symmetric matrices every λ is real.
• Orthonormal set. Pairwise perpendicular vectors, each of length 1.
• Orthogonal matrix. Q with Q^T Q = I, so Q^−1 = Q^T; its columns are orthonormal.
• Spectral decomposition. A = QDQ^T, with D diagonal (eigenvalues) and Q's columns the orthonormal eigenvectors.

Formulas

• Statement: if A is real symmetric (n × n), then A = QDQ^T for diagonal D and orthogonal Q.
• Eigenvalues: the diagonal of D holds λ1, …, λn, all real, counting multiplicity.
• Powers: A^k = QD^kQ^T: raise the eigenvalues to the kth power in the same basis.
• Assumptions and limits: A must be real and symmetric; a non-symmetric matrix may have complex eigenvalues and need not diagonalize at all.

Worked Example

Diagonalize A = [[4, 1], [1, 4]] orthogonally.

1. Characteristic polynomial: det(A − λI) = (4 − λ)² − 1 = λ² − 8λ + 15 = (λ − 5)(λ − 3). Eigenvalues: λ1 = 5, λ2 = 3.
2. For λ1 = 5: (A − 5I)x = 0 gives −x1 + x2 = 0, so x = (1, 1). Normalize: q1 = (1/√2, 1/√2).
3. For λ2 = 3: (A − 3I)x = 0 gives x1 + x2 = 0, so x = (1, −1). Normalize: q2 = (1/√2, −1/√2). Note q1 · q2 = 0, as the theorem promises.
4. Assemble Q = [[1/√2, 1/√2], [1/√2, −1/√2]] and D = [[5, 0], [0, 3]].
5. Verify A = QDQ^T: QD = [[5/√2, 3/√2], [5/√2, −3/√2]]. Row 1 of QD times column 1 of Q^T: (5/√2)(1/√2) + (3/√2)(1/√2) = 5/2 + 3/2 = 4. Row 1 times column 2: 5/2 − 3/2 = 1. Row 2 times column 2: 5/2 + 3/2 = 4. The product is [[4, 1], [1, 4]] = A. Correct.

Common Misconceptions

• Applying it to any matrix. A = QDQ^T is guaranteed only for symmetric matrices; a general matrix may not even be diagonalizable.
• Thinking repeated eigenvalues break orthogonality. Symmetric matrices still supply a full orthonormal eigenbasis for repeated eigenvalues.
• Confusing Q^−1 with Q^T generally. For non-orthogonal diagonalization P^−1AP = D the inverse is not the transpose.

Real-World Applications

• Principal component analysis: eigenvectors of the covariance matrix give the directions of maximum variance.
• Vibration analysis: eigenvectors of the stiffness matrix are the natural modes; eigenvalues give squared frequencies.
• Optimization: the symmetric Hessian's eigenvalues classify a critical point as minimum, maximum, or saddle.

The spectral theorem reduces every real symmetric matrix to a diagonal matrix seen through a rotation: A = QDQ^T with real eigenvalues and orthonormal eigenvectors. Whenever symmetry appears in a problem, this decomposition is the first tool to reach for.