Tensor Products Explained: Intuition, Definition, Examples

The tensor product V ⊗ W is a new vector space built from two vector spaces V and W, designed so that bilinear maps out of V × W become ordinary linear maps out of V ⊗ W. Its basic building blocks are symbols v ⊗ w, one for every pair of vectors, combined using the rules of bilinearity.

The real world is multilinear: stress relates two directions at once, quantum systems combine by tensoring state spaces, and multi-axis data tables are tensors.

Key Concepts

• Elementary tensor. The symbol v ⊗ w; a general tensor is a sum of these, not always a single one.
• Bilinearity rules. (v1 + v2) ⊗ w = v1 ⊗ w + v2 ⊗ w, v ⊗ (w1 + w2) = v ⊗ w1 + v ⊗ w2, (cv) ⊗ w = v ⊗ (cw) = c(v ⊗ w).
• Dimension formula. dim(V ⊗ W) = (dim V)(dim W); the ei ⊗ fj form a basis.
• Universal property. Every bilinear B: V × W → U factors as B(v, w) = T(v ⊗ w) for a unique linear T.
• Rank-one tensors. v ⊗ w corresponds to a rank-one outer-product matrix in coordinates.

Formulas

• Basis: {ei ⊗ fj} is a basis, giving mn basis tensors.
• Outer product: u ⊗ v is the m × n matrix with entries uivj, of rank 1 (unless u or v is zero).
• Kronecker product: (A ⊗ B)(v ⊗ w) = (Av) ⊗ (Bw).
• Assumptions and limits: the dimension formula needs finite-dimensional spaces.

Worked Example

Build a basis of R² ⊗ R³ and compute one outer product.

1. Take the standard bases e1 = (1, 0), e2 = (0, 1) of R² and f1 = (1, 0, 0), f2 = (0, 1, 0), f3 = (0, 0, 1) of R³.
2. The six tensors e1 ⊗ f1, e1 ⊗ f2, e1 ⊗ f3, e2 ⊗ f1, e2 ⊗ f2, e2 ⊗ f3 form a basis, so dim(R² ⊗ R³) = 2 × 3 = 6.
3. Compute the elementary tensor u ⊗ v for u = (1, 2) and v = (3, −1, 2) as an outer product: entry (i, j) is uivj.
4. Row 1: 1 × (3, −1, 2) = (3, −1, 2). Row 2: 2 × (3, −1, 2) = (6, −2, 4). So u ⊗ v = [[3, −1, 2], [6, −2, 4]].
5. Check rank: row 2 is exactly 2 × row 1, so the matrix has rank 1, as every nonzero elementary tensor must.

Common Misconceptions

• Thinking v ⊗ w = w ⊗ v. The product is not commutative; they live in different spaces when V ≠ W.
• Assuming every tensor is elementary. Most are sums; the smallest number needed is the tensor's rank.
• Confusing tensors with matrices. A matrix can encode a linear map or a bilinear pairing, but those are different ideas.

Real-World Applications

• Quantum computing: two qubits live in C² ⊗ C²; entangled states are the non-elementary tensors.
• Continuum mechanics: the stress tensor pairs surface direction with force direction.
• Machine learning: multi-way data arrays are handled with tensor decompositions.

The tensor product turns bilinearity into linearity: v ⊗ w is bilinear in its factors, dim(V ⊗ W) = (dim V)(dim W), and every bilinear map factors through it. From quantum entanglement to the stress inside a bridge, multilinear phenomena live in tensor product spaces.