Systems of Nonlinear Equations: Newton's Method Intro Guide

Real problems rarely have one equation and one unknown. A chemical reaction, a circuit, or a robot arm is described by a system of nonlinear equations — several equations that must all hold at once, with unknowns in squares, products, or trigonometric functions. Newton's method extends to this setting: instead of dividing by f′(x), we solve a linear system built from the Jacobian matrix, which collects all partial derivatives of all equations.

Almost every serious engineering calculation ends here: circuit simulators, structural analysis, weather models, and machine-learning optimizers all repeatedly solve nonlinear systems, with the vector version of Newton's method as their engine. It turns the one-dimensional Newton formula from a trick into a general-purpose tool.

Key Concepts

• System: F(x) = 0, with F a vector of n functions and x a vector of n unknowns; a solution zeroes every equation at once.
• Jacobian matrix J: the n × n matrix of J_ij = ∂f_i/∂x_j — it plays the role of f′ in the one-dimensional method.
• Initial guess vector: even more sensitive to the starting point than the 1D version — a poor guess can diverge or land on a different solution.
• Convergence check: stop when the size (norm) of F(x) or the size of the step is below the tolerance.

Formulas

• Newton step for systems: x^(k+1) = x^(k) − J(x^(k))^−1 F(x^(k)). In practice the inverse is never formed; instead solve J(x^(k)) s = −F(x^(k)) for the step s and set x^(k+1) = x^(k) + s.
• Jacobian entries: J_ij = ∂f_i/∂x_j, evaluated at the current guess.
• Convergence: quadratic near a solution with nonsingular Jacobian, assuming F is smooth.

Worked Example

Solve f₁ = x² + y² − 4 = 0 and f₂ = x − y = 0 (circle of radius 2 meets line y = x) from (1, 1). The Jacobian is J = [[2x, 2y], [1, −1]]. At (1, 1): F = (−2, 0), J = [[2, 2], [1, −1]]. Solve J·s = (2, 0): the second row gives s₁ = s₂, the first gives 4s₁ = 2, so s₁ = s₂ = 0.5 → new point (1.5, 1.5). Next: F = (0.5, 0), J = [[3, 3], [1, −1]] → s₁ = s₂ = −0.083333 → (1.416667, 1.416667). True solution (√2, √2) ≈ (1.414214, 1.414214): two steps are accurate to two decimals — quadratic convergence again.

Common Misconceptions

• "The Jacobian is always invertible." It can be singular at some points, just as f′ can be zero in one dimension. Then the linear system has no unique step and the method breaks down.
• "One Newton step solves the system." Only if the system is linear. For nonlinear systems each step is just an approximation, and several iterations are needed.
• "Nonlinear systems have one solution." The circle and the line above intersect at two points, (√2, √2) and (−√2, −√2). Different starting guesses can find different solutions.

Real-World Applications

• Electrical circuit simulators (like SPICE) solve thousands of nonlinear equations for voltages and currents at each time step.
• Robotics: inverse kinematics computes the joint angles that place a robot hand at a target position.
• Chemistry: equilibrium concentrations in reactions with several species satisfy coupled nonlinear equations.
• Machine learning: training neural networks solves (approximately) huge nonlinear systems for the weights.

Newton's method for systems replaces division by f′ with solving a linear system from the Jacobian: x^(k+1) = x^(k) − J^−1F. It keeps quadratic speed, needs a good initial guess, and powers the equation-solving inside nearly every engineering simulation.