Numerical Differentiation Formulas Explained with Examples

Numerical differentiation estimates the derivative of a function using only its values at nearby points — no calculus rules needed. If you know a car's position at 2.0 seconds and at 2.1 seconds, its average velocity over that interval approximates the instantaneous velocity at 2.05 seconds. That simple idea, refined with symmetric point arrangements, is the whole subject: replace the limit in the derivative definition with values at a small but finite spacing h.

This matters whenever the function is not given by a formula: experimental data, simulation output, or a black-box computer program. Optimization algorithms, physics engines, and machine-learning trainers all need gradients of functions they can only evaluate, and numerical differentiation provides them. It also teaches a deep lesson: making h smaller reduces the formula error but amplifies round-off error, so there is an optimal h.

Key Concepts

• Forward difference: uses f(x) and f(x + h); simple but only first-order accurate.
• Backward difference: uses f(x − h) and f(x); the mirror image of forward.
• Central difference: uses f(x + h) and f(x − h) symmetrically; the errors partly cancel, giving second-order accuracy — much better for the same h.
• Step size h: the spacing between sample points, in the same units as x. Too large gives formula (truncation) error; too small gives round-off error from subtracting nearly equal numbers.

Formulas

• Forward difference: f′(x) ≈ (f(x + h) − f(x)) / h, with truncation error O(h).
• Backward difference: f′(x) ≈ (f(x) − f(x − h)) / h, with truncation error O(h).
• Central difference: f′(x) ≈ (f(x + h) − f(x − h)) / (2h), with truncation error O(h²). This assumes f is smooth (three times differentiable).
• Second derivative: f″(x) ≈ (f(x + h) − 2f(x) + f(x − h)) / h², with truncation error O(h²).

Worked Example

Estimate f′(2) for f(x) = x³, whose true derivative is 3x² = 12 at x = 2. Take h = 0.1. Forward difference: (f(2.1) − f(2))/0.1 = (9.261 − 8)/0.1 = 1.261/0.1 = 12.61, an error of 0.61. Central difference: (f(2.1) − f(1.9))/0.2 = (9.261 − 6.859)/0.2 = 2.402/0.2 = 12.01, an error of only 0.01. The central formula is sixty times more accurate with the same step size, because its leading error terms cancel. Halving h to 0.05 would cut the central error roughly by four (O(h²)) but only halve the forward error (O(h)) — another reason central differences are preferred.

Common Misconceptions

• "Smaller h is always better." Below about h = 10^−8 in double precision, subtracting f(x + h) − f(x − h) loses digits to round-off faster than the formula improves. There is an optimal h balancing the two errors.
• "Numerical derivatives are exact." They are approximations with a known error order. For exact derivatives of a formula, use calculus or automatic differentiation instead.
• "You need the function's formula." No — only its values at the sample points. That is the entire point: it works on measured data where no formula exists.

Real-World Applications

• Physics simulations: velocities and accelerations from position data of experiments.
• Optimization: gradient descent approximates gradients when analytic ones are unavailable.
• Image processing: edge detection computes brightness gradients with difference formulas.
• Finance: option "Greeks" like delta are estimated by bumping inputs and differencing.

Numerical differentiation replaces the derivative's limit with function values at spacing h: central differences, (f(x+h) − f(x−h))/(2h), give O(h²) accuracy and beat one-sided formulas. The step size must balance truncation error against round-off, making h a genuine design choice rather than an afterthought.