Numerical differentiation

Summary

Numerical differentiation approximates the derivative of a function from its values at finitely many points. The basic tools are difference quotients such as $\frac{f(x+h) - f(x-h)}{2h}$, whose error is controlled by Taylor's theorem. A small step $h$ reduces the truncation error but magnifies the round-off error, so the step must be chosen with care. Systematic formulas are given by finite difference coefficients, and automatic differentiation is an alternative that avoids truncation error altogether.