Numerical differentiation

Remarks

Example: the central difference $D_h f(x) = \frac{f(x + h) - f(x - h)}{2h}$ for $k = 1$, with stencil $-1, 1$ and $a = (-\tfrac12, \tfrac12)$, has order $2$. Check with Taylor's theorem for $f \in C^3$: $f(x \pm h) = f(x) \pm h f'(x) + \frac{h^2}{2} f''(x) \pm \frac{h^3}{6} f'''(\xi_\pm)$, so $D_h f(x) - f'(x) = O(h^2)$. The forward difference $\frac{f(x + h) - f(x)}{h}$ has only order $1$, with error $\frac{h}{2} f''(\xi)$. Non-example: $\frac{f(x+h) + f(x)}{h}$ approximates no derivative; for $f = 1$ it equals $\frac{2}{h}$, which does not tend to $f'(x) = 0$. In floating-point arithmetic with unit round-off $u$, the forward difference has total error about $\frac{h}{2}\abs{f''} + \frac{2u\abs{f}}{h}$, which is smallest for $h$ of the order of $\sqrt{u}$.