Numerical analysis

Start here

• ConceptCondition number

  A condition number measures how much the output of a problem can change, relative to the size of a small relative change in its input. For an invertible matrix $A$ and a matrix norm, it is $\kappa(A) = \norm{A} \norm{A^{-1}}$. It bounds the relative error in the solution of $Ax = b$ caused by a relative error in $b$. A problem with a large condition number is ill-conditioned: rounding errors in the data can be strongly amplified, whatever algorithm is used.

• ConceptNumerical analysis

  Numerical analysis is the study of algorithms that compute approximate solutions to problems of continuous mathematics. It designs methods for tasks such as solving equations, computing integrals, finding eigenvalues and solving differential equations. It also proves how large the error of such a method can be, and how fast the method converges. Two typical sources of error are the truncation error of a finite method and the round-off error of floating-point arithmetic.

• ConceptFloating-point arithmetic

  Floating-point arithmetic is the way computers represent real numbers and compute with them. A floating-point number is a fixed number of significant digits times a power of a base, such as $1.0110 \times 2^{-3}$. The result of each operation is rounded to a nearby floating-point number, so most computations carry a small round-off error. Its relative precision is measured by the machine epsilon, and the IEEE 754 standard fixes the formats used in practice.

• ConceptIEEE 754 standard

  IEEE 754 is the technical standard for floating-point arithmetic used by almost all modern processors. It fixes the binary formats, such as single precision and double precision, and how their bits encode sign, exponent and significand. It requires the basic operations to be correctly rounded, and it defines special values: signed zeros, infinities, NaN and subnormal numbers. Because of it, the same floating-point program gives the same results on different machines, which makes error analysis portable.

• ConceptMachine epsilon

  Machine epsilon measures the relative precision of a system of floating-point arithmetic. It is the gap between $1$ and the next larger floating-point number, or, in another common convention, half of this gap. The half-gap, called the unit roundoff, bounds the relative error of rounding a real number to the nearest floating-point number. Almost every error bound in numerical analysis is stated in terms of it.

• ConceptUnit in the last place

  The unit in the last place, or ulp, of a floating-point number is the value of its last significant digit position. It is the spacing between neighbouring floating-point numbers at that size. Errors of computed results are often stated in ulps. For example, a result correctly rounded to nearest is within half an ulp of the exact value. The ulp of $1$ is the machine epsilon.