Appendix J. Common Identities and Formulas

This appendix collects frequently used identities from linear algebra, matrix algebra, vector calculus, and numerical computation. The goal is reference rather than proof. Most formulas are proved earlier in the text.

J.1 Algebraic Identities

Difference of Squares

$$ a^2-b^2=(a-b)(a+b). $$

Binomial Expansion

$$ (x+y)^n = \sum_{k=0}^n \binom{n}{k} x^{n-k}y^k. $$

Geometric Series

For (x\neq 1),

$$ 1+x+x^2+\cdots+x^n = \frac{x^{n+1}-1}{x-1}. $$

If

$$ |x|<1, $$

then the infinite series converges:

$$ \sum_{k=0}^{\infty}x^k = \frac{1}{1-x}. $$

Quadratic Formula

For

$$ ax^2+bx+c=0, \qquad a\neq 0, $$

the solutions are

$$ x = \frac{-b\pm\sqrt{b^2-4ac}}{2a}. $$

J.2 Complex Number Identities

For

$$ z=a+bi, $$

the conjugate is

$$ \overline{z}=a-bi. $$

Modulus

$$ |z| = \sqrt{a^2+b^2}. $$

Product with Conjugate

$$ z\overline{z} = |z|^2. $$

Reciprocal

If (z\neq 0),

$$ z^{-1} = \frac{\overline{z}}{|z|^2}. $$

Euler Formula

$$ e^{i\theta} = \cos\theta+i\sin\theta. $$

Polar Multiplication

$$ (re^{i\theta})(se^{i\phi}) = rs,e^{i(\theta+\phi)}. $$

J.3 Vector Identities

Dot Product

For

$$ x,y\in\mathbb{R}^n, $$

$$ x\cdot y = x^Ty = \sum_{i=1}^n x_i y_i. $$

Euclidean Norm

$$ |x|_2 = \sqrt{x^Tx}. $$

Distance Formula

$$ d(x,y) = |x-y|. $$

Cauchy-Schwarz Inequality

$$ |\langle x,y\rangle| \leq |x|,|y|. $$

Triangle Inequality

$$ |x+y| \leq |x|+|y|. $$

Parallelogram Identity

$$ |x+y|^2+|x-y|^2 = 2|x|^2+2|y|^2. $$

J.4 Matrix Addition and Multiplication

Matrix Addition

If (A,B\in F^{m\times n}),

$$ (A+B){ij} = a{ij}+b_{ij}. $$

Matrix Multiplication

If

$$ A\in F^{m\times n}, \qquad B\in F^{n\times p}, $$

then

$$ (AB){ij} = \sum{k=1}^n a_{ik}b_{kj}. $$

Associativity

$$ A(BC)=(AB)C. $$

Distributivity

$$ A(B+C)=AB+AC. $$

$$ (A+B)C=AC+BC. $$

Scalar Compatibility

$$ (cA)B = A(cB) = c(AB). $$

Noncommutativity

In general,

$$ AB\neq BA. $$

J.5 Transpose Identities

Transpose of Sum

$$ (A+B)^T = A^T+B^T. $$

Transpose of Product

$$ (AB)^T = B^TA^T. $$

Double Transpose

$$ (A^T)^T=A. $$

Inverse of Transpose

$$ (A^T)^{-1} = (A^{-1})^T. $$

when (A) is invertible.

J.6 Conjugate Transpose Identities

Conjugate Transpose of Product

$$ (AB)^* = B^A^. $$

Double Conjugate Transpose

$$ (A^)^=A. $$

Inverse Relation

$$ (A^)^{-1} = (A^{-1})^. $$

for invertible (A).

J.7 Determinant Identities

Determinant of Product

$$ \det(AB) = \det(A)\det(B). $$

Determinant of Transpose

$$ \det(A^T)=\det(A). $$

Determinant of Inverse

$$ \det(A^{-1}) = \frac{1}{\det(A)}. $$

Determinant of Triangular Matrix

For triangular (A),

$$ \det(A) = \prod_{i=1}^n a_{ii}. $$

Invertibility Criterion

$$ A \text{ invertible} \iff \det(A)\neq 0. $$

J.8 Trace Identities

Definition

$$ \operatorname{tr}(A) = \sum_{i=1}^n a_{ii}. $$

Linearity

$$ \operatorname{tr}(A+B) = \operatorname{tr}(A) + \operatorname{tr}(B). $$

Scalar Multiplication

$$ \operatorname{tr}(cA) = c,\operatorname{tr}(A). $$

Cyclic Property

$$ \operatorname{tr}(AB) = \operatorname{tr}(BA). $$

More generally,

$$ \operatorname{tr}(ABC) = \operatorname{tr}(BCA) = \operatorname{tr}(CAB). $$

J.9 Inverse Identities

Inverse of Product

$$ (AB)^{-1} = B^{-1}A^{-1}. $$

Identity Inverse

$$ I^{-1}=I. $$

Inverse of Diagonal Matrix

If

$$ D=\operatorname{diag}(d_1,\ldots,d_n), $$

with all (d_i\neq 0), then

$$ D^{-1} = \operatorname{diag} \left( \frac{1}{d_1}, \ldots, \frac{1}{d_n} \right). $$

J.10 Rank Identities

Rank Bound

If

$$ A\in F^{m\times n}, $$

then

$$ \operatorname{rank}(A) \leq \min(m,n). $$

Rank-Nullity Theorem

For a linear map

$$ T:V\to W, $$

$$ \dim(V) = \operatorname{rank}(T) + \operatorname{nullity}(T). $$

Rank of Product

$$ \operatorname{rank}(AB) \leq \min( \operatorname{rank}(A), \operatorname{rank}(B) ). $$

J.11 Orthogonality Identities

Orthogonal Matrix

$$ Q^TQ=I. $$

Unitary Matrix

$$ U^*U=I. $$

Norm Preservation

If (Q) is orthogonal,

$$ |Qx|_2 = |x|_2. $$

Orthogonal Projection

If (P) is an orthogonal projection,

$$ P^2=P, \qquad P^T=P. $$

J.12 Eigenvalue Identities

Eigenvalue Equation

$$ Av=\lambda v. $$

Characteristic Polynomial

$$ p_A(\lambda) = \det(\lambda I-A). $$

Sum of Eigenvalues

The sum of eigenvalues equals the trace:

$$ \sum_i \lambda_i = \operatorname{tr}(A). $$

Product of Eigenvalues

The product of eigenvalues equals the determinant:

$$ \prod_i \lambda_i = \det(A). $$

Similarity Invariance

If

$$ B=P^{-1}AP, $$

then (A) and (B) have the same eigenvalues.

J.13 Diagonalization Identities

If

$$ A=PDP^{-1}, $$

then

$$ A^k = PD^kP^{-1}. $$

If

$$ D= \operatorname{diag}(\lambda_1,\ldots,\lambda_n), $$

then

$$ D^k = \operatorname{diag} (\lambda_1^k,\ldots,\lambda_n^k). $$

J.14 Singular Value Decomposition

If

$$ A=U\Sigma V^*, $$

then:

Property	Formula
(U) unitary	(U^*U=I)
(V) unitary	(V^*V=I)
Singular values	Diagonal entries of (\Sigma)
Eigenvalues of (A^*A)	(\sigma_i^2)

Frobenius Norm from Singular Values

$$ |A|_F^2 = \sum_i \sigma_i^2. $$

Spectral Norm

$$ |A|2 = \sigma{\max}(A). $$

J.15 Least Squares Formulas

For the least squares problem

$$ \min_x |Ax-b|_2^2, $$

the normal equations are

$$ A^TAx=A^Tb. $$

If the columns of (A) are linearly independent, then

$$ x = (A^TA)^{-1}A^Tb. $$

Projection Matrix

The orthogonal projection onto the column space of (A) is

$$ P = A(A^TA)^{-1}A^T. $$

J.16 Calculus Identities

Derivative of Power

$$ \frac{d}{dx}x^n = nx^{n-1}. $$

Product Rule

$$ (fg)' = f'g+fg'. $$

Chain Rule

$$ (f\circ g)' = (f'\circ g)g'. $$

Gradient of Quadratic Form

If

$$ f(x)=x^TAx, $$

then

$$ \nabla f(x) = (A+A^T)x. $$

If (A) is symmetric,

$$ \nabla f(x)=2Ax. $$

Hessian of Quadratic Form

If (A) is symmetric,

$$ \nabla^2(x^TAx)=2A. $$

J.17 Matrix Calculus Identities

Derivative of Linear Form

$$ \nabla_x(c^Tx)=c. $$

Derivative of Norm Squared

$$ \nabla_x |x|_2^2 = 2x. $$

Derivative of Least Squares Objective

If

$$ f(x)=|Ax-b|_2^2, $$

then

$$ \nabla f(x) = 2A^T(Ax-b). $$

J.18 Numerical Computation Identities

Residual

For approximate solution (\widehat{x}),

$$ r=b-A\widehat{x}. $$

Relative Error

$$ \frac{|x-\widehat{x}|}{|x|}. $$

Condition Number

$$ \kappa(A) = |A|,|A^{-1}|. $$

Floating-Point Model

$$ \operatorname{fl}(a\circ b) = (a\circ b)(1+\delta), \qquad |\delta|\leq u. $$

J.19 Probability and Statistics Identities

Mean

For data points (x_1,\ldots,x_n),

$$ \mu = \frac{1}{n} \sum_{i=1}^n x_i. $$

Variance

$$ \operatorname{Var}(x) = \frac{1}{n} \sum_{i=1}^n (x_i-\mu)^2. $$

Covariance Matrix

For centered vectors (x_i),

$$ C = \frac{1}{n} \sum_{i=1}^n x_ix_i^T. $$

Covariance matrices are symmetric and positive semidefinite.

J.20 Fourier and Orthogonality Identities

Fourier Coefficient

$$ c_k = \langle f,\phi_k\rangle. $$

Orthogonality Relation

$$ \langle \phi_i,\phi_j\rangle = 0, \qquad i\neq j. $$

Parseval Identity

$$ |f|^2 = \sum_k |c_k|^2. $$

J.21 Common Matrix Factorizations

Factorization	Form
LU decomposition	(A=LU)
QR decomposition	(A=QR)
Cholesky decomposition	(A=LL^T)
Eigenvalue decomposition	(A=PDP^{-1})
Singular value decomposition	(A=U\Sigma V^*)
Schur decomposition	(A=QTQ^*)

J.22 Summary

The identities in this appendix appear repeatedly throughout linear algebra, numerical computation, optimization, statistics, and applied mathematics.

Several themes recur:

Theme	Representative identity
Structure preservation	((AB)^T=B^TA^T)
Geometry	(\langle x,y\rangle=x^Ty)
Invertibility	(\det(A)\neq 0\iff A^{-1}\text{ exists})
Orthogonality	(Q^TQ=I)
Spectral structure	(Av=\lambda v)
Optimization	(A^TAx=A^Tb)
Numerical analysis	(\kappa(A)=|A||A^{-1}|)

These formulas form the computational and theoretical vocabulary of linear algebra.