Fisher–Neyman factorization theorem

Remarks

In the discrete case the proof is a direct computation: if the factorization holds, then $P_\theta(X = x \mid T = t) = h(x)/\sum_{y : T(y) = t} h(y)$, which does not depend on $\theta$; conversely one takes $g_\theta(t) = P_\theta(T = t)$ and $h(x) = P(X = x \mid T = T(x))$. The general case is due to Halmos and Savage. It first replaces $\mu$ by a probability measure $\lambda = \sum_j c_j P_{\theta_j}$, a countable convex combination equivalent to the whole family. It then uses Radon–Nikodym derivatives with respect to $\lambda$. Example: for i.i.d. $\mathcal{N}(\mu, \sigma^2)$ data, $$f_{(\mu, \sigma^2)}(x) = (2\pi\sigma^2)^{-n/2} \exp\left(-\frac{\sum_i x_i^2 - 2\mu \sum_i x_i + n\mu^2}{2\sigma^2}\right),$$ so $T(x) = \left(\sum_i x_i, \sum_i x_i^2\right)$ is sufficient for $(\mu, \sigma^2)$, with $h = 1$. Example: for i.i.d. uniform data on $[0, \theta]$, $f_\theta(x) = \theta^{-n}\mathbf{1}\set{\max_i x_i \le \theta}\mathbf{1}\set{\min_i x_i \ge 0}$, so $\max_i x_i$ is sufficient. The domination hypothesis is needed for the densities $f_\theta$ to exist, so without it the criterion cannot even be stated.

Going deeper

For a student

The factorization criterion is the standard way to find sufficient statistics in practice. For an exponential family with density $h(x)\exp(\eta(\theta) \cdot T(x) - A(\theta))$, it shows at once that $T$ is sufficient. It also shows that the likelihood function, up to a factor free of $\theta$, depends on the data only through any sufficient statistic, which links sufficiency with the likelihood principle. It is the first step towards the Rao–Blackwell theorem and the Lehmann–Scheffé theorem.