Mixture model

Remarks

Equivalently, let $Z$ be a random label with $P(Z = k) = \pi_k$, and let $X$ have density $f_{\theta_k}$ given $Z = k$. Then $X$ has density $f$, so a mixture model is a latent variable model with hidden label $Z$. Example: $f(x) = \tfrac{1}{2} \varphi(x) + \tfrac{1}{2} \varphi(x - 3)$, where $\varphi$ is the standard normal density, is a two-component Gaussian mixture with $\pi_1 = \pi_2 = 1/2$. Non-example: $\tfrac{1}{2} \varphi(x) + \tfrac{1}{4} \varphi(x - 3)$ has weights adding up to $3/4$, so it is not a mixture and not even a probability density. The parameters are identifiable at best up to a permutation of the components, since relabelling the components does not change $f$. The same definition with discrete distributions replacing densities gives mixtures of discrete distributions.