---
tag: 8X7293647G
kind: definition
name: Score
msc: [62F10, 62B10]
defines: CP97GRVTQ0
uses: [05FSTXM92W, 0NS2QYVC4Y, 0DCT0QZVQ1]
sources:
  - url: https://en.wikipedia.org/wiki/Informant_function
---

## Definition

Let $\set{f(\cdot; \theta) : \theta \in \Theta}$ be a family of probability densities with respect to a measure $\nu$, with $\Theta$ an open subset of $\R^k$.
Let $x$ be a point with $f(x; \theta) > 0$ near a parameter $\theta$, and assume that $\theta \mapsto \log f(x; \theta)$ is differentiable at $\theta$.
The *score* of $\theta$ at $x$ is the [[gradient|0NS2QYVC4Y]] of the log-[[likelihood|05FSTXM92W]]:
$$s(\theta; x) = \nabla_\theta \log f(x; \theta) = \left(\frac{\partial \log f(x; \theta)}{\partial \theta_1}, \dots, \frac{\partial \log f(x; \theta)}{\partial \theta_k}\right)^T.$$
For i.i.d. data $x_1, \dots, x_n$, the score of the sample is $\sum_{i=1}^n s(\theta; x_i)$.

## Remarks

If differentiation under the integral sign is allowed in $\int f(x; \theta)\, d\nu(x) = 1$, then $E_\theta[s(\theta; X)] = 0$, where $E_\theta$ is the [[expectation|0DCT0QZVQ1]] when $X$ has density $f(\cdot; \theta)$.
Its covariance matrix $E_\theta[s(\theta; X) s(\theta; X)^T]$ is then the Fisher information.

Example: for $f(x; \mu) = (2\pi)^{-1/2} e^{-(x - \mu)^2/2}$, $\log f(x; \mu) = -\frac{1}{2}\log(2\pi) - \frac{1}{2}(x - \mu)^2$, so $s(\mu; x) = x - \mu$, which has mean $0$ and variance $1$ when $X \sim \mathcal{N}(\mu, 1)$.

Non-example: for the uniform densities $f(x; \theta) = \theta^{-1}\mathbf{1}[0 < x < \theta]$ with $\theta > 0$, the map $\theta \mapsto \log f(x; \theta)$ is not differentiable (it jumps from $-\infty$) at $\theta = x$.
Where it is defined, the score is $-1/\theta$, whose mean is not $0$: the support depends on $\theta$, and differentiation under the integral fails.