Score

Remarks

If differentiation under the integral sign is allowed in $\int f(x; \theta)\, d\nu(x) = 1$, then $E_\theta[s(\theta; X)] = 0$, where $E_\theta$ is the expectation when $X$ has density $f(\cdot; \theta)$. Its covariance matrix $E_\theta[s(\theta; X) s(\theta; X)^T]$ is then the Fisher information.

Example: for $f(x; \mu) = (2\pi)^{-1/2} e^{-(x - \mu)^2/2}$, $\log f(x; \mu) = -\frac{1}{2}\log(2\pi) - \frac{1}{2}(x - \mu)^2$, so $s(\mu; x) = x - \mu$, which has mean $0$ and variance $1$ when $X \sim \mathcal{N}(\mu, 1)$.

Non-example: for the uniform densities $f(x; \theta) = \theta^{-1}\mathbf{1}[0 < x < \theta]$ with $\theta > 0$, the map $\theta \mapsto \log f(x; \theta)$ is not differentiable (it jumps from $-\infty$) at $\theta = x$. Where it is defined, the score is $-1/\theta$, whose mean is not $0$: the support depends on $\theta$, and differentiation under the integral fails.