{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $X_1, \\dots, X_n$ be observations with values in a set $\\mathcal{X}$, and let $\\Theta$ be a parameter set.\nLet $\\rho \\colon \\mathcal{X} \\times \\Theta \\to \\R$ be a function.\nAn *M-estimator* with loss $\\rho$ is an [[estimator|0TTNRP5PFQ]] $\\hat{\\theta}_n$ that minimizes\n$$\\theta \\mapsto \\sum_{i=1}^n \\rho(X_i, \\theta)$$\nover $\\Theta$.\n\n## Remarks\n\nWhen $\\Theta \\subseteq \\R^k$ is open and $\\rho$ is differentiable in $\\theta$ with gradient $\\psi(x, \\theta) = \\nabla_\\theta \\rho(x, \\theta)$, an interior minimizer solves $\\sum_{i=1}^n \\psi(X_i, \\theta) = 0$.\nAn estimator defined as a root of such an equation, for a given function $\\psi$, is also called an M-estimator (of $\\psi$-type), or a Z-estimator.\n\nExample: with $\\rho(x, \\theta) = (x - \\theta)^2$ on $\\R$, the sum is minimized at the [[sample mean|0QNRBHWV35]], since its derivative $-2\\sum_i (X_i - \\theta)$ vanishes only at $\\theta = \\bar{X}$.\nWith $\\rho(x, \\theta) = \\abs{x - \\theta}$, the minimizers are the sample medians.\nWith $\\rho(x, \\theta) = -\\log f(x; \\theta)$ for a family of densities, the M-estimator is the [[maximum likelihood estimator|0FREPFKNS7]].\n\nNon-example: the midrange $(\\min_i X_i + \\max_i X_i)/2$ minimizes $\\max_i \\abs{X_i - \\theta}$, a maximum over the observations rather than a sum of terms $\\rho(X_i, \\theta)$, so this description does not exhibit it as an M-estimator.\n\nUnder regularity conditions, $\\sqrt{n}(\\hat{\\theta}_n - \\theta_0)$ is asymptotically normal with the sandwich covariance $A^{-1} B A^{-T}$, where $A = E[\\nabla_\\theta \\psi(X, \\theta_0)]$ and $B = E[\\psi(X, \\theta_0)\\psi(X, \\theta_0)^T]$, and $\\theta_0$ minimizes $E[\\rho(X, \\theta)]$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "4503R8BVF8",
    "kind": "definition",
    "msc": [
      "62F10",
      "62F35",
      "62G35"
    ],
    "name": "M-estimator",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/M-estimator"
      }
    ],
    "tag": "7A55AWN7EV",
    "uses": [
      "0TTNRP5PFQ",
      "0FREPFKNS7",
      "0QNRBHWV35"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/7A55AWN7EV/",
  "links": [
    {
      "field": "defines",
      "target": "4503R8BVF8"
    },
    {
      "field": "uses",
      "target": "0TTNRP5PFQ"
    },
    {
      "field": "uses",
      "target": "0FREPFKNS7"
    },
    {
      "field": "uses",
      "target": "0QNRBHWV35"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/7A55AWN7EV.md",
  "msc": [
    "62F10",
    "62F35",
    "62G35"
  ],
  "name": "M-estimator",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/7A5/7A55AWN7EV.md",
  "status": null,
  "tag": "7A55AWN7EV",
  "uri": "m2://7A55AWN7EV",
  "url": "https://tamnd.github.io/math2/wiki/m-estimator-definition/",
  "writing": {
    "tag": "7A55AWN7EV",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}