{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nConsider a [[hidden Markov model|HJVM390354]] with hidden states $\\set{1, \\dots, N}$ and observation symbols $\\set{1, \\dots, M}$.\nIts parameter $\\theta = (\\pi, A, B)$ consists of an initial distribution $\\pi_i$, a transition matrix $a_{ij}$ and emission probabilities $b_i(k)$.\nLet $y_1, \\dots, y_T$ with $T \\ge 2$ be an observed sequence.\n\nFor the current $\\theta$, the forward and backward variables are\n$$\\alpha_t(i) = P_\\theta(Y_1 = y_1, \\dots, Y_t = y_t, X_t = i), \\qquad \\beta_t(i) = P_\\theta(Y_{t+1} = y_{t+1}, \\dots, Y_T = y_T \\mid X_t = i),$$\ncomputed by $\\alpha_1(i) = \\pi_i b_i(y_1)$, $\\alpha_{t+1}(j) = \\sum_i \\alpha_t(i) a_{ij} b_j(y_{t+1})$, $\\beta_T(i) = 1$ and $\\beta_t(i) = \\sum_j a_{ij} b_j(y_{t+1}) \\beta_{t+1}(j)$.\nWith $L = \\sum_i \\alpha_T(i)$, set\n$$\\gamma_t(i) = \\frac{\\alpha_t(i) \\beta_t(i)}{L}, \\qquad \\xi_t(i, j) = \\frac{\\alpha_t(i) a_{ij} b_j(y_{t+1}) \\beta_{t+1}(j)}{L}.$$\nThe *Baum–Welch algorithm* repeats the following update until the [[likelihood|0VW9E4QXR6]] $L$ stops changing:\n$$\\pi_i \\leftarrow \\gamma_1(i), \\qquad a_{ij} \\leftarrow \\frac{\\sum_{t=1}^{T-1} \\xi_t(i, j)}{\\sum_{t=1}^{T-1} \\gamma_t(i)}, \\qquad b_i(k) \\leftarrow \\frac{\\sum_{t \\,:\\, y_t = k} \\gamma_t(i)}{\\sum_{t=1}^{T} \\gamma_t(i)}.$$\n\n## Remarks\n\nHere $\\gamma_t(i)$ is the [[conditional probability|0990W5PN13]] that $X_t = i$ given the observations, and $\\xi_t(i, j)$ is the conditional probability that $X_t = i$ and $X_{t+1} = j$.\nThe update is one step of the [[expectation–maximization algorithm|11N1AXSXYK]], so the likelihood never decreases from one iteration to the next.\nIt may converge to a local maximum rather than to the global maximum, so several starting points are often used.\n\nExample: with $N = 1$ every $\\gamma_t(1)$ equals $1$, and the update gives $b_1(k) = \\#\\set{t : y_t = k} / T$.\nThis is the observed frequency of symbol $k$, which is the maximum likelihood estimate in that case.\n\nNon-example: the Viterbi algorithm uses the same model but returns the single most likely hidden path for fixed parameters.\nIt does not update $\\theta$, so it is not the Baum–Welch algorithm.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "W7P6CCSMX7",
    "kind": "definition",
    "msc": [
      "62M05",
      "62F10"
    ],
    "name": "Baum–Welch algorithm",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Baum–Welch_algorithm"
      }
    ],
    "tag": "T5TJRYZPFX",
    "uses": [
      "HJVM390354",
      "11N1AXSXYK",
      "0VW9E4QXR6",
      "0990W5PN13"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/T5TJRYZPFX/",
  "links": [
    {
      "field": "defines",
      "target": "W7P6CCSMX7"
    },
    {
      "field": "uses",
      "target": "HJVM390354"
    },
    {
      "field": "uses",
      "target": "11N1AXSXYK"
    },
    {
      "field": "uses",
      "target": "0VW9E4QXR6"
    },
    {
      "field": "uses",
      "target": "0990W5PN13"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/T5TJRYZPFX.md",
  "msc": [
    "62M05",
    "62F10"
  ],
  "name": "Baum–Welch algorithm",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/T5T/T5TJRYZPFX.md",
  "status": null,
  "tag": "T5TJRYZPFX",
  "uri": "m2://T5TJRYZPFX",
  "url": "https://tamnd.github.io/math2/wiki/baum-welch-algorithm-definition/",
  "writing": {
    "tag": "T5TJRYZPFX",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}