{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Counterexample\n\nThe claim refuted is this.\nIf treatment $A$ has a higher success rate than treatment $B$ in every subgroup, then $A$ has the higher rate in the whole population.\n\nConsider two subgroups, mild and severe cases, summarised in a [[contingency table|PANSMHE0SW]] for each subgroup.\n\n1. Mild cases: $A$ succeeds in $9$ of $10$ patients ($90\\%$), $B$ in $80$ of $100$ patients ($80\\%$).\n2. Severe cases: $A$ succeeds in $30$ of $100$ patients ($30\\%$), $B$ in $2$ of $10$ patients ($20\\%$).\n3. All cases: $A$ succeeds in $39$ of $110$ patients (about $35\\%$), $B$ in $82$ of $110$ patients (about $75\\%$).\n\nSo $A$ is better in each subgroup, yet $B$ is better overall.\n\n## Remarks\n\nThe reason is that the overall rate is a weighted average of the subgroup rates, and the weights differ between the treatments.\nMost patients given $A$ are severe cases, and most patients given $B$ are mild cases.\nIn terms of [[conditional probability|0990W5PN13]], $P(S \\mid A, G) \u003e P(S \\mid B, G)$ for each subgroup $G$ does not imply $P(S \\mid A) \u003e P(S \\mid B)$.\nThe same reversal can happen for correlations: a trend in each group can be reversed when the groups are pooled.\nWhich comparison is relevant depends on the causal structure, not on the numbers alone.\n\n## Explanations\n\n### L1 human\n\nA treatment can look better in every group of patients and still look worse when all patients are counted together.\nThis happens when the treatments were given to very different mixes of patients.\nHere the better treatment was mostly given to the hard cases, so its overall success rate is dragged down.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "kind": "counterexample",
    "msc": [
      "62H17",
      "62H20"
    ],
    "name": "Simpson's paradox",
    "same_as": {
      "wikidata": "Q757290",
      "wikipedia": "Simpson's paradox"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Simpson%27s_paradox"
      }
    ],
    "tag": "T060JHSH05",
    "uses": [
      "PANSMHE0SW",
      "0990W5PN13"
    ]
  },
  "kind": "counterexample",
  "link": "https://tamnd.github.io/math2/node/T060JHSH05/",
  "links": [
    {
      "field": "uses",
      "target": "PANSMHE0SW"
    },
    {
      "field": "uses",
      "target": "0990W5PN13"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/T060JHSH05.md",
  "msc": [
    "62H17",
    "62H20"
  ],
  "name": "Simpson's paradox",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/T06/T060JHSH05.md",
  "status": null,
  "tag": "T060JHSH05",
  "uri": "m2://T060JHSH05",
  "url": "https://tamnd.github.io/math2/wiki/simpsons-paradox/",
  "writing": {
    "tag": "T060JHSH05",
    "kind": "counterexample",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      },
      {
        "what": "edge refutes or not_instance_of out",
        "have": 0,
        "want": 1
      }
    ]
  }
}