After the news that an American soldier apparently committed mass murder in Afghanistan, some reported that he had served multiple combat tours, had suffered a head injury, and had marital problems. Were those factors the cause of his actions?
That line of reasoning did not resonate with some of those closest to the situation, such as this fellow soldier, who was quoted as saying:
I know a lot of people who’ve done multiple deployments, and they always come back so level headed, that I’ve seen.
Last week I had one of those embarrassing moments when I tried to explain something statistical in class using example numbers off the top of my head. It was odds ratios. A bedrock of social science research, odds ratio reporting often causes confusion. In my case, the odds of me screwing up the example are much greater when I don’t first work it out with a spreadsheet.
Now odds ratios are indirectly in the news, and the seminar meets again tonight, so I worked up a quick example using mass murder, and comparing it to something more mundane: cavities (using fake data).
The first general problem is just the idea of causal effects in general. Smoking causes people to die from lung cancer, and the majority of smokers don’t die from lung cancer. It increases the risk of lung cancer, the odds of dying from lung cancer. It causes lung cancer, just not all the time. Fortunately for anti-smoking education efforts, lung cancer is common enough that it’s easy to see the connection. But with rare events — like mass murder — this is easily confused.
Again, made-up data. Suppose there are 20,000 soldiers on a base, and 18,000 of them are on their first deployment. From previous research, we believe that .01% of soldiers commit mass murder, so we expect (statistically) 2 mass murders on the base. But when a soldier on his fourth deployment commits mass murder, someone hypothesizes that it’s caused by the stress of his multiple deployments.
Among those serving multiple deployments, the mass murder rate on this base is .05%. If the overall rate is going to be .01%, resulting in two mass murders (which we hope is not the case), the rate for those on first deployments would have to be a little more than half that, about .0055%. In that case, I would say mass murder is “9.091-times more common” among those serving multiple deployments, because .05/.0055 = 9.091.
The odds of committing mass murder for the two groups are 1/1,999 and 1/17,999 respectively, and the ratio of those odds is 9.004. So I would say the “odds of mass murder are 9.004-times greater” among those serving multiple deployments. That’s a big difference. It doesn’t mean deployments cause mass murder, but the hypothesis is still standing.
The difference between the rate ratio (9.091) and the odds ratio (9.004) isn’t big, but with more common events this can add to the confusion. I’ve added a (fictional data) cavity example for comparison. The hypothesis here is that eating sugar causes cavities.
Overall, 5% of kids get cavities (ok, bad example), and the odds of getting cavities are 500/9,500, or .05263. In the cavity example, as with the mass murders, it just so happens that the two groups are expected to experience the same number of cavities, but the “no sugar” group is 9-times larger (as if). So the relative rates are .25000/.02778 = 9.0. And the odds ratio is (250/750)/(250/8,750) = .3333/.0286 = 11.667.
Take-home points from these fake examples:
- The vast majority of soldiers don’t commit mass murder regardless of whether they are on their first deployment or not, and the vast majority of kids don’t get cavities regardless of whether they eat sugar or not.
- The odds ratios for mass murder associated with multiple deployments, and cavities associated with sugar, are very large. That is true even though the total number of mass murders, like the total number of kids with cavities, is expected to be the same in each pair of comparison groups.
- We cannot reject the possibility that these hypotheses are not true. Further research may be warranted.
If I screwed this example up, too, please let so thanks for letting me know so I can could correct it.)