Brad Wilcox has written up his best case for how marriage protects women and girls from violence. I discussed his initial post earlier, but the blowup has prompted me to provide more general advice for the critical data citizen — reader, writer, and editor — who has to decide what to believe when someone comes at them with a data story.
I have some tips about that at the end, but first this elaborate setup.
The information in this section is true
Consider three stories:
- When Melanie Thernstrom’s toddler, Kieran, first ate cheese, he immediately had a massive allergic attack. His face swelled, his skin turned red and scaly, and he started gasping for breath. They jumped in their car and rushed to the hospital, where doctors were able to save him.
- Chicago mother Tynisha Hilliard had six children in the car when someone opened fire. “Mommy, I’m shot,” said her nine-year-old boy from the back seat. Hilliard immediately sped to the nearest hospital. “My reaction was to save my son. That’s all I can do, save my son,” she said. After emergency surgery for a gunshot wound to the chest, the boy was expected to survive.
- When Dodgers catcher A. J. Ellis’s wife, Cindy, went into labor, they hopped in the car and headed for NYU hospital, normally a 35-minute drive. Despite racing through traffic with a police escort, they didn’t make it in time – the baby was born in the back seat – but they arrived at the hospital moments later, met by an emergency crew that whisked mother and child to care and safety in the hospital.
What do these stories have in common? Children’s lives saved by cars.
Is this part of a wider phenomenon? I know what you’re thinking: The pollution from cars hurts children, the vast resources devoted to infrastructure for cars could be spent instead in ways that help children, the need for gas causes wars all the time, and the individualism promoted by car culture contributes to social isolation instead of community efficacy.
Maybe. But let’s theorize a little. Here are three ways cars might be good for children’s health:
- Kids whose families have cars can get them to doctors in an emergency. Considering that in modern societies a lot of what kills children is various kinds of accidents and medical emergencies, this could be a major advantage.
- Say what you want about individualism, but it’s emerged as a modern character trait in tandem with the cultural shift that brought us the view of children as priceless individuals. Car culture is a major prop of individualism, so it’s reasonable to hypothesize that people who drive individual cars are more totally devoted to their priceless individual children’s well-being (rather than, say, the well-being of children in general).
- Being able to transport oneself at will — any time, any place — may create a sense of self-efficacy, of mastery over one’s environment, which makes people refuse to accept failure (or illness or death), and thus devote themselves more confidently to their survival and the survival of their children.
Don’t take a theoretical word for it, though — let’s go to the data. Here are three small studies.
Cars and children’s health across countries
First we examine the relationship between the number of passenger cars per capita and the rate of child malnutrition in 110 countries (all the countries in the World Bank’s database that have measures of both variables in the last 10 years — mostly poor countries). The largest — India, China, Brazil, and the USA — are highlighted (click to enlarge).
This is a very strong relationship. This single variable, cars per capita, statistically explains no less than 67% of the variation in child malnutrition rates.
But, you liberals object, cars are surely more common in wealthier countries, so this relationship may be spurious. Sure, income and cars are positively correlated (r=.86, in fact). But when I fit a regression model with both per capita income and per capita cars, cars still have a highly significant statistical association with malnutrition (p<.001). (All the regression models are in the appendix at the end.)
Cars and child death rates across US states
Second, we take a closer look within the United States. Here there is a lot less variation in both the number of cars and the condition of children. Still, there is a clear relationship between private cars per person and the death rate of children and teenagers: Children are substantially less likely to die in states with more privately owned passenger cars (click to enlarge).
Again, there is less variation in income between U.S. states than there is between countries of the world. But to make sure this is not just a function of state income, I fit a regression model with cars and a control for median household income. The statistical effect of private cars remains significant at the p<.05 level, confirming it is unlikely to be due to chance.
Car commuting and children’s disabilities within the US
Third, let’s go still further, not just comparing US states but comparing children according to the car-driving habits of their parents within the US. For this I got data on children’s disabilities (four kinds of disability) and the means of transportation to work for their parents using the 2010-2012 American Community Survey, with a sample of more than 700,000 children ages 5-11.
Sure enough, children who live with parents who drive to work are substantially less likely to have disabilities than those who don’t live with a parent who drives to work:
Again, could this be because richer families are more likely to include car-driving parents? The regressions (below) show that, although it is true that children in richer households are less likely to have disabilities, the statistical effect of parents’ commuting method remains highly significant in the model that includes household income.
In summary: Children are less likely to be malnourished if they live in a country with more cars per person; they are less likely to die if they live in a state with more cars per person, and they are less likely to have disabilities if they live with parents who commute to work by car. All of these relationships are statistically significant with controls for income (of the country, state, or family). These are facts.
One interpretation
Compare this analysis to the question of marriage and violence. In their piece for the Washington Post (discussed here), Brad Wilcox and Robin Fretwell Wilson wrote about #YesAllWomen:
This social media outpouring makes it clear that some men pose a real threat to the physical and psychic welfare of women and girls. But obscured in the public conversation about the violence against women is the fact that some other men are more likely to protect women, directly and indirectly, from the threat of male violence: married biological fathers. The bottom line is this: Married women are notably safer than their unmarried peers, and girls raised in a home with their married father are markedly less likely to be abused or assaulted than children living without their own father.
With the facts above I can accurately offer this parallel construction:
Some cars pose a real threat to the health and safety of children. But obscured in the public conversation about auto safety, pollution, and environmental degradation is the fact that some other cars are more likely to protect children, directly and indirectly, from threats to their health and safety: cars driven by their own, responsible, caring parents. The bottom line is this: Children in places with more cars — and in families where parents commute by car — are notably healthier than peers without cars.
At the end of his followup post, Brad concludes:
Of course, none of these studies definitively prove that marriage plays a causal role in protecting women and children. But they are certainly suggestive. What we do know is this: Intact families with married parents are typically safer for women and children. … That’s why the conversation about violence against women and girls … should incorporate the family factor into efforts to reduce the violence facing women and girls.
I am equally confident in my conclusion:
Of course, my brief studies don’t definitively prove that cars plays a causal role in protecting children’s health and safety. But they are certainly suggestive. What we do know is this: Societies and families with cars are typically safer and healthier for children. That’s why the conversation about children’s well-being should incorporate the car factor into efforts to reduce the harms too many children continue to experience.
Another interpretation
Both the marriage story and the car story are misleading data manipulations that substitute data volume for analytical power and present results in a way intended to pitch a conclusion rather than tell the truth.
When is a non-causal story “certainly suggestive”? When the person giving you the pitch wants you to believe the conclusion.
Please do not conclude from this that all data stories are equally corrupt, and everyone just picks the version that agrees with their preconception. Not all academics lie or distort their findings to fit their personal, political, or scientific conclusions. I may be more motivated to criticize Brad Wilcox because I disagree with his conclusions (and there may be people I agree with who use bad methods that I haven’t debunked), but that doesn’t mean I’m dishonest in my interpretation and presentation of evidence. Like a real climate scientist debunking climate-change deniers, I am happy that discrediting him is both morally good and scientifically correct (and I think that’s not a coincidence).
There are two main problems with both the cars story and the marriage story. First is selection into the independent variable condition (marriage and car ownership). People end up in these conditions partly because of their values on the dependent variable. For example, women in marriages are less likely to be raped on average because women don’t want to marry men who have raped them, or likely will rape them — the absence of rape causes marriage. In the case of children with disabilities, there is evidence that children’s disabilities increase the odds their parents will divorce (which means at least one of the parents isn’t in the household and so can’t be a car-commuting parent in the ACS data).
The other main problem is omitted variables. Other things cause both family violence and children’s health, and these are not adequately controlled even if researchers tell you they control for them. Controlling for household income (and other easily-measured demographics) does not capture all the benefits and privileges that married (or car-owning) people have and transfer to their children. For tricky questions of selection and omitted variables, we need to get closer to experimental conditions in order to provide causal explanations.
Tips for critical reading
So, based on Wilcox’s car story and my car story, here are practical tips to help you avoid getting hoodwinked by a propagandist with a PhD — or a data journalist looking at a mountain of data and a tight deadline. These are some things to watch out for:
Scatter plot proof
Impressive bivariate relationships; they may be presented with mention of control variables but no mention of adjusted effect size. That’s what I did with my scatter plots above. If you have adjusted results but don’t show them, it’s selling a small net effect with a big unadjusted label. (Wilcox examples here; Mark Regnerus does this, too.)
Axis truncation
A classic example is the Obama food stamp meme, but Wilcox had a great example a few years ago when he wanted to show the drop in divorce that resulted from hard times pulling families together during the recession. If you assume divorce is always going up (it fell for decades), this looks like a dramatic change (he called it “the first annual dip since 2005”):
No head-to-head comparison of alternative explanations
This is a lot to ask, but real social scientists take seriously the alternative explanations for what they observe, and try to devise ways to test them against each other. Editors often see this as a low-hanging fruit for removal, because cutting it both shortens the piece and strengthens the argument. In the rape versus marriage story, Wilcox nodded to the alternative explanation that “women in healthy, safe relationships are more likely to select into marriage” — which he called “part of the story” — but he offered nothing to help a reader or editor adjudicate the relative size of that “part” of the story. This connects to the next red flag.
Greater than zero proof
Sometimes just showing that something exists at all is offered as evidence of its importance. That’s why I included three anecdotes about children being saved by private passenger cars — it happened, it’s real. The trick is to identify whether something matters in addition to existing. Here’s a Wilcox example where he showed that a tiny number of people said they didn’t divorce because of the recession; here’s an example in which Nate Cohn at the NYTimes Upshot said that 2% of Hispanics changing their race to White was “evidence consistent with the theory that Hispanics may assimilate as white Americans.” Neither of these provide any comparison to show how important these discoveries were relative to anything else — other reasons people delay divorce? other reasons for race-code changes? — they just exist. This is reasonable if you’re discovering a new subatomic particle, but with social behavior it’s less impressive.
Piles of studies
The reason I presented the car results as the three separate “studies” was to make the point that you can have a lot of studies, but if none of them prove your point it doesn’t matter. For example, in his post Wilcox linked to a series of publications about how children whose parents weren’t married were more likely to be sexually abused, but none of them handle the problem of selection into marriage I described above. Similarly, a generation of research showed that women who have babies as teenagers suffer negative economic consequences, but those effects were all exaggerated because people didn’t take selection into account (women with poor economic prospects are more likely to have babies as teenagers).
Describing one side of inequality as a social good
Let’s say that, in street fights, the person with a gun beats the person with a knife more than 50% of the time. Do we conclude people should have more guns? Some benefits are absolute and have no zero-sum quality to them. (I can’t think of any, but I assume there are some.) Normally, however, we’re talking about relative benefits. The benefits of marriage, or the economic benefits of education, are measured relative to people who aren’t married or schooled.
The typical description of such a pattern is, “This causes a good outcome, we should have more of it.” But we should always consider whether the best thing, socially, might be to reduce the benefit — that is, solve the problems of the people who don’t have the asset in question — rather than try to increase the number of people with the asset.
The benefit of cars that comes from being able to get to the hospital quicker may only be relative to the poor suckers stuck in an ambulance while your personal cars are blocking up Manhattan.
Ambulance stuck in Manhattan, by Philip Cohen
Appendix: Regression results
