While we’re making up overwrought terms.
W. Bradford Wilcox and Lyman Stone, both fellows who are fellows at the Institute for Family Studies (still waiting for the acknowledgment for correcting their data error), have a bad new post up called “The Happiness Recession.”
I made some annotations on their essay using the excellent Hypothesis tool. Now you can read the essay with my comments here. Click on the little arrow at the top right to expand the comments. Here are the highlights, with figures.
Note: Get a free Hypothesis account and you can annotate any page on the web, including in closed groups if you want to, and you can share your Hypothesis profile so people can see all the things you’ve annotated, like mine. You can also comment on SocArXiv papers this way.
The main problem is that there is no evidence of a happiness recession. They make their case with this figure from the General Social Survey:
To people not familiar with the General Social Survey, like, apparently, whoever edited this piece at the Atlantic, that 2018 drop in happiness might look dramatic. A lot of publications are used to mentioning margins of error when they report survey results. But maybe because WilcoxStone reminded the editor in the opening paragraph that the GSS is “a key barometer of American social life,” they didn’t bother. It seems so authoritative (and it’s a great, indispensable resource, publicly funded, and freely available for all researchers). But it’s a sample survey, and it’s not that big. Here are the actual numbers of men ages 18-34 who answered the happiness question on the survey:
It’s a little funky because of the weights I’m not showing, but in round numbers, if 8 young men had said “very happy” instead of “pretty happy” in 2018, WilcoxStone wouldn’t have been able to write this article — the percentage wouldn’t have changed. So, it’s a great survey, but you have to know what you’re doing with it, and you have to be honest about it.
There are different ways to assess significance, and no one rule, but I made this figure showing the percentage of young adults describing themselves as “very happy” in each year of the survey, with a simple p<.05 significance test for whether each year is different from 2018.
The 2018 level is the lowest point estimate, but it’s not distinguishable from about half the previous survey years at conventional levels of statistical significance. Eight pretty happy men doing all the work here.
A lot of what’s ridiculous about the post follows from this simple manipulation — pretending an insignificant change is very important. One other thing was totally wrong, though.
WilcoxStone have some incoherent theory about how friendship might play a role in the happiness recession. Maybe they were expecting to see a decline in friendship, to support their get-married-in-church-and-stay-there predetermined conclusion. Anyway, they produce this wrong figure:
I say “wrong” because, although they never define “regularly,” and neither does the GSS, men in the survey spend more evenings with their friends, and nothing seems to match these numbers. Maybe you can figure it out. Here are the distributions for young men and women in 2018 — see if you can see how they could get 64% for women and 35% for men, the last points in their figure:
In response to my tweets about this, linking to the annotated post, Stone responded that they “just use a different definition of regular,” before clarifying that they “just cut it a different way,” and finally acknowledging that “maybe there was a mislabeling in that,” and promising to look into it.
Anyway, they conclude their analysis with some counterfactual models to explain the “happiness trend.” No details are provided, and definitely nothing like standard errors or significance tests (and if the friend contact variable is screwed up, it’s useless). Here are the results:
If you look carefully, you can see that the biggest difference comes from adjusting for sexual frequency — which has declined in the sample, and sexual frequency is associated with higher happiness scores. That dotted light blue line ends at 29%. And the simulations start at 28%. So, if sexual frequency were held constant, they proclaim, happiness among young adults would be a point higher. Just kidding, they don’t give the number, they just say, “If Americans still had sex like they did in 2008, or even 2012, we might be a much happier country.” OK then.
After apparently reading my annotations, Stone was effusive on Twitter, delighted that, “And I think anyone who reads his annotations will see that he basically doesn’t refute anything we say. :)”
I guess you could say:
Anyway, see if you can figure out this conclusion:
Thus, while most of the decline in happiness [there is no decline in happiness -pnc] is about declining sex, that’s not the end of the story. Declining sex is at least partly about family and religious changes that make it harder for people to achieve stable, coupled life at a young age. If we’d like more young adults to experience the joy of sex, we will have to either revive these institutions or find new ways to kindle love in the rising generation.
Your guess is as good as mine. But if it just means go to church, get married, and then have sex, and you’ll be happy, I don’t think they needed a bologna statistical analysis to reach that.
Trust
Publications like the Atlantic put out a lot of content, much of which they get for free or low cost. A lot of that cheap material comes from academics (which Wilcox sort of is). And some comes from think tank snake oil salespeople (which Wilcox definitely is), who use their right-wing, tax-subsidized foundation money to slime the public square. It might be too much to expect that publications with little budget for writers would somehow have the ability to vet statistical analyses. So they run on trust. People like Wilcox, who has a long proven record of lying about research, and Stone, prey on the credulity of the economically precarious media. People who know better should speak up and help limit the damage.
