Distorting data on divorce at the National Marriage Project

This has really gone too far.

The National Marriage Project, under the directorship of W. Bradford Wilcox, a tenured sociologist at the U. Virginia, is telling some tall tales, courtesy of a grant from the Templeton Foundation‘s “Foundations of Marital Generosity Project.”

I’m sorry I never got around to writing this, even though the report in question came out months ago, and was covered uncritically all over the place. I thought I had resolved the issue with a definitive blog post on his shenanigans in 2009, but apparently he’s still at it.

Sliver linings

To show the serious financial nature of the recession for families (and how men are better at math), the report has this clip art on the cover:

In Business Week, the story was headlined, “Recession Strengthens Some Marriages”:

“In the face of a major trauma, in this case financial, some people are hurt by it in ways that have a long-lasting effect,” said Bradford Wilcox, director of the university’s National Marriage Project. “Other people are more resilient and grow stronger. I think that’s what is happening here with marriage.”

That is what Wilcox calls the “silver lining” of the recession — actually, two silver linings.

Wilcox spent Templeton’s money on a survey of 1,197 people — which he asserts (but does not show) was “nationally representative.” Bullet point #1 is: “Many couples report that the recession has deepened their commitment to marriage.” That is, (a) 29% of married people under age 46 agreed that, “the recession has deepened my commitment to my marriage.”  Further, he reports, (b) half of those with the stronger commitment say they are in a “very happy marriage,” compared with just one-quarter of those who didn’t experience the positive boost. What to make of that?

Wilcox is sure that (a) is causing (b): “Those who have redoubled their marital commitment as a result of the recession are much more likely to be in a very happy marriage,” he says triumphantly. My first thought was that people in happy relationships make the most of hard times, so they say the recession was good for them: (b) is causing (a). But now I think we can’t tell anything from this poor-quality data, since both questions were asked at the same time, and both are so subjective.

Wilcox gets even better mileage from an even more dubious silver-lining bullet point #2: “Among those who were considering a divorce prior to the recession, a large minority of couples say the recession caused them to postpone or put aside divorce.” In other words, he says, the recession is causing couples to forestall their divorces.

He writes that “about five percent” of the respondents “say they were considering divorce or separation prior to the recession.” Of that number, he writes, “38 percent say that the recession has caused them to put aside divorce or separation.” Looks to me like that’s (1,197*.05)*.38 = about 22 people, or less than 2% of the sample. That’s not a lot to hang your second bullet point on (did they even ask what else might have caused people to postpone their divorces, or are we to assume the recession is the only possible factor?).

But what is the meaning of the number of people “considering divorce or separation” who decide to “put aside” their plans? Is that an indicator of a “silver lining”? Is that good news for marriage? If these couples are in the bottom 5% of the couple-happiness distribution, then isn’t it possible that divorce is a good outcome for them? Of course, we don’t know from the data. But it’s great news if the mission of your University-letterheaded outfit is, in part, “to develop strategies for strengthening marriage” – and you define “strengthening marriage” as increasing the number of marriages.

He goes on:

Though the survey cannot estimate the number of marriages that dissolved as a result of the recession, it appears that some, at least, have been saved for now. Moreover, the results of this survey are consistent with data from the 2010 State of Our Unions report, which indicated that divorce rates have fallen since the Great Recession began (emphasis added).

I hope the Templeton Foundation is glad they paid for a report that “indicated” this important information (and by “indicated” I mean “looked up online”) — and then another report to quote the first one authoritatively. As I more patiently explained 18 months ago, the divorce rate has fallen 25 out of the last 27 years — making this whole argument another typical case of deliberate time-axis truncation.

But back up to that 38% of the 5% who had been considering separation or divorce but backed off for now. Is 38% a lot? What if it had been 28%, or 18%? Would that have been “consistent with” the falling divorce rates? It appears Wilcox has applied the wily non-zero standard of proof: if anyone fits his preconception, the preconception is confirmed.

Business Week wrote:

Exactly how many marriages have been saved is unknown, but the U.S. divorce rate fell 7 percent between 2006 and 2009, said Wilcox, who is also a sociology professor [thanks for rubbing that in -pnc] at the university.

How helpful of him to provide that information. Wilcox might have been even more helpful by saying, “a 7% decline in the divorce rate over 4 years? Well, that hasn’t been seen since the years 2004, 2003, 1997, 1996 and 1995!”

Instead, Wilcox chose to manipulate the tendency of Americans to assume divorce rates are always going up, and the blind trust of under-trained journalists, snookered by a PhD and a university letterhead on the PDF report. He may as well have provided them with this infographic I’ve been kind enough to produce (using the crude divorce rate, scaled from 3.39 to 3.59):

Oops – that seems to be what he did really inflict on the Denver Post, which published this actual news story:

Colorado divorce rate plummets during recession …The divorce rate in Colorado has dropped to 4.2 percent per 1,000 people, a rate hit only once — in 2003 — in the past 21 years. In 2007, the year the recession began, the rate was 4.4 percent, according to the National Center for Health Statistics.

That’s right: Falling from 4.4 to 4.2 — a drop of 4.5%, which is exactly the average rate of change over the last decade – is a plummet by the time Wilcox gets done with a reporter. And he’s there, once again, with the silver lining: “For some people, the recession led them to become more aware of the ties that bind, how spouses, parents, in-laws and kids stick with you when times are tough.” Must be.

So?

By using private foundation money and taking his results directly to the media, the National Marriage Project bypasses the peer-review system — but they use the image of that system to bolster their prestige and authority. (I’m not so naive that I think peer review prevents distortion and fact-twisting for ideological ends. But I think it usually helps.)

I also write outside the peer-review system on this blog. My career status and the university seal on my letterhead, however, have been peer reviewed. My research was published through the peer review process, and my tenured appointments were approved after reviews by faculty with expertise in my areas of research at other universities. I can publish whatever I want, and claim myself as an expert — but unless what I write is peer reviewed, the reader runs the risk that I’m really an ophthalmologist performing heart surgery. So, read my blog posts critically, and check my sources if you’re skeptical. There, was that so hard?

Note: I don’t require a specific word count from my students, but I suggest what a successful paper usually requires. For my introductory course, I recommend a final paper of 1,250-1,750 words. Wilcox’s report, just FYI, is 1,387 words, and includes no references. (For comparison, this blog post report is also 1,387 words.) I could publish this blog as a PDF and call it a “report,” and I might even be able to get the University news service to publicize it, especially if it came with external funding. But I think that would be unethical. (But if you insist, here’s what it might look like.)

39 Comments

Filed under In the news, Me @ work

39 responses to “Distorting data on divorce at the National Marriage Project

  1. yeldah

    Brilliant.

  2. Great post!
    I fear it’s even worse than you suggest. Since all of our universities are strapped, right wing hacks like Bradley wind up being big players at their universities and in the discipline at large. Let’s face it, even at only 10% overhead, he’s mor than paying his salary. And, he’s a big conservative Christian puppy, who has considerable influence over what gets funded by those right wing foundations. Nobody gives a fuck where the money comes from, or what it does to people’s research agendas (remember those!). Indeed, Templeton now exerts considerable influence over the data collected, even by “public” surveys like the GSS. And, if their wet dream of defunding all of the social sciences comes true, right wing foundations like Templeton will be the only funders for public opinion research. They will get to tell us what Americans think about whatever they care to ask….

  3. Mine too, especially seeing your report. I think that you should send it to the press office to see what they do with it!

  4. Excellent, although I think the spoof report needs some snappy bullet points.

  5. Pingback: W. Bradford Wilcox responds « Family Inequality

  6. Pingback: Recession, divorce? « Family Inequality

  7. Pingback: Patriarchy, Recession, and Marital Hapiness « Iranianredneck’s Weblog

  8. Pingback: Performance-based System For University Research « Times of Texas

  9. Pingback: Misreading Marriage Statistics - Hit & Run : Reason Magazine

  10. Pingback: Misreading Marriage Statistics | Daily Libertarian

  11. Pingback: Weekend Favors – July 23 & 24 « accidentallyyours

  12. Pingback: Is it a “marriage problem”? « Family Inequality

  13. Pingback: NPR reports on a report « Family Inequality

  14. Pingback: Is fertility ready to rebound? « Family Inequality

  15. Pingback: This cohabitation-causes-bad-parenting thing « Family Inequality

  16. Pingback: A different media-statistic-trend divorce story « Family Inequality

  17. Pingback: Cohabitation as an inertia problem « Family Inequality

  18. Pingback: Google searches foretold Census report of divorce increase? « Family Inequality

  19. Pingback: Crying out for more babies « Family Inequality

  20. Pingback: Health Insurance Coverage in the US; Fertility Rates in the World; And More Stories « Welcome to the Doctor's Office

  21. Pingback: Single parents, crime and incarceration « Family Inequality

  22. Pingback: Church saves marriage, and produces curious coefficients « Family Inequality

  23. Pingback: National Marriage Project Does S***ty Research Again; People Always Thought Students Couldn’t Write; and More « Welcome to the Doctor's Office

  24. Pingback: Family Inequality’s 2011 top 10 « Family Inequality

  25. Pingback: Charles Murray on his propaganda playing field « Family Inequality

  26. Pingback: Regnerus Scandal: Prominent Sociologist Delivers Devastating Professional Evaluation | The New Civil Rights Movement

  27. Pingback: Debate debate on single mothers and crime « Family Inequality

  28. Pingback: Conservatives doubling down on single mothers and crime « Family Inequality

  29. Ned Flaherty

    Philip Cohen, thanks so much for putting Mr. Wilcox in proper perspective.

    If you’re up for it, his latest annual report came out this morning. For your sake, I’ll leave just this one criticism:

    The subtitle is gratuitously deceptive. This report is not “the President’s Marriage Agenda” because the President of the United States didn’t write it, never saw it, and didn’t approve using his office as an advert in the title on the front cover. Belatedly inviting him on PDF page 13 to subscribe does not justify this product-placement marketing ploy.

  30. Pingback: Sadly, No! » Today in Non-Sequitur

  31. Pingback: Teaching family inequality: Some posts by subject | Family Inequality

  32. Pingback: ‘More managerial than intellectual’: How right-wing Christian money brought us the Regnerus study | Family Inequality

  33. Thank you for your work! I hope you don’t mind, I posted a link to this post on the New York Times web site in reply to a Ross Douthat post.
    http://douthat.blogs.nytimes.com/2013/03/27/what-was-the-familys-least-bad-decade/?comments%23permid=13&smid=tw-share
    Best regards,
    StephenKMackSD

  34. Pingback: If the National Marriage Project told you it was going to rain, would you bring an umbrella? | Family Inequality

  35. Pingback: Fatherhood’s transformations: What if someone checked the facts? | Family Inequality

  36. Welcome to Think Tank America – where a team of “scientists” with a declared goal collect “data” and publish “articles” to support their goals which the struggling media then publish as “facts”.

    (I saw statistics suggesting that more and more people get their information from non-journalistic sources – how about an article on that?)

  37. Pingback: Divorce drop and rebound: paper in the news | Family Inequality

  38. Pingback: Deciphering a well-told data story, cars are good for kids edition | Family Inequality

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s