Distorting data on divorce at the National Marriage Project

This has really gone too far.

The National Marriage Project, under the directorship of W. Bradford Wilcox, a tenured sociologist at the U. Virginia, is telling some tall tales, courtesy of a grant from the Templeton Foundation‘s “Foundations of Marital Generosity Project.”

I’m sorry I never got around to writing this, even though the report in question came out months ago, and was covered uncritically all over the place. I thought I had resolved the issue with a definitive blog post on his shenanigans in 2009, but apparently he’s still at it.

Sliver linings

To show the serious financial nature of the recession for families (and how men are better at math), the report has this clip art on the cover:

In Business Week, the story was headlined, “Recession Strengthens Some Marriages”:

“In the face of a major trauma, in this case financial, some people are hurt by it in ways that have a long-lasting effect,” said Bradford Wilcox, director of the university’s National Marriage Project. “Other people are more resilient and grow stronger. I think that’s what is happening here with marriage.”

That is what Wilcox calls the “silver lining” of the recession — actually, two silver linings.

Wilcox spent Templeton’s money on a survey of 1,197 people — which he asserts (but does not show) was “nationally representative.” Bullet point #1 is: “Many couples report that the recession has deepened their commitment to marriage.” That is, (a) 29% of married people under age 46 agreed that, “the recession has deepened my commitment to my marriage.”  Further, he reports, (b) half of those with the stronger commitment say they are in a “very happy marriage,” compared with just one-quarter of those who didn’t experience the positive boost. What to make of that?

Wilcox is sure that (a) is causing (b): “Those who have redoubled their marital commitment as a result of the recession are much more likely to be in a very happy marriage,” he says triumphantly. My first thought was that people in happy relationships make the most of hard times, so they say the recession was good for them: (b) is causing (a). But now I think we can’t tell anything from this poor-quality data, since both questions were asked at the same time, and both are so subjective.

Wilcox gets even better mileage from an even more dubious silver-lining bullet point #2: “Among those who were considering a divorce prior to the recession, a large minority of couples say the recession caused them to postpone or put aside divorce.” In other words, he says, the recession is causing couples to forestall their divorces.

He writes that “about five percent” of the respondents “say they were considering divorce or separation prior to the recession.” Of that number, he writes, “38 percent say that the recession has caused them to put aside divorce or separation.” Looks to me like that’s (1,197*.05)*.38 = about 22 people, or less than 2% of the sample. That’s not a lot to hang your second bullet point on (did they even ask what else might have caused people to postpone their divorces, or are we to assume the recession is the only possible factor?).

But what is the meaning of the number of people “considering divorce or separation” who decide to “put aside” their plans? Is that an indicator of a “silver lining”? Is that good news for marriage? If these couples are in the bottom 5% of the couple-happiness distribution, then isn’t it possible that divorce is a good outcome for them? Of course, we don’t know from the data. But it’s great news if the mission of your University-letterheaded outfit is, in part, “to develop strategies for strengthening marriage” — and you define “strengthening marriage” as increasing the number of marriages.

He goes on:

Though the survey cannot estimate the number of marriages that dissolved as a result of the recession, it appears that some, at least, have been saved for now. Moreover, the results of this survey are consistent with data from the 2010 State of Our Unions report, which indicated that divorce rates have fallen since the Great Recession began (emphasis added).

I hope the Templeton Foundation is glad they paid for a report that “indicated” this important information (and by “indicated” I mean “looked up online”) — and then another report to quote the first one authoritatively. As I more patiently explained 18 months ago, the divorce rate has fallen 25 out of the last 27 years — making this whole argument another typical case of deliberate time-axis truncation.

But back up to that 38% of the 5% who had been considering separation or divorce but backed off for now. Is 38% a lot? What if it had been 28%, or 18%? Would that have been “consistent with” the falling divorce rates? It appears Wilcox has applied the wily non-zero standard of proof: if anyone fits his preconception, the preconception is confirmed.

Business Week wrote:

Exactly how many marriages have been saved is unknown, but the U.S. divorce rate fell 7 percent between 2006 and 2009, said Wilcox, who is also a sociology professor [thanks for rubbing that in -pnc] at the university.

How helpful of him to provide that information. Wilcox might have been even more helpful by saying, “a 7% decline in the divorce rate over 4 years? Well, that hasn’t been seen since the years 2004, 2003, 1997, 1996 and 1995!”

Instead, Wilcox chose to manipulate the tendency of Americans to assume divorce rates are always going up, and the blind trust of under-trained journalists, snookered by a PhD and a university letterhead on the PDF report. He may as well have provided them with this infographic I’ve been kind enough to produce (using the crude divorce rate, scaled from 3.39 to 3.59):

Oops – that seems to be what he did really inflict on the Denver Post, which published this actual news story:

Colorado divorce rate plummets during recession …The divorce rate in Colorado has dropped to 4.2 percent per 1,000 people, a rate hit only once — in 2003 — in the past 21 years. In 2007, the year the recession began, the rate was 4.4 percent, according to the National Center for Health Statistics.

That’s right: Falling from 4.4 to 4.2 — a drop of 4.5%, which is exactly the average rate of change over the last decade — is a plummet by the time Wilcox gets done with a reporter. And he’s there, once again, with the silver lining: “For some people, the recession led them to become more aware of the ties that bind, how spouses, parents, in-laws and kids stick with you when times are tough.” Must be.

So?

By using private foundation money and taking his results directly to the media, the National Marriage Project bypasses the peer-review system — but they use the image of that system to bolster their prestige and authority. (I’m not so naive that I think peer review prevents distortion and fact-twisting for ideological ends. But I think it usually helps.)

I also write outside the peer-review system on this blog. My career status and the university seal on my letterhead, however, have been peer reviewed. My research was published through the peer review process, and my tenured appointments were approved after reviews by faculty with expertise in my areas of research at other universities. I can publish whatever I want, and claim myself as an expert — but unless what I write is peer reviewed, the reader runs the risk that I’m really an ophthalmologist performing heart surgery. So, read my blog posts critically, and check my sources if you’re skeptical. There, was that so hard?

Note: I don’t require a specific word count from my students, but I suggest what a successful paper usually requires. For my introductory course, I recommend a final paper of 1,250-1,750 words. Wilcox’s report, just FYI, is 1,387 words, and includes no references. (For comparison, this blog post report is also 1,387 words.) I could publish this blog as a PDF and call it a “report,” and I might even be able to get the University news service to publicize it, especially if it came with external funding. But I think that would be unethical. (But if you insist, here’s what it might look like.)