Appearance on Fox News Channel explained

Recently I was invited to be interviewed by Fox News Channel for “a series of stories about changing demographics and how they’re impacting politics, policy, and our culture.” Specifically, the producer said they wanted to interview me about “your recent research on millennials and marriage and divorce rates.”

This raised the recurring question faced by responsible academics: Should I appear on Fox News? For those of us who love attention, it’s hard to say no, but I did consider saying no. I figured the segment would be a right-slanted take, but also hoped that since it was for a news program, rather than an opinion program, it might be moored to reality, and I thought I might have a chance to interject something useful, or at least true. (This differs from my previous appearance, with Tucker Carlson.) Whether to differentiate at all between news and opinion on FNC is an interesting question in itself.

So I did it, and it aired yesterday. Since I lent it legitimacy I should also correct the errors they made. Comments below the video:

Here are some comments and corrections. First, the beginning is just a fear-of-change narrative:

“As we head into 2019 you may look back and think about how much has changed, not just in the past year, but in your life. And it’s not just you. America’s population, our culture, it is all changing.”

It’s setting viewers up for doom, where change is ominous out of control, the audience tearing down that precedes the build up of the authoritarian leader. Anyway, that’s to be expected, along with the boilerplate right-wing statements about marriage, women, welfare, and single mothers, which I won’t detail here.

They never did ask me about my research on marriage and divorce, but we did talk about fertility. So then he says:

“The US is facing a demographic crisis that JFK could not have imagined: A fertility rate of 1.8 percent. That means the US is not producing enough to sustain its population.”

Don’t ask what JFK has to do with this. But the fertility rate is not “1.8 percent,” it’s 1.8 projected births per woman, and it’s not a demographic crisis.

In the interview, I tried to focus on inequality and insecurity in every answer, figuring that was the angle they might let into the piece. This is what they ended up using:

“The reasons behind these demographic changes are complicated. [Philip Cohen:] One of the reasons people have fewer children is because they’re unsure about the future. They’re unsure about the costs of raising those children, especially the costs of education. And the student loan debt is a huge crisis that everybody knows about.”

I’m happy with this, a true statement, not distorted or taken out of context. The chyron they put below me is bad, however: “Lower U.S. Fertility Rates Creating Society Upheaval.” “Upheaval” is a strong word, but in any event the causality is reversed: social instability is driving lower U.S. fertility rates. Whatever effects falling fertility will have on society, they’re not here yet anyway.

Then immigration:

“The US is compensating for lower fertility rates with another demographic change: an increased reliance on immigration.”

The US doesn’t exactly have a policy of responding to falling fertility by welcoming immigrants. But it’s true that immigration is buttressing the US from the potential effects of slower population growth. In the last 25 years the immigrant share of the labor force has increased from 12 percent to 19 percent. That is pretty clearly the solution — if we need one — to falling population growth. But this quote from Victor Davis Hanson, Hoover Institution is ridiculous:

“In the case of the right, they want people to work more cheaply than native-born citizens. And on the left they want a further argument, or an agenda for big government.”

It’s true the right wants immigrants to help keep labor costs down. The idea that the left wants immigrants to bolster the argument for big government is just idiotic. This is creating a narrative where the system/swamp/Washington is destroying the culture.

Finally, the conclusion brings it back to fear of change:

“These demographic changes help to partly explain the resurgence of socialism in the United States. A Gallup poll from August found that young adult Americans are more positive about socialism – 51 percent – than they are about capitalism – 45 percent. That’s a 12-point swing in only two years.”

I have no idea how you connect “these demographic changes” to the (excellent) rise in positive perceptions about socialism. But the 12-point change in two years was only in young adults’ (age 18-29) attitudes toward capitalism. During that time their attitude toward socialism declined as well, so the gap went from -2 to +6, or an eight-point swing. Here’s the trend from Gallup:

In conclusion, I got to say something I wanted to say, and it added something to the piece they wouldn’t otherwise have included. Whether that makes it worth participating in this I can’t say.

The segment above was the first of three. I discuss the other two here.

Equal-education and wife-more-education married couples don’t have sex less often

In my review of Mark Regnerus’s book, Cheap Sex, I wrote: “The book is an extended rant on the theme, ‘Why buy the cow when you can get the milk for free?’ wrapped in a misogynist theory about sexual exchange masquerading as economics, and motivated by the author’s misogynist religious and political views.”

Someone just reposted an old book-rehash essay of Regnerus’s called, “The Death of Eros.” In it he links to my post documenting the decline in sexual frequency among married couples in the General Social Survey. In marriage, Regnerus writes, “equality is the enemy of eros,” before selectively characterizing some research about the relationship between housework and sex. (Here’s a recent analysis finding egalitarian couples don’t have sex less.)

But I realized I never looked at sexual frequency in married couples by the relative education of the spouses, which is available in the GSS. So here’s a quick take: Married man-woman couples in which the wife has equal or more education don’t have sex less frequently.

I modeled sexual frequency (an interval scale from “not at all” = 0 to “4+ times per week” = 6 as a function of age, age-squared, respondent education, respondent sex, decade, and relative education (wife has lower degree, wife has same degree, wife has higher degree). The result is in this figure. Note the means are between 3 (“2-3 times per month”) and 4 (“weekly”). Stata code for GSS below.

OK, that’s it. Here’s the code (I prettied the figure a little by hand afterwards):

*keep married people
keep if marital==1

* with non-missing own and spouse education
keep if spdeg<4 & degree<4
recode age (18/29=18) (30/39=30) (40/49=40) (50/59=50) (60/109=60), gen(agecat)
recode year (1970/1979=1970) (1980/1989=1980) (1990/1999=1990) (2000/2008=2000) (2010/2016=2010), gen(decade)
gen erosdead = spdeg>degree
gen equal=spdeg==degree

gen eros=0
replace eros=1 if spdeg<degree & sex==1
replace eros=2 if spdeg==degree
replace eros=3 if spdeg>degree & sex==1

replace eros=1 if spdeg>degree & sex==2
replace eros=3 if spdeg<degree & sex==2

label define de 1 "wife less"
label define de 2 "equal", add
label define de 3 "wife more", add
label values eros de

reg sexfreq i.sex i.agecat i.decade i.degree i.eros [weight=wtssall]
reg sexfreq i.sex c.age##c.age i.degree i.eros##i.decade [weight=wtssall]
margins i.eros##i.decade
marginsplot, recast(bar) by(decade)

Note: On 25 Dec 2018 I fixed a coding error and replaced the figure; the results are the same.

Wives’ share of couple income update

This is an update of previous reports with some new analysis at the end.

In my book Enduring Bonds I showed the distribution of income within different-sex married couples from 1970 to 2014. Here is the updated trend to 2017:

The change from 2014 is a modest continuation. Here’s the detail from 2017, with the couples reporting exactly-even incomes broken out in the middle:

In 2017, for different-sex couples with wife age 18-64:

• 26% of wives earn more than their husbands (up from 15% in 1990 and 7% in 1970).
• The average wife-who-earns-more takes home 69% of the couple’s earnings. The average for higher-earning husbands is 79%.
• It is 8.3-times more common for a husband to earn all the money than a wife (18.7% versus 2.3%).

In the book I offer the following summary:

Actually, this triplet pattern fits a lot of trends regarding gender inequality: yes, lots of change, but most of it decades ago, and not quite as fundamental as it looks.

New breakdown

At the request of Stephanie Coontz, I ran the 2017 numbers by income bracket (and including all ages).  I broke the couples into the bottom 10% (under $27,000), the 10-25th percentile (to $47,000), the 25th-5th (to $80,000), the 50th-75th (to $130,000), the 75th-90th (to $202,000), and the top 10% ($202,000+). Here is the income distribution within couples for each income bracket, with a few points labeled for clarity:

A key point here is that although wives rarely earn the dominant share of income, most couples rely on the wife’s income to maintain their standard of living. For example, a couple at the median, $80,000, would have to drastically alter their lifestyle without the 40-49% share contributed by the wife’s income. Breadwinning in its 1950s connotation is is distracting from this contemporary reality, and we should probably drop the term.

Decadally-biased marriage recall in the American Community Survey

Do people forget when they got married?

In demography, there is a well-known phenomenon known as age-heaping, in which people round off their ages, or misremember them, and report them as numbers ending in 0 or 5. We have a measure, known as Whipple’s index, that estimates the extent to which this is occurring in a given dataset. To calculate this you take the number of people between ages 23 and 62 (inclusive), and compare it to five-times the number of those whose ages end in 0 or 5 (25, 30 … 60), so there are five-times as many total years as 0 and 5 years.

If the ratio of 0/5s to the total is less than 105, that’s “highly accurate” by the United Nations standard, a ratio 105 to 110 is “fairly accurate,” and in the range 110 to 125 age data should be considered “approximate.”

I previously showed that the American Community Survey’s (ACS) public use file has a Whipple index of 104, which is not so good for a major government survey in a rich country. The heaping in ACS apparently came from people who didn’t respond to email or mail questionnaires and had to be interviewed by Census Bureau staff by phone or in person. I’m not sure what you can do about that.

What about marriage?

The ACS has a great data on marriage and marital events, which I have used to analyze divorce trends, among other things. Key to the analysis of divorce patterns is the question, “When was this person last married?” (YRMARR) Recorded as a year date, this allows the analyst to take into account the duration of marriage preceding divorce or widowhood, the birth of children, and so on. It’s very important and useful information.

Unfortunately, it may also have an accuracy problem.

I used the ACS public use files made available by IPUMS.org, combining all years 2008-2017, the years they have included the variable YRMARR. The figure shows the number of people reported to have last married in each year from 1936 to 2015. The decadal years are highlighted in black. (The dropoff at the end is because I included surveys earlier than those years.)

Yikes! That looks like some decadal marriage year heaping. Note I didn’t highlight the years ending in 5, because those didn’t seem to be heaped upon.

To describe this phenomenon, I hereby invent the Decadally-Biased Marriage Recall index, or DBMR. This is 10-times the number of people married in years ending in 0, divided by the number of people married in all years (starting with a 6-year and ending with a 5-year). The ratio is multiplied by 100 to make it comparable to the Whipple index.

The DBMR for this figure (years 1936-2015) is 110.8. So there are 1.108-times as many people in those decadal years as you would expect from a continuous year function.

Maybe people really do get married more in decadal years. I was surprised to see a large heap at 2000, which is very recent so you might think there was good recall for those weddings. Maybe people got married that year because of the millennium hoopla. When you end the series at 1995, however, the DBMR is still 110.6. So maybe some people who would have gotten married at the end of 1999 waited till New Years day or something, or rushed to marry on New Year’s Eve 2000, but that’s not the issue.

Maybe this has to do with who is answering the survey. Do you know what year your parents got married? If you answered the survey for your household, and someone else lives with you, you might round off. This is worth pursuing. I restricted the sample to just those who were householders (the person in whose name the home is owned or rented), and still got a DBMR of 110.7. But that might not be the best test.

Another possibility is that people who started living together before they were married — which is most Americans these days — don’t answer YRMARR with their legal marriage date, but some rounded-off cohabitation date. I don’t know how to test that.

Anyway, something to think about.

Breaking Millennial divorce drop news explained

[With updates as new stories come in.]

---

Millennials are fun to disparage.

Phones and selfies are all that they cherish.

And what’s par for the course, they have ruined divorce.

‘Cuz Millennials hang on to their ______.

—Wait Wait Don’t Tell Me, 9/29/18

The divorce paper I posted two weeks ago, “The Coming Divorce Decline,” suddenly took off in the media the other day (blog post | paper | data and code). I’ve now written an op-ed about the findings for The Hill, including this:

I am ambivalent about these trends. Divorce is often painful and difficult, and most people want to avoid it. The vast majority of Americans aspire to a lifelong marriage (or equivalent relationship). So even if it’s a falling slice of the population, I’m not complaining that they’re happy. Still, in an increasingly unequal society and a winner-take-all economy, two-degree couples with lasting marriages may be a buffer for the select few, but they aren’t a solution to our wider problems.

Here’s my media scrapbook, with some comment about open science process at the end.

The story was first reported by Ben Steverman at Bloomberg, who took the time to read the paper, interview me at some length, send the paper to Susan Brown (a key expert on divorce trends) for comment, and produce figures from the data I provided. I was glad that his conclusion focused on the inequality angle from my interpretation:

“One of the reasons for the decline is that the married population is getting older and more highly educated,” Cohen said. Fewer people are getting married, and those who do are the sort of people who are least likely to get divorced, he said. “Marriage is more and more an achievement of status, rather than something that people do regardless of how they’re doing.”

Many poorer and less educated Americans are opting not to get married at all. They’re living together, and often raising kids together, but deciding not to tie the knot. And studies have shown these cohabiting relationships are less stable than they used to be.

Fewer divorces, therefore, aren’t only bad news for matrimonial lawyers but a sign of America’s widening chasm of inequality. Marriage is becoming a more durable, but far more exclusive, institution.

The Bloomberg headline was, “Millennials Are Causing the U.S. Divorce Rate to Plummet.” Which proved irresistible on social media. I didn’t use the terms “millennials” (which I oppose), or “plummet,” but they don’t fundamentally misrepresent the findings.

Naturally, though, the Bloomberg headline led to other people misrepresenting the paper, like Buzzfeed, which wrote, “Well, according to a new study, millennials are now also ‘killing’ divorce.” Neither I nor Bloomberg said anyone was “killing” divorce; that was just a Twitter joke someone made, but Buzzfeed was too metameta to pick up on that. On the other hand, never complain about a Buzzfeed link, and they did link to the paper itself (generating about 800 clicks in a few days).

Then Fox 5 in New York did a Skype interview with me, and hit the bar scene to talk over the results (additional footage courtesy of my daughter, because nowadays you provide your own b-roll):

The next day Today did the story, with additional information and reporting from Bowling Green’s National Center for Family and Marriage Research, and Pew.

The Maryland news office saw the buzz and did their own story, which helped push it out.

An article in Atlantic featured an interview with Andrew Cherlin putting the trends in historical context. Rachelle Hampton in Slate tied the divorce trend to a Brookings report showing marriage is increasingly tied to higher education. On KPCC, AirTalk hosted a discussion with Megan Sweeney and Steven Martin. On Wisconsin Public Radio, Stephanie Coontz widened the discussion to put changes in marriage and divorce in historical perspective.

Rush Limbaugh read from the Bloomberg article, and was just outraged: “Now, who but deranged people would look at it this way?”

How anybody thinks like this… You have to work to be this illogical. I don’t know where this kind of thing comes from, that a plummeting divorce rate is a bad sign for America in the left’s crazy world of inequality and social justice and their quest to make everybody the same. So that’s just an example of the… Folks, that is not… That kind of analysis — and this is a sociology professor at the University of Maryland. This is not stable. That kind of thinking is not… It’s just not normal. Yet there it is, and it’s out there, and it’s be widely reported by the Drive-By Media, probably applauded and supported by others. So where is this coming from? Where is all of this indecency coming from? Why? Why is it so taking over the American left?

The Limbaugh statement might have been behind this voicemail I received from someone who thinks I’m trying to “promote chaos” to “upend the social order”:

I had a much more reasonable discussion about marriage, divorce, and inequality in this interview with Lauren Gilger in KJZZ (Phoenix public radio).

The Chicago Tribune editorial board used the news to urge parents not to rush their children toward marriage:

This waiting trend may disturb older folks who followed a more traditional (rockier?) path and may be secretly, or not so secretly, wondering if there’s something wrong with their progeny. There isn’t. Remember: Unlike previous generations, many younger people have a ready supply of candidates at their fingertips in the era of Tinder and other dating apps. They can just keep swiping right. Our advice for parents impatient to marry off a son or daughter? Relax. The older they get, the less likely you’ll be stuck paying for the wedding.

The Catholic News Agency got an expert to chime in, “If only we could convince maybe more of them to enter into marriage, we’d be doing really well.”

I don’t know how TV or local news work, but somehow this is on a lot of TV stations. Here’s a selection.

Fox Business Network did a pretty thorough job.

Some local stations added their own reporting, like this one in Las Vegas:

And this one in Buffalo:

And this one in Boise, which brought in a therapist who says young people aren’t waiting as long to start couples therapy.

Jeff Waldorf on TYT Nation did an extended commentary, blaming capitalism:

---

Open science process

Two things about my process here might concern some people.

The first is promoting research that hasn’t been peer reviewed. USA Today was the only report I saw that specifically mentioned the study is not peer reviewed:

The study, which has not been published in a peer-reviewed journal, has been submitted for presentation at the 2019 Population Association of America meeting, an annual conference for demographers and sociologists to present research.

But, when Steverman interviewed me I emphasized to him that it was not peer-reviewed and urged him to consult other researchers before doing the story — he told me he had already sent it to Susan Brown. Having a good reporter consult a top expert who’s read the paper is as good a quality peer review as you often get. I don’t know everything Brown told him, but the quote he used apparently showed her endorsement of the main findings:

“The change among young people is particularly striking,” Susan Brown, a sociology professor at Bowling Green State University, said of Cohen’s results. “The characteristics of young married couples today signal a sustained decline [in divorce rates] in the coming years.”

For the story to be clear enough to become a news event, the research often has to be pretty simple. That’s the case here: what I’m doing is looking at an easily-identified trend and providing my interpretation of it. If this has to be peer-reviewed, then almost anything an academic says should be. Of course, I provided the publicly verifiable data and code, and there are a lot of people with the skills to check this if it concerned them.

On the other hand, there is a lot of research that is impossible to verify that gets reported. Prominent examples include the Alice Goffman ethnographic book and the Raj Chetty et al. analysis of confidential IRS data. These were big news events, but whether they were peer reviewed or not was irrelevant because the peer reviewers had no way to know if the studies were right. My conclusion is that sharing research is the right thing to do, and sharing it with as much supporting material as you can is the responsible way to do it.

The second concern is over the fact that I posted it while it was being considered for inclusion in the Population Association of America meetings. This is similar to posting a paper that is under review at a journal. Conference papers are not reviewed blind, however, so it’s not a problem of disclosing my identity, but maybe generating public pressure on the conference organizers to accept the paper. This happens in many forms with all kinds of open science. I think we need to see hiding research as a very costly choice, one that needs to be carefully justified — rather than the reverse. Putting this in the open is the best way to approach accountability. Now the work of the conference organizers, whose names are listed in the call for papers, can be judged fairly. And my behavior toward the organizers if they reject it can also be scrutinized and criticized.

Although I would love to have the paper in the conference, in this case I don’t need this paper to be accepted by PAA, as it has already gotten way more attention than I ever expected. PAA organizers have a tough job and often have to reject a lot of papers for reasons of thematic fit as well as quality. I won’t complain or hold any grudges if it gets rejected. There’s a lot of really good demography out there, and this paper is pretty rudimentary.

Visualizing family modernization, 1900-2016

After this post about small multiple graphs, and partly inspired by two news reports I was interviewed for — this Salt Lake Tribune story about teen marriage, and this New York Times report mapping age at first birth — I made some historical data figures.

These visualizations use decennial census data from 1900 to 1990, and then American Community Survey data for 2001, 2010, and 2016; all data from IPUMS.org. (I didn’t use the 2000 Census because marital status is messed up in that data, with a lot of people who should be never married coded as married, spouse absent; 2001 ACS gets it done.)

An important, simple way of illustrating the myth-making around the 1950s is with marriage age. Contrary to the myth that the 1950s was “traditional,” a long data series show the period to be unique. The two trends here, teen marriage and divorce, both show the modernization of family life, with increasing individual self-determination and less restricted family choices for women.

First, I show the proportion of teenage women married in each state, for each decade from 1900 to 2016. The measure I used for this is the proportion of 19- and 20-year-olds who have ever been married (that is, including those married, divorced, and widowed). It’s impossible to tell exactly how many people were married before their 20th birthday, which would be a technical definition of teen marriage, but the average of 19 and 20 should do it, since it includes some people are on the first day of their 19th year, and some people are on the last day of their 20th, for an average close to exact age 20.

I start with a small multiple graph of the trend on this measure in every state (click all figures to enlarge). Here the states are ordered by the level of teen marriage in 2016, from Maine lowest (<1%) to Utah (14%):

This is useful for seeing that the basic pattern is universal: starting the century lower and rising to a peak in 1960, then declining steeply to the present. But that similarity, and smaller range in the latest data, make it hard to see the large relative differences across states now. Here are the 2016 levels, showing those disparities clearly:

Neither the small multiples nor the bars help you see the regional patterns and variations. So here’s an animated map that shows both the scale of change and the pattern of variation.

This makes clear the stark South/non-South divide, and how the Northeast led the decline in early marriage. Also, you can see that Utah, which is such a standout now, did not have historically high teen marriage levels, the state just hasn’t matched the decline seen nationally. Their premodernism emerged only in relief.

Divorce

Here I again used a prevalence measure. This is just the number of people whose marital status is divorced, divided by the number of married people (including separated and divorced). It’s a little better than just the percentage divorced in the population, because it’s at least scaled by marriage prevalence. But it doesn’t count divorces happening, and it doesn’t count people who divorced and then remarried (so it will under-represent divorce to the extent that people remarry). Also, if divorced people die younger than married people, it could be messed up at older ages. Anyway, it’s the best thing I could think of for divorce rates by state all the way back to 1900.

So, here’s the small multiple graph, showing the trend in divorce prevalence for all states from 1900 to 2016:

That looks like impressive uniformity: gradual increase until 1970, then a steep upward turn to the present. These are again ordered by the 2016 value, from Utah at less than 20% to New Mexico at more than 30% — smaller variation than we saw in teen marriage. That steep increase looks dramatic in the animated map, which also reveals the regional patterns:

Technique

The strategy for both trends is to download microdata samples from all years, then collapse the files down to state averages by decade. The linear figures are Stata scatter plots by state. The animated maps use maptile in Stata (by Michael Stepner) to make separate image files for each map, which I then imported into Photoshop to make the animations (following this tutorial).

The downloaded data, codebooks, Stata code, and images, are all available in an Open Science Framework project here. Feel free to adapt and use. Happy to hear suggestions and alternative techniques in the comments.

More marriage promotion failure evidence

Photo PNC / Flickr CC: https://flic.kr/p/9xNLad

I have another entry in the Cato Unbound forum on the Success Sequence. This is a response to a post by Brad Wilcox, which responded to my initial post, “The failure of the success sequence.”

With that context, I reprint it here.

---

In the second round of comments here, Brad Wilcox chose to focus on my argument that marriage promotion doesn’t work—that is, it doesn’t lead to more marriages. I have two brief responses to his comments.

First, Wilcox asserts that I have ignored salient evidence, and he mentions two studies. He writes:

But Cohen did not do justice to the existing literature on the HMI [Healthy Marriage Initiative] or of interventions like those used within it. For instance, he ignores evidence of modest success for the Oklahoma Marriage Initiative in fostering family stability (the longest running local effort working on this issue) and research that found that spending on the HMI was “positively associated with small changes in the percentage of married adults in the population” (italics in the original).

However, in my essay I linked to my book, Enduring Bonds: Inequality, Marriage, Parenting, and Everything Else That Makes Families Great and Terrible. There I dealt with the subject in much greater depth.

In particular, with regard to the claim that marriage promotion was associated with more marriage, the link is to this study (paywalled) in the journal Family Relations, by Alan Hawkins, Paul Amato, and Andrea Kinghorn. In my book I devote more than two pages to debunking this single study in detail. Since Wilcox appears not inclined to read my analysis in the book, I provide some key excerpts here:

[Hawkins, Amato, and Kinghorn] attempted to show that the marriage promotion money had beneficial effects at the population level.

They statistically compared state marriage promotion funding levels to the percentage of the population that was married and divorced, the number of children living with two parents or one parent, the nonmarital birth rate, and the poverty and near-poverty rates for the years 2000–2010. This kind of study offers an almost endless supply of subjective, post hoc decisions for researchers to make in their search for some relationship that passes the official cutoff for “statistical significance.” Here’s an example of one such choice these researchers made to find beneficial effects (no easy task, apparently): arbitrarily dividing the years covered into two separate periods. Here is their rationale: “We hypothesized that any HMI effects were weaker (or nonexistent) early in the decade (when funding levels were uniformly low) and stronger in the second half of the decade (when funding levels were at their peak).”

This is wrong. If funding levels were low and there was no effect in the early period, and then funding levels rose and effects emerged in the later period, then the analysis for all years should show that funding had an effect; that is the point of the analysis. This decision does not pass the smell test. Having determined that this decision would help them show that marriage promotion was good, they went on to report their beneficial effects, which were “significant” if you allowed them a 90 percent confidence (rather than the customary 95 percent, which is kosher under some house rules).

However, then they admitted their effects were significant only with Washington, D.C., included. Our nonstate capital city is a handy wiggle-room device for researchers studying state-level patterns; you can justify including it because it’s a real place, or you can justify excluding it because it’s not really a state. It turns out that the District of Columbia had per capita marriage promotion funding levels about nine times the average. With an improving family well-being profile during the period under study, this single case (out of fifty-one) could have a large statistical effect on the overall pattern. Statistical outliers are like the levers you learned about in physics—the further they are from the average, the more they can move the pile. To deal with this extreme outlier, they first cut the independent variable in half for D.C., bringing it down to about 4.4 times the mean and a third higher than the next most-extreme state, Oklahoma (itself pretty extreme). That change alone cut the number of significant effects on their outcomes down from six to three.

Then, performing a tragic coup de grâce on their own paper, they removed D.C. from the analysis altogether, and nothing was left. They didn’t quite see it that way, however: “But with the District of Columbia excluded from the data, all of the results were reduced to nonsignificance. Once again, most of the regression coefficients in this final analysis were comparable to those in Table 2 in direction and magnitude, but they were rendered nonsignificant by a further increase in the size of the standard errors.”

Really. These kinds of shenanigans give social scientists a bad name. (Everything that is nonsignificant is that way because of the [relative] size of the standard errors—that’s what nonsignificant means.) And what does “comparable in direction and magnitude” mean, exactly? This is the kind of statement one hopes the peer reviewers or editors would check closely. For example, with D.C. removed, the effect of marriage promotion on two-parent families fell 44 percent, and the effect on the poor/near-poor fell 78 percent. That’s “comparable” in the sense that they can be compared, but not in the sense that they are similar. Again, the authors helpfully explain that “the lack of significance can be explained by the larger standard errors.” That’s just another way of saying their model was ridiculously dependent on D.C. being in the sample and that removing it left them with nothing.

Oh well. Anyway, please keep giving the programs money, and us money for studying them: “In sum, the evidence from a variety of studies with different approaches targeting different populations suggests a potential for positive demographic change resulting from funding of [Marriage and Relationship Education] programs, but considerable uncertainty still remains. Given this uncertainty, more research is needed to determine whether these programs are accomplishing their goals and worthy of continued support.”

In short, this paper provides no evidence that HMI funding increased marriage rates or family wellbeing.

The other link Wilcox provides (“modest success for the Oklahoma Marriage Initiative”) goes to an essay on his website by the same Alan Hawkins. The evidence about Oklahoma’s “modest success” in that essay is limited to a broken link to another page on Wilcox’s site, and—I find this hard to even believe—an estimate of the effects of HMI funding in Oklahoma extrapolated from the paper I discussed above! That is, they took the very bad models from that paper and used them to predict how much the funding should have mattered in Oklahoma based on the level of funding there (and remember, Oklahoma was an outlier in that analysis). There was no estimate of the actual effect in Oklahoma. In fact, as I explained in a followup debunking, Oklahoma during this period experienced a greaterdecline in married-parent families than the rest of the country, even as they sucked up much more than their share of marriage promotion funds. This is, to put it mildly, not good social science. (The Oklahoma program, incidentally, is the subject of an excellent book by Melanie Heath: One Marriage Under God.)

Wilcox also argues that I am too demanding of federal programs, expecting demonstrable success. He concludes, “If the United States had adapted Cohen’s standard a half century ago, this would have resulted in the elimination of scores of federally funded programs that now garner hundreds of billions of dollars every year in public spending—from job training to Head Start.”

Amazingly, because Wilcox has made this argument before, I also addressed it in my book. Specifically, I wrote:

Of course, lots of programs fail. And, specifically, some studies have failed to show that kids whose parents were offered Head Start programs do better in the long run than those whose parents were not. But Head Start is offering a service to parents who want it, a service that most of them would buy on their own if it were not offered free. Head Start might fail at lifting children out of poverty while successfully providing a valuable, need-based service to low-income families.

As you can imagine, I am all for giving free marriage counseling to poor people if they want it (along with lots of other free stuff, including healthcare and childcare). And if they like it and keep using it, I might define that program as a success. But it’s not an antipoverty program.

Finally, in response to the idea that we just need more funding and more research to know if marriage promotion works, here’s my suggestion: in the studies testing marriage promotion programs, have a third group—in addition to the program and control group—who just get the cash equivalent to the cost of the service (a few thousand dollars). Then check to see how well the group getting the cash is doing compared with those getting the service. That’s the measure of whether this kind of policy is a success.