Tag Archives: marriage promotion

More marriage promotion failure evidence

broken lightbulb

Photo PNC / Flickr CC: https://flic.kr/p/9xNLad

I have another entry in the Cato Unbound forum on the Success Sequence. This is a response to a post by Brad Wilcox, which responded to my initial post, “The failure of the success sequence.”

With that context, I reprint it here.


In the second round of comments here, Brad Wilcox chose to focus on my argument that marriage promotion doesn’t work—that is, it doesn’t lead to more marriages. I have two brief responses to his comments.

First, Wilcox asserts that I have ignored salient evidence, and he mentions two studies. He writes:

But Cohen did not do justice to the existing literature on the HMI [Healthy Marriage Initiative] or of interventions like those used within it. For instance, he ignores evidence of modest success for the Oklahoma Marriage Initiative in fostering family stability (the longest running local effort working on this issue) and research that found that spending on the HMI was “positively associated with small changes in the percentage of married adults in the population” (italics in the original).

However, in my essay I linked to my book, Enduring Bonds: Inequality, Marriage, Parenting, and Everything Else That Makes Families Great and Terrible. There I dealt with the subject in much greater depth.

In particular, with regard to the claim that marriage promotion was associated with more marriage, the link is to this study (paywalled) in the journal Family Relations, by Alan Hawkins, Paul Amato, and Andrea Kinghorn. In my book I devote more than two pages to debunking this single study in detail. Since Wilcox appears not inclined to read my analysis in the book, I provide some key excerpts here:

[Hawkins, Amato, and Kinghorn] attempted to show that the marriage promotion money had beneficial effects at the population level.

They statistically compared state marriage promotion funding levels to the percentage of the population that was married and divorced, the number of children living with two parents or one parent, the nonmarital birth rate, and the poverty and near-poverty rates for the years 2000–2010. This kind of study offers an almost endless supply of subjective, post hoc decisions for researchers to make in their search for some relationship that passes the official cutoff for “statistical significance.” Here’s an example of one such choice these researchers made to find beneficial effects (no easy task, apparently): arbitrarily dividing the years covered into two separate periods. Here is their rationale: “We hypothesized that any HMI effects were weaker (or nonexistent) early in the decade (when funding levels were uniformly low) and stronger in the second half of the decade (when funding levels were at their peak).”

This is wrong. If funding levels were low and there was no effect in the early period, and then funding levels rose and effects emerged in the later period, then the analysis for all years should show that funding had an effect; that is the point of the analysis. This decision does not pass the smell test. Having determined that this decision would help them show that marriage promotion was good, they went on to report their beneficial effects, which were “significant” if you allowed them a 90 percent confidence (rather than the customary 95 percent, which is kosher under some house rules).

However, then they admitted their effects were significant only with Washington, D.C., included. Our nonstate capital city is a handy wiggle-room device for researchers studying state-level patterns; you can justify including it because it’s a real place, or you can justify excluding it because it’s not really a state. It turns out that the District of Columbia had per capita marriage promotion funding levels about nine times the average. With an improving family well-being profile during the period under study, this single case (out of fifty-one) could have a large statistical effect on the overall pattern. Statistical outliers are like the levers you learned about in physics—the further they are from the average, the more they can move the pile. To deal with this extreme outlier, they first cut the independent variable in half for D.C., bringing it down to about 4.4 times the mean and a third higher than the next most-extreme state, Oklahoma (itself pretty extreme). That change alone cut the number of significant effects on their outcomes down from six to three.

Then, performing a tragic coup de grâce on their own paper, they removed D.C. from the analysis altogether, and nothing was left. They didn’t quite see it that way, however: “But with the District of Columbia excluded from the data, all of the results were reduced to nonsignificance. Once again, most of the regression coefficients in this final analysis were comparable to those in Table 2 in direction and magnitude, but they were rendered nonsignificant by a further increase in the size of the standard errors.”

Really. These kinds of shenanigans give social scientists a bad name. (Everything that is nonsignificant is that way because of the [relative] size of the standard errors—that’s what nonsignificant means.) And what does “comparable in direction and magnitude” mean, exactly? This is the kind of statement one hopes the peer reviewers or editors would check closely. For example, with D.C. removed, the effect of marriage promotion on two-parent families fell 44 percent, and the effect on the poor/near-poor fell 78 percent. That’s “comparable” in the sense that they can be compared, but not in the sense that they are similar. Again, the authors helpfully explain that “the lack of significance can be explained by the larger standard errors.” That’s just another way of saying their model was ridiculously dependent on D.C. being in the sample and that removing it left them with nothing.

Oh well. Anyway, please keep giving the programs money, and us money for studying them: “In sum, the evidence from a variety of studies with different approaches targeting different populations suggests a potential for positive demographic change resulting from funding of [Marriage and Relationship Education] programs, but considerable uncertainty still remains. Given this uncertainty, more research is needed to determine whether these programs are accomplishing their goals and worthy of continued support.”

In short, this paper provides no evidence that HMI funding increased marriage rates or family wellbeing.

The other link Wilcox provides (“modest success for the Oklahoma Marriage Initiative”) goes to an essay on his website by the same Alan Hawkins. The evidence about Oklahoma’s “modest success” in that essay is limited to a broken link to another page on Wilcox’s site, and—I find this hard to even believe—an estimate of the effects of HMI funding in Oklahoma extrapolated from the paper I discussed above! That is, they took the very bad models from that paper and used them to predict how much the funding should have mattered in Oklahoma based on the level of funding there (and remember, Oklahoma was an outlier in that analysis). There was no estimate of the actual effect in Oklahoma. In fact, as I explained in a followup debunking, Oklahoma during this period experienced a greaterdecline in married-parent families than the rest of the country, even as they sucked up much more than their share of marriage promotion funds. This is, to put it mildly, not good social science. (The Oklahoma program, incidentally, is the subject of an excellent book by Melanie Heath: One Marriage Under God.)

Wilcox also argues that I am too demanding of federal programs, expecting demonstrable success. He concludes, “If the United States had adapted Cohen’s standard a half century ago, this would have resulted in the elimination of scores of federally funded programs that now garner hundreds of billions of dollars every year in public spending—from job training to Head Start.”

Amazingly, because Wilcox has made this argument before, I also addressed it in my book. Specifically, I wrote:

Of course, lots of programs fail. And, specifically, some studies have failed to show that kids whose parents were offered Head Start programs do better in the long run than those whose parents were not. But Head Start is offering a service to parents who want it, a service that most of them would buy on their own if it were not offered free. Head Start might fail at lifting children out of poverty while successfully providing a valuable, need-based service to low-income families.

As you can imagine, I am all for giving free marriage counseling to poor people if they want it (along with lots of other free stuff, including healthcare and childcare). And if they like it and keep using it, I might define that program as a success. But it’s not an antipoverty program.

Finally, in response to the idea that we just need more funding and more research to know if marriage promotion works, here’s my suggestion: in the studies testing marriage promotion programs, have a third group—in addition to the program and control group—who just get the cash equivalent to the cost of the service (a few thousand dollars). Then check to see how well the group getting the cash is doing compared with those getting the service. That’s the measure of whether this kind of policy is a success.

2 Comments

Filed under Me @ work

The failure of the success sequence

This essay was originally published as part of a forum on the success sequence sponsored by the Cato Institute, featuring Michael Tanner, Isabel Sawhill, and Brad Wilcox.

The success sequence is often (mistakenly) attributed to the 2009 book Creating an Opportunity Society by the Brookings Institution’s Ron Haskins and Isabel Sawhill. “First comes education,” they wrote. “Then comes a stable job that pays a decent wage, made decent by the addition of wage supplements and work supports if necessary. Finally comes marriage, followed by children.” They called for “marketing campaigns and educational programs to change social norms: to bring back the success sequence as the expected path for young Americans.”

The only issue here is marriage, as the rest is obvious to everyone. And in that regard this model of social change is wholly unproven and without precedent. Seat belt laws and anti-smoking campaigns, always cited by success sequence advocates, are not comparable. Those are daily habits easily addressed by legal regulations and tax policy (seat belts are required by law; with taxes, the price of cigarettes has more than tripled since 1980). The decline in marriage is a massive global trend driven by economic development and cultural adaptation. And the decline in teen pregnancy, to which success sequencers also point as a precedent for public information campaigns, flows with rather than against that underlying trend. As I detail in my new book, Enduring Bonds: Inequality, Marriage, Parenting, and Everything Else That Makes Families Great and Terrible, the drop in teen birth was part of the general increase in the age at which women have children, driven by the expansion of their educational and professional opportunities.

That idea of using public information campaigns to preach “marriage culture” echoed the futile proclamations of a previous generation. In a Hoover Institution symposium in 1996, former vice president Dan Quayle wrote that, “when it comes to strengthening families … we also desperately need help from nongovernment institutions like the media and the entertainment community.” Taking up the call with even more zeal, in 2001 Heritage Foundation fellow Patrick Fagan declared it was time to add three W’s to the common three R’s of schooling. “We need to stress something just as fundamental [as reading, writing, and arithmetic],” he wrote. “Call it the three W’s: work, wedlock and worship. … Put all three in the lives of parents and children, and they thrive.”* Five years later, another Heritage fellow said of the three W’s, “According to the social science data, if these three fundamentals are in place, government social policy is virtually unnecessary.” In 2012, the National Marriage Project, under director W. Bradford Wilcox, was again calling for “community-based and focused public service announcements” and a Hollywood “conversation” to promote marriage.

Meanwhile, slightly more liberal think tank denizens had discretely replaced “worship” with education, but they stuck to the basic idea that the problem with poor people is that they’re doing life wrong—and the “three somethings” formula. In a 2006 report for the National Campaign to Prevent Teen Pregnancy, Barbara Dafoe Whitehead and Marline Pearson wrote that it was time to “teach teens the rules of the success sequence,” which they defined as, “Finish high school, or better still, get a college degree; wait until your twenties to marry; and have children after you marry.” (Three things is a favorite formula of Chinese social engineers we well, as with Jiang Zemin’s “Three Represents” and Hu Jintao’s “Three Supremes”—but China combines such slogans with centralized education and state repression to increase their salience.)

Today, more than two decades after Quayle’s plea, 17 years after the three W’s, 12 years after the first “success sequence” proclamation, and one president after the National Marriage Project pitched its “President’s Marriage Agenda,” movement leaders are still calling for “Public and private social marketing campaigns on behalf of marriage and the ‘success sequence’,” to quote Wilcox and Wendy Wang’s latest report. Neither the policy nor the campaign to promote the policy have changed appreciably over the years, although the definition of the success sequence has varied from author to author. And in all this time, I could not find one academic study, outside of those published by think tanks, that seriously evaluates the claims of the success sequence.

What Could Go Wrong?

Today’s success sequence movement is puzzling in part because it fails to recognize—or admit—the extent to which its adherents already won. After the landmark 1996 welfare reform act, the federal government pumped more than $1 billion into national marriage promotion programs (the Healthy Marriage and the Responsible Fatherhood initiatives). This was cause for great celebration in the movement, as it should have been. In 2004, a Heritage Foundation report gushed, “The President’s Healthy Marriage Initiative is a future-oriented, preventive policy. It will foster better life-planning skills—encouraging couples to develop loving, committed marriages before bringing children into the world.”

It didn’t. The previous decade’s marriage promotion programs sent the same message the “success sequence” promoters do today. But where is the recognition that they failed? Rigorous evaluations of the marriage promotion efforts showed unequivocally that they produced no increase in marriage, not even among the people coerced into sitting for hours in relationship skills courses required to qualify for welfare benefits. As most readers probably know, in the years after welfare reform, marriage rates have continued to fall, and they have fallen fastest for those with less than a college education, the very population the programs were supposed to help. Even though pro-marriage billboards dotted the highways and FedEx delivered thousands of new-daddy care packages to hospitals. In fact, the only people more likely to marry after all these years of conservative activism are gays and lesbians. (This history is also reviewed in my book.)

Does this mean it’s bad advice to get an education, get a job, and find a permanent partner before having children? Of course not. But the success sequence is bad public policy, which is not the same thing at all. For public policy the question is, what will we accomplish with this money, compared with other things we could spend it on (or nothing at all)? Will the proposed campaigns have any positive effect on family outcomes? And if so, would they be better than some other way of spending money, like giving it to poor people, which is what most rich countries do, along with jobs, paid family leave, health care, and preschool education? Specifically, the rationale for spending money on these campaigns assumes that there are people who are on the fence about the success sequence, whose minds might be changed by the campaign, and that those altered decisions would lead to better outcomes in the future for those specific people. There is simply no evidence to support anything like that chain of events. Despite the ad nauseam repetition of the obvious fact that educated, employed, and (much less importantly) married people are less likely to be poor, there is no evidence at all that convincing people who are not one of those things of their importance will cause a reduction in poverty rates.

Given the well-documented desire of most young adults to finish high school, get a job, and get married—if the opportunity to follow that course presents itself—there is no reason to think the people reached by the proposed campaigns would not either already plan to follow the sequence or rightly suspect that it is not feasible for them. The decision to delay childbearing in hopes of marrying first rests on assumptions about the future—education, economics, relationships, health, stability—that the target population simply cannot makeabout their own destinies in today’s economic and social context. Improve the basic equation, the material expectations of young adults, and you won’t need a campaign to change behavior.

When women have more to lose, they delay parenthood. The college students in my classes, overwhelmingly women (I teach sociology of the family), almost all want to get married and then have children after they finish college. They understand that their marriage prospects will improve after college, and they don’t want children to interfere with their education or career launch. So, why shouldn’t we tell all women, especially those with poorer education and career prospects, to follow this course as well? Success sequencers believe it’s hypocritical to hoard this advice and only dispense it to the children of privilege. But you can’t wish away education, career, and marriage uncertainty or impose order on instability by force of will. If we’re not prepared to guarantee all women the same opportunities as those in my classes have, it’s not reasonable to demand the same attachment to the success sequence that those opportunities make feasible. In the absence of that guarantee, you’re simply asking, or requiring, poor people to delay (until “they’re ready,” in Sawhill’s terms, meaning not poor) or forego having children, one of the great joys of life, and something we should consider a human right.

In addition, what signals will a federal “success sequence” program send? What message will these campaigns send to people who are currently materially underserved by the welfare state, and people who don’t have the option to pursue the sequence because stable partners, education, or jobs aren’t available to them? What message will it send to the majority of Americans who are in a position to look down upon, and act against, those who become, in Sawhill’s chilling phrase, “norm breakers”?

And here race becomes especially salient. Black women have low marriage rates and black single mothers have high poverty rates. They face marriage markets with drastic shortages of eligible men, as Michael Tanner noted in the essay that opened this discussion. Not coincidentally, the history of welfare politics in the United States is intricately bound up with the history of racism against black women, who have been labeled pathological and congenitally dependent. The idea that delaying parenthood until marriage is a choice one makes is highly salient and prized by the white middle class, and the fact that black women often don’t have that choice makes them the objects of scorn for their perceived lax morals. The framing of the success sequence plays into this dynamic. For example, Ron Haskins has argued that welfare reform was needed to “[change] the values and the approach to life of people on welfare that they have to do their part.” The image of the poor welfare “taker” has a race and a gender in America.

In their book, Haskins and Sawhill proudly acknowledge that their cause was out of step with contemporary society. “To those who argue that this goal is old-fashioned or inconsistent with modern culture,” they wrote, “we argue that modern culture is inconsistent with the needs of children.” That may by a reasonable ideological position, but it’s no way to make public policy. The success sequence is a political meme repeated in highly similar form over more than a generation of public policy debates, without yet having any discernible impact for the better. The third “step” or “norm” in particular—marriage—has already been promoted with massive federal subsidies for almost two decades. The first two, education and jobs, are terrific ideas, obvious for good reasons, and not in need of much normative boosting, and we should turn our attention to improving the opportunity for more people to attain them.

* Thanks to Shawn Fremstad for this nugget.

6 Comments

Filed under Politics

Who are you gonna marry? That one big assumption marriage promotion gets totally wrong

First preamble, then new analysis.

One critique of the marriage promotion movement is that it ignores the problem of available spouses, especially for Black women. Joanna Pepin and I addressed this with an analysis of marriage markets in this paper. White women ages 20-45, who are more than twice as likely to marry as Black women, live in metro areas with an average of 118 unmarried White men per 100 unmarried White women. Black women, on the other hand, face markets with only 78 single men per 100 single women. This is one reason for the difference in marriage rates; given very low rates of intermarriage, especially for Black women, some women essentially can’t marry.

But surely some people are still passing up potential marriages, or so the marriage promoters would have us believe, and in so doing they undermine their own futures and those of their children. Even if you can get past the sex ratio problem, you still have the issue of the benefits of marriage. Of course married people, and their kids, are better off on average. (There are great methodological lessons to be learned from their big lie use of this fact.) But who gets those benefits? The intellectual water-carriers of the movement, principally Brad Wilcox and his co-authors, always describe the benefits of increasing marriage as if the next marriage to occur will provide the same benefits as the average existing marriage. I wrote about how this wrong in Enduring Bonds:

The idea that the “benefits” of marriage—that is, the observed association between marriage and nonpoverty—would accrue to single mothers if they “simply” married their current partners is bonkers. The notion of a “marriage market” is not perfect, but there is something like a marriage queue that arranges people from most likely to least likely to marry. When you say, “Married people are better off than single people,” a big part of what you’re observing is that, on average, the richer, healthier, better-at-relationships people are at the front of that queue, more likely to marry and then to display what look like the benefits of marriage. Those at the back of the queue, who are more (if not totally) “unmarriageable,” clearly aren’t going to have those highly beneficial marriages if they “simply” marry the closest person.

In fact, I assume this problem has gotten worse as marriage has become more selective, as “it’s increasingly the most well off who are getting and staying married,” and those who aren’t marrying “may not have the assets that lead to marriage benefits: skills, wealth, social networks, and so on.”

Note on race

People who promote marriage don’t like to talk about race, but if it weren’t for race — and racism — they would never have gotten as far as they have in selling their agenda. They use supposedly race-neutral language to talk about fatherhood and a “culture of marriage” and “sustainably escaping poverty,” in ways that are all highly relevant to Black families and racial disparities. If you think the problem of marriage is that poor people are not marrying enough, you should not avoid the fact that you’re talking about race. Black women, especially mothers, are much less likely to be married than most other groups of women, even at the same level of income or education (last I checked Black college graduates were 5-times more likely than White college graduates to be single when they had a baby). So, don’t avoid that this is about race, own it  — the demographic facts and political machinations in this area are all highly interwoven with race. I do this analysis, like the paper Joanna and I did, separately for Black and White women, because that’s the main faultline in this area. The code I share below is adaptable to use with other groups as well.

Data illustration

In this data exercise I try to operationalize something like that marriage market queue, to show that women who are least likely to marry are also least likely to enter an economically beneficial marriage if they did marry. See how you like this, and let me know what you think. Or take the data and code and come up with a different way of doing it.

The logic is to take a sample of never-married women, and women who just got married in the last year, and predict membership in the latter group. This generates a predicted probability of marrying for each woman, and it means I can look at the never-married women and see which among them are more or less likely to marry in a given year. For example, based on the models below, I would estimate that a Black woman under age 25, with less than a BA degree, who had a job with less-than-average earnings, has a 0.4% probability of marrying in one year. On the other hand, if she were age 25+, with a BA degree and above-average earnings, her chance of marrying rises to 3.5% per year. (Round numbers.)*

Next, I look at the husbands of women who married men in the year prior to the survey, and I assign them economic scores on an 11-point scale (this is totally arbitrary): up to four points for education, up to four points for earnings, and up to three points for employment level (weeks and hours worked in the previous year). So, a woman whose husband has a high school education, earned $30,000 last year, and worked full-time, year-round, would have 7 points.

Finally, I show the relationship between the odds of marriage for women who didn’t get married and the economic score of the men they would have married if they did.

There are two descriptive conclusions, which I assumed I would find: (1) women who get married marry men with better economic scores than the women who don’t get married would if they did get married; and, (2) the greater the odds of marriage, the better the economic prospects of the man they would marry. The substantive conclusion from this is that marriage promotion, if it could get more people to marry, would pull from the women on the lower rungs of marriage probability, so those new marriages would be less economically beneficial than the average marriage, and the use of married people’s characteristics to project the benefits of marriage for unmarried people is wrong. Like I said, I already believed this, so this is a way of confirming it or showing the extent to which it fits my expectations. (Or, I could be wrong.)

Here are the details.

I use the 2012-2016 five-year American Community Survey data from IPUMS.org (for larger sample). The sample is women ages 18-44, not living in group quarters, single-race Black or White, non-Hispanic, and US-born. I further limited the sample to those who never married, and those who are married for the first time in the previous 12 months. That condition — just married — is the dependent variable in a model predicting odds of first marriage. (Women with female spouses or partners are excluded, too.) The variables used to predict marriage are age (and its square), education, earnings in the previous year (logged), and having no earnings in the previous year (these women are most likely to marry), disability status, metro area residence, and state dummy variables. It’s a simple model, not trying for statistical efficiency but rather the best prediction of marriage odds. Then I use the same set of variables, limiting the analysis to just-married women, to predict their husbands’ economic scores. The regression models are in a table at the end.**

Figure 1 shows how the prediction models assign marriage probabilities. White women have much higher odds of marrying, and those who married have higher odds than those who didn’t, which is reassuring. In particular, a large proportion of never-married Black women are predicted to have very low odds of marrying (click to enlarge).

f1

Figure 2 shows the distribution of husbands’ economic scores for Black and White women who married and those who didn’t. The women who didn’t marry have lower predicted husband scores, with the model giving them husbands with a mode of about 7.0 for Whites and 6.5 for Blacks (click to enlarge).

f2

Finally, the last figure includes only never-married women. It shows the relationship between predicted marriage probability and predicted husband score, using median splines. So, for example, the average unmarried Black woman has a marriage probability of about 1.7%. Figure 3 shows that her predicted husband would have a median score of about 6.4. So he could be a full-time, full-year worker with a high school education, earning $19,000 per year, which would not be enough to lift her and one child out of poverty. The average never-married White woman has a predicted marriage probability of 5.1%, and her imaginary husband has a score of about 7.4 (e.g., a similar husband, but earning $25,000 per year).

f3

Figure 3 implies  what I thought was obvious at the beginning: the further down the marriage market queue you go, the worse the economic prospects of the men they would marry, if there were men for them to marry (whom they wanted to marry, and who wanted to marry them).

I will now be holding my breath while marriage promotion activists develop a more sensible set of assumptions for their assessment of the benefits of the promoted marriages they assure us they will be able to conjure if only we give them a few billion more dollars.

I’m posting the data and code used on the Open Science Framework, here. Please feel free to work with it and let me know what you come up with!


* This looks pretty similar to what Dohoon Lee did in this paper, including his figures, and since I was on his dissertation committee, and read his paper, which has similar figures, I credit him with this idea — I should have remembered earlier.

** Here are the regression models used to (1) predict marriage, and then (2) predict husband’s economic scores.

marriage models.xlsx

 

8 Comments

Filed under Me @ work

Unequal marriage markets for Black and White women

Joanna Pepin and I have posted a new paper titled, “Unequal marriage markets: Sex ratios and first marriage among Black and White women.” In the paper, we find that the marriage markets of Black and White women are very different, with Black women living in metropolitan areas that have many fewer single men than White women do. And, in a regression model with other important predictors of marriage, this unmarried sex ratio is strongly associated with the odds of marrying.

We count this as evidence on the side of “structure” over “culture” in the debates over the decline in marriage. Here’s the main result, showing Black and White women in 172 metro areas (scaled for size), and the difference in sex ratios (the horizontal spread), the difference in marriage rates (the vertical spread), and the statistical effect of sex ratios on marriage (the slopes).

mmpif2

In a nutshell: As you move from left to right, there are more men, and higher odds of marriage. And almost all the White women are up and to the right compared with the Black women. One implication is that this could be one reason why marriage promotion programs in the welfare system aren’t working.

There are a couple of noteworthy innovations here. First, we used the American Community Survey marital events data, which is marriage happening (did you get married in the last year?) rather than just existing (are you married?). This is a better way to assess what might influence marriage. Second, young people, especially single young people who might be getting married, move around a lot. So what is their marriage market? It’s impossible to say exactly, but we define it as the metro area where they lived one year earlier, rather than just where they live now. (This is especially important because the people who move may move because they just got married.)

The paper is on SocArXiv, where if you follow the links you get to the project page, where we put most of the data and code. The paper is under review now, and we’d love to know if you find any mistakes or have any suggestions.

(This began with a blog post four years ago in which I critiqued a NYT Magazine piece by Anne Lowrie about using marriage to cure poverty. Then we presented a first pass at the American Sociological Association in 2014, and I put some of the descriptive statistics in my textbook, and we made a short video out of it, in which I said, “So, larger social forces — the economy, job discrimination, incarceration policies, and health disparities — all impinge on the ability of individuals to shape their own family lives.” Along the way, I presented some about it here and there, while thinking of new ways to measure marriage inequalities.)

6 Comments

Filed under Me @ work, Research reports

Who’s happy in marriage? (Not just rich, White, religious men, but kind of)

I previously said there was a “bonafide trend back toward happiness” within marriage, for the years 2006 to 2012. This was based on the General Social Survey trend going back 1973, with married people responding to the question, “Taking all things together, how would you describe your marriage?”

Since then, the bonafide trend has lost its pop. Here’s my updated figure:

hapmar16

I repeated this analysis controlling for age, race/ethnicity, and education, and year specified in quadratic form. This shows happiness falling to a trough at 2004 and then starting to trend back. But given the last two points, confidence in that rebound is weak. Still a solid majority are happy with their marriages.

Who’s happy?

But who are those happy in marriage people? Combining the last three surveys, 2012, 2014, and 2016, this is what we get (effect of age and non-effect of education not shown). Note the y-axis starts at 50%.

hapmar16c

So to be happy in marriage, my expert opinion is you should become male and White, see yourself as upper class, go to church all the time, and have extreme political views. And if you’re not all those things, don’t let the marriage promoters tell you what your marriage is going to be like.

Note: I previously analyzed the political views thing before, so this is an update to that. On trends and determinants of social class identification, see this post.)


Here’s my Stata code, written to run on the full GSS through 2016 data. Play along at home!

set maxvar 10000
use "GSS7216_R1a.dta", clear
gen since73 = year-1973
gen rwgt = round(wtssall)
keep if year >1972
gen verhap=0
replace verhap=1 if hapmar==1
logit verhap i.sex c.age##c.age i.degree i.race c.since73##c.since73 [weight=rwgt]
margins, at(since73=(0(1)43))
recode attend (1/3=1) (4/6=2) (7/8=3), gen(attendcat)
logit verhap i.sex c.age##c.age i.degree i.race i.class i.attendcat i.polviews if year>2010 [weight=rwgt]
margins sex race class attendcat polviews if year>2010

 

7 Comments

Filed under Research reports

’16 and Pregnant’ and less so

3419870216_fded1624d2_z

From Flickr/CC: https://flic.kr/p/6dcJgA

Regular readers know I have objections to the framing of teen pregnancy, as a thing generally and as a problem specifically, separate from the rising age at childbearing generally (see also, or follow the teen births tag).

In this debate, one economic analysis of the effect of the popular MTV show 16 and Pregnant has played an outsized role. Melissa Kearney and Phillip Levine showed that was more decline in teen births in places where the show was popular, and attempted to establish that the relationship was causal — that the show makes people under age 20 want to have babies less. As Kearney put it in a video promoting the study: “the portrayal of teen pregnancy, and teen childbearing, is something they took as a cautionary tale.” (The paper also showed spikes in Twitter and Google activity related to birth control after the show aired.)

This was very big news for the marriage promotion people, because it was taken as evidence that cultural intervention “works” to affect family behavior — which really matters because so far they’ve spent $1 billion+ in welfare money on promoting marriage, with no effect (none), and they want more money.

The 16 and Pregnant paper has been cited to support statements such as:

  • Brad Wilcox: “Campaigns against smoking and teenage and unintended pregnancy have demonstrated that sustained efforts to change behavior can work.”
  • Washington Post: “By working with Hollywood to develop smart story lines on popular shows such as MTV’s ’16 and Pregnant’ and using innovative videos and social media to change norms, the [National Campaign to Prevent Teen and Unplanned Pregnancy] has helped teen pregnancy rates drop by nearly 60 percent since 1991.”
  • Boston Globe: “As evidence of his optimism, [Brad] Wilcox points to teen pregnancy, which has dropped by more than 50 percent since the early 1990s. ‘Most people assumed you couldn’t do much around something related to sex and pregnancy and parenthood,’ he said. ‘Then a consensus emerged across right and left, and that consensus was supported by public policy and social norms. . . . We were able to move the dial.’ A 2014 paper found that the popular MTV reality show ’16 and Pregnant’ alone was responsible for a 5.7 percent decline in teen pregnancy in the 18 months after its debut.”

I think a higher age at first birth is better for women overall, health permitting, but I don’t support that as a policy goal in the U.S. now, although I expect it would be an outcome of things I do support, like better health, education, and job opportunities for people of color and people who are poor.

Anyway, this is all just preamble to a new debate from a reanalysis and critique of the 16 and Pregnant paper. I haven’t worked through it enough to reach my own conclusions, and I’d like to hear from others who have. So I’m just sharing the links in sequence.

The initial paper, posted as a (non-peer reviewed) NBER Working Paper in 2014:

Media Influences on Social Outcomes: The Impact of MTV’s 16 and Pregnant on Teen Childbearing, by Melissa S. Kearney, Phillip B. Levine

This paper explores how specific media images affect adolescent attitudes and outcomes. The specific context examined is the widely viewed MTV franchise, 16 and Pregnant, a series of reality TV shows including the Teen Mom sequels, which follow the lives of pregnant teenagers during the end of their pregnancy and early days of motherhood. We investigate whether the show influenced teens’ interest in contraceptive use or abortion, and whether it ultimately altered teen childbearing outcomes. We use data from Google Trends and Twitter to document changes in searches and tweets resulting from the show, Nielsen ratings data to capture geographic variation in viewership, and Vital Statistics birth data to measure changes in teen birth rates. We find that 16 and Pregnant led to more searches and tweets regarding birth control and abortion, and ultimately led to a 5.7 percent reduction in teen births in the 18 months following its introduction. This accounts for around one-third of the overall decline in teen births in the United States during that period.

A revised version, with the same title but slightly different results, was then published in the top-ranked American Economic Review, which is peer-reviewed:

This paper explores the impact of the introduction of the widely viewed MTV reality show 16 and Pregnant on teen childbearing. Our main analysis relates geographic variation in changes in teen childbearing rates to viewership of the show. We implement an instrumental variables (IV) strategy using local area MTV ratings data from a pre-period to predict local area 16 and Pregnant ratings. The results imply that this show led to a 4.3 percent reduction in teen births. An examination of Google Trends and Twitter data suggest that the show led to increased interest in contraceptive use and abortion.

Then last month David A. Jaeger, Theodore J. Joyce, and Robert Kaestner posted a critique on the Institute for the Study of Labor working paper series, which is not peer-reviewed:

Does Reality TV Induce Real Effects? On the Questionable Association Between 16 and Pregnant and Teenage Childbearing

We reassess recent and widely reported evidence that the MTV program 16 and Pregnant played a major role in reducing teen birth rates in the U.S. since it began broadcasting in 2009 (Kearney and Levine, American Economic Review 2015). We find Kearney and Levine’s identification strategy to be problematic. Through a series of placebo and other tests, we show that the exclusion restriction of their instrumental variables approach is not valid and find that the assumption of common trends in birth rates between low and high MTV-watching areas is not met. We also reassess Kearney and Levine’s evidence from social media and show that it is fragile and highly sensitive to the choice of included periods and to the use of weights. We conclude that Kearney and Levine’s results are uninformative about the effect of 16 and Pregnant on teen birth rates.

And now Kearney and Levine have posted their response on the same site:

Does Reality TV Induce Real Effects? A Response to Jaeger, Joyce, and Kaestner (2016)

This paper presents a response to Jaeger, Joyce, and Kaestner’s (JJK) recent critique (IZA Discussion Paper No. 10317) of our 2015 paper “Media Influences on Social Outcomes: The Impact of MTV’s 16 and Pregnant on Teen Childbearing.” In terms of replication, those authors are able to confirm every result in our paper. In terms of reassessment, the substance of their critique rests on the claim that the parallel trends assumption, necessary to attribute causation to our findings, is not satisfied. We present three main responses: (1) there is no evidence of a parallel trends assumption violation during our sample window of 2005 through 2010; (2) the finding of a false placebo test result during one particular earlier window of time does not invalidate the finding of a discrete break in trend at the time of the show’s introduction; (3) the results of our analysis are robust to virtually all alternative econometric specifications and sample windows that JJK consider. We conclude that this critique does not pose a serious threat to the interpretation of our 2015 findings. We maintain the position that our earlier paper is informative about the causal effect of 16 and Pregnant on teen birth rates.

So?

There are interesting methodological questions here. It’s hard to identify the effects of interventions that are swimming with the tide of change. In fact, the creation of the show, the show’s popularity, the campaign to end teen pregnancy, and the rising age at first birth may all be outcomes of the same general historical trend. So I’m not that invested in the answer to this question, though I am very interested.

There are also questions about the publication process, which I am very invested in. That’s why I work to promote a working paper culture among sociologists (through the SocArXiv project). The original paper was posted on a working paper site without peer review, but NBER is for economists who already are somebody, so that’s a kind of indirect screening. Then it was accepted in a top peer-reviewed journal (somewhat revised), but that was after it had received major attention and accolades, including a New York Times feature before the working paper was even released and a column devoted to it by Nicholas Kristof.

So is this a success story of working paper culture gone right — driving attention to good work faster, and then also drawing the benefits of peer review through the traditional publication process? (And now continuing with open debate on non-gated sites). Or is it a case of political hype driving attention inside and outside of the academy — the kind of thing that scares researchers and makes them want to retreat behind the slower, more process-laden research flow which they hope will protect them from exposure to embarrassment and protect the public from manipulation by the credulous news media. I think the process was okay even if we do conclude the paper wasn’t all it was made out to be. There were other reputational systems at work — faculty status, NBER membership, New York Times editors and sources — that may be as reliable as traditional peer review, which itself produces plenty of errors.

So, it’s an interesting situation — research methods, research implications, and research process.

2 Comments

Filed under Research reports

The new Wilcox thing is complete bologna and/or just dishonest

Update: after Wilcox updated his report with the complete data, I now conclude the report is just dishonest, not complete bologna. See below. 

Brad Wilcox and Nicholas Zill have a new report on Brad’s Institute for Family Studies website, about Arizona: “Stronger Families, Better Schools: Families and High School Graduation Across Arizona.” It bears a strong resemblance to a previous report, about Florida, “Strong Families, Successful Schools: High School Graduation and School Discipline in the Sunshine State.” Together these two give you a feeling like taking the first two bites of a leftover giant burrito that might have gone a little bad, and realizing there are probably about 48 more bites to go.

Anyway, this is about the Arizona one. I’ll first raise the possibility that it’s complete bologna – as in, fraudulent or error-ridden – and then discuss how it’s conclusions are dishonest at best even if the analysis is not technically wrong but rather just presented terribly.

Update: with the report corrected to show the complete data, the analysis now replicates fine. So I set aside the bologna issue. I leave this section here just so you can see the research design, but the main argument is in the next section.

First, the bologna issue

The report uses demographic data from 99 Arizona school districts to model graduation rates, and the gender gap in graduation rates. Their conclusion, based on two regression models using districts as the units of analysis and demographic indicators as the predictors, is this:

In Arizona, public school districts with better-educated and more married parents boast higher high school graduation rates. Gender equity is also greater in districts with more married parents. That is, boys come closer to matching the high school graduation rates of girls in districts with more married-parent families. Moreover, married parenthood is a better predictor of these two high school graduation outcomes than are child poverty, race, and ethnicity in public school districts across the Grand Canyon state.

To pad out the report, they also include appendix tables, so it’s theoretically possible to replicate their regressions. Unfortunately, unless I’m missing something, they don’t replicate. I wouldn’t normally bother rerunning someone’s regression, especially when the argument they’re building is so wrong-headed (see below), but just because we know from long experience that Wilcox does not behave honestly (in methods, ethics, and ethics) what the heck.

The report says, “Graduation rates and male/female graduation ratios for the 99 Arizona school districts in our study are shown in Table A1 in the Appendix.” Table A2 then lists the districts again, with the demographic variables. Unfortunately, table A2 only includes 83 districts – and the 16 missing are exactly those from Indian-Oasis to Paradise Valley in the alphabetical list of district names, so apparently an error handling the data. So I could only use 83 of the 99 for the regressions. Since I don’t know when they lost those 16 districts, I don’t know if it was before or after running the regressions (there are no Ns or standard errors on their regression tables).

For each of their dependent variables – graduation rate, and the male/female ratio in graduation rates – they list bivariate correlations, and adjusted betas from as multivariate regression. Here are their figures, with mine next to them. The key differences are highlighted:

aztab

If they’re using 99 cases and I have 83 (actually 81 for the gender cap because of missing data), you would expect some difference. But these are very similar, including the bivariate correlations and the R-squareds for the models.

The weird thing is that the biggest difference is exactly on their biggest claim: “married parenthood is a better predictor of these two high school graduation outcomes than are child poverty, race, and ethnicity…” That is based on the assertion that .29 is larger than -.28 (very luck for them, that tiny, insignificant difference in  magnitude!). In my model the minority-size effect is more than twice as large as the marriage-parenthood effect. So, huh. It’s definitely possible Brad simply lied about his results and made up a few numbers. (And I’m just using the data they include in the report.) But now let’s pretend he didn’t.

Update: with the complete data I can report that those two betas are actually .2865 (.29!) versu .2847 (.28!). The idea that one is a “better” predictor than the other is clearly not serious. Further, for some reason (we can only guess), they combined percent Black, Hispanic, and American Indian together into “minority,” which produced the .28 result. If they had entered them into the model separately, they would find that Hispanic and American Indian effects are each bigger than the married parent effect, as I show here:


So much for the headline result. Anyway, back to the argument…

Policy, shmolicy

The point of the analysis is to make policy recommendations. They conclude:

If the state enjoyed more stable families, it might also see better educational outcomes among its children. It’s for that reason that Arizona should consider measures designed to strengthen and stabilize families.

Their recommendations to that end are vocational education and marriage promotion.

Private and public initiatives to provide social marketing on behalf of marriage could prove helpful. Campaigns against smoking and teenage pregnancy have taught us that sustained efforts to change behavior can work.

First, I’m not an education specialist (and neither are they), but shouldn’t there be some kind of policy variables in this analysis, like per-pupil spending, or teacher salaries, or something about curriculum or programming? It’s unusual to use only demographic variables and then conclude that what we need is a policy to change the demographics. It’s just not a serious analysis. (Please also please remember that “controlling for income” is not an adequate control for economic conditions and status.)

But second, given the first billion dollars of money spent promoting marriage produced absolutely no increase in marriage, is there any possible way Brad legitimately thinks this is the best way to improve graduation rates?

These are just two ideas. More should be explored. The bottom line: policymakers, educators, business leaders, and religious leaders in Arizona need to address the fragile foundations of family life if they hope for the state’s children to lead the nation in academic achievement.

Does this report really support that “bottom line”? Would it be better to spend money promoting marriage than to spend the same amount of money on some effort to improve schools? That’s obviously a dumb idea, but is it possible he really believes it? These are the only policies proposed. Maybe I’m wrong, but I doubt he believes it. I think he wants to promote marriage promotion programs for other reasons: to fund him and his compatriots, to support pro-marriage ideology, and so on. Not to improve graduate rates in Arizona schools. But, maybe I’m wrong.

And a laptop

I think what Brad is really doing is noise noise statistics statistics marriage-is-good expertise trust me fund me. The details clearly aren’t that important.

Meanwhile, not coincidentally, things are looking up for Brad at the Institute for Family Studies (IFS), the organization he created to handle the foundation-money rake. He started in 2011 as president / director of IFS at a salary of $35,000. After paying himself a paltry $9,999 and in 2012, he started improving his productivity, paying himself $50,000 in 2013, and then $80,400 in 2014 as a Senior Fellow, the last year for which I found a 990 form. Much of that money is coming from the Bradley Foundation (which also funded the Regnerus/Wilcox study) — their 2015 report lists $75,000 for IFS, so projections are good for next year. This is, of course, on top of what he gets for his service to the public at the University of Virginia.

The IFS disclosure forms also show purchase of a MacBook Pro. Which might or might not have been for Brad.

bradslaptop

I do not make this case, and make it personally, because I disagree with Brad about politics. There are lots of people I disagree with even more than him, and I don’t spend all day criticizing them. The dishonesty offends me because it’s work and issues I care about, it hurts real people, I’m well situated to expose it, and his corporate-Christian-right megaphone is big, so it shouldn’t go unchallenged.

2 Comments

Filed under In the news