Author Archives: Philip N. Cohen

Michigan Black college completion falters (with consequences)

Yesterday the Supreme Court ruled that Michigan voters have the Constitutional right to ban the state’s government from using race-specific policies. The immediate implication for Michigan, and other states, is for university admissions polices. So now if the state wants to pass a law allowing children of alumni easier admission to the University of Michigan, it’s a simple act of the legislature; but if they want to consider race in their admissions, they will need to amend the state constitution.

The University of Michigan has been at the center of national affirmative action debates for several decades (at least since I arrived there in 1988). I previously reported that court decisions against the state’s affirmative action policy led to a precipitous decline in Black students entering the University in the 2000s, as shown in this graph:

That’s just the University of Michigan, an important school, but only one. (The New York Times has a graphic showing enrollment trends in a series of states with affirmative action bans.) For the whole state of Michigan, Black college graduation rates fell further behind the national average over the last decade. Here is the percent of Black 25-29 year-olds who have completed college, from 1970 to 2012, nationally versus in Michigan alone, for women (left) and men (right):


Source: 1970-2000 Decennial Censuses and 2010-2012 American Community Survey, via IPUMS.

During the 2000s, the national-Michigan gap widened from 2.3 points to 4.1 points for men, and from 3.4 to 4.8 points for women.

I am not expert in the legal arguments over this, so I can’t analyze the decision (here’s one good take). But regardless of whether it’s bad law, I think it’s bad policy.

Yesterday in a tweet I picked on the new, data-heavy news operations run by (from left to right) David Leonhardt (NY Times Upshot), Ezra Klein (Vox), and Nate Silver (Five Thirty Eight) for having very White-looking staff teams:


I don’t know any more about what goes into their hiring decisions than I do about what goes into University of Michigan admission decisions (and I know they have staff beyond these featured writers). I’m sure they all want talented people with a wide range of perspectives and skills. But the outcome in both the media and college situations is bad. It limits the perspectives presented, undermines progress toward racial-ethnic equality, and contributes to the inertia that stymies the potential of future leaders.


Filed under In the news

Does gay marriage make straight men hate children?

A few comments on a recent brief against marriage equality in Utah. But first some background.

As public opinion has shifted so dramatically on same-sex marriage, there has been some consternation about the ill treatment of those left behind — those opposed to marriage equality — as if they were nothing but common racists, whose hateful motivations may be divined from their policy conclusions rather than from knowing the love in their hearts.

Barry Deutsch has written a great response to this, pointing out that the sophisticated racists during the debate over interracial marriage made the same claim that the anti-marriage equality people make today. They were not motivated by hatred, they were not racist, they merely opposed a new, untested form of marriage that happens to go against tradition and the natural order, and would probably harm children. Especially the children.

Oh, no. Gay marriage is coming. Should I catch her? Photo by Mike Baird from the Flickr Creative Commons

Oh, no. Gay marriage is coming. Do I catch her? Photo by Mike Baird from the Flickr Creative Commons

Run, hide, double down

The smart conservative money in the last year or two has moved away from all this. Among those public intellectuals who labored to block their gay and lesbian fellow citizens from crossing the threshold of matrimony (under the terms of their choosing, at least), there are three approaches.

  • The most openly forward-looking, such as David Blankenhorn, publicly reversed course and threw in the towel. Blankenhorn’s Institute for American Values has shifted to the movement against gambling (joining a sadly low-rent effort that unites Blankenhorn with the likes of Barrett Duke, a veteran of the crusade against the “homosexual special rights agenda“).
  • The more duplicitous, such as Brad Wilcox, simply avoid discussing the issue in public. Hard to believe these folks have no opinions on the subject, considering Wilcox’s efforts to generate research in opposition to marriage equality. But his new Institute for Family Studies (IFS) seems not to have mentioned this issue – even though its nominal president, Richard Brake, was (and is at press-time still listed as) National Education Director for the Intercollegiate Studies Institute, which has as its mission preventing the spread of a “relativism that rejects an objective moral order.” (The Lynde and Harry Bradley Foundation, which paid for some of the Regnerus study to prevent marriage equality, also funds ISI.)
  • Finally, a contingent of obdurate cranks continues to resist the new moral order, marriage equality included. I wrote about two of them, Mark Regnerus and Douglas Allen, who testified in Michigan’s recent losing battle. But this group also includes Alan Hawkins and Jason Carroll, two professors of Family Life at Brigham Young University.

Hawkins and Carroll

I hadn’t read, until recently, the amicus brief filed by Hawkins and Carroll in Utah’s attempt to stop (or re-stop) marriage equality, which is available here. Before I describe it, though, a quick word about these two. Hawkins has showed up here for his shoddy research in defense of (straight) marriage promotion. He and Carroll have both done paid work for the federal marriage promotion campaign. And they are both part of the Wilcox brand, Hawkins as a contributor to the IFS blog and Carroll as a co-author of his Knot Yet report.

At BYU, Hawkins has expressed concern about how modernity might affect the ability of Family Life graduates to get jobs:

“A very real risk is that there will possibly be formal litmus tests in graduate programs out there,” Hawkins said. “We’re already seeing informal ones in some graduate programs. It’s not just saying, ‘I’m willing to work with same-sex couples and families.’ It’s more than work, it’s that students’ beliefs and attitudes will have to align with the new, contemporary definition of marriage.”

In other words, in the new relativist moral order, it may be difficult to get a job or spot in graduate school in say, family therapy, if you believe your legally married gay or lesbian clients don’t have a right to get married on their way to spirit prison, or worse. To some of us, I suspect this is pretty close to the definition of progress.

Anyway, in the Utah case, the state recently dumped Regnerus’s argument that same-sex marriage directly harms children, in favor of the argument that same-sex marriage hurts straight marriage. (I played around with this empirically a little when Utah first appealed the federal court’s decision to overturn their marriage ban.) Hawkins and Carroll attempt to make this case theoretically.

They pretty much sum it up in the table contents, which directs the reader to page 18 if they want to read this:

Traditional, gendered marriage is the most important way heterosexual men create their masculine identities. Marriage forms and channels that masculinity into the service of their children and society. Redefining marriage to include same-sex couples would eliminate gender as a crucial element of marriage and thus undermine marriage’s power to shape and guide masculinity for those beneficial ends.

The details involve a lot of untestable assertions about how (straight) marriage shapes men’s masculinity, followed by what read as not only untestable but frankly paranoid assertions about how this would all change if marriage were to lose its gendered character. Because, all the bad things that are already happening to marriage will only be amplified by letting more gay people get married:

Many of the historical supports that have traditionally preserved men’s involvement in their children’s lives have been eroding for contemporary families. Historically high rates of non-marital cohabitation, out-of-wedlock childbirth, and marital divorce have dramatically altered the landscape of fathering, leaving unprecedented numbers of children growing up with uncertain or nonexistent relationships with their fathers. …any signal that men’s contributions are not central to children’s well-being threatens to further decrease the likelihood that they will channel their masculine identities into responsible fathering. We believe the official de-gendering of marriage sends just such a signal.

Yes, the very existence of gay marriage will encourage the evolutionary tendency of (straight) men to neglect their children. They go on to concede that such an indirect effect would be hard to detect. But that doesn’t make it any less important:

To be sure, these risks associated with same-sex marriage may be difficult to disentangle from negative effects from other strong social changes. After all, we believe a de-gendered understanding of marriage is an additional force in a larger trend that is uncoupling sexuality, marriage, and parenthood and making men’s connections to children weaker. Thus, it may be difficult to separate statistically the potential effects of de-gendering marriage from the effects stemming from powerful forces to which it is related, such as the sexual revolution, the divorce revolution, and the single-parenting revolution. That these effects are intertwined with the effects of other powerful forces, however, does not diminish their importance or the harms they can impose on marriage.

Of course, the same could be said of all the negative effects of the sexual revolution, divorce revolution, and single-parenting revolution — which are just a little too difficult to detect, what with all the increase in women’s status and independence, decrease in crime and family violence, increased educational attainment (for men and women), rising life expectancy and plummeting teen birth rates that have accompanied these catastrophic family changes.

If anyone really believes this stuff, it is still hard to believe that they believe the courts will go for it in the post-Windsor era.


Filed under Politics

How well do teen test scores predict adult income?

Now with new figures and notes added at the end!

The short answer is, pretty well. But that’s not really the point.

In a previous post I complained about various ways of collapsing data before plotting it. Although this is useful at times, and inevitable to varying degrees, the main danger is the risk of inflating how strong an effect seems. So that’s the point about teen test scores and adult income.

If someone told you that the test scores people get in their late teens were highly correlated with their incomes later in life, you probably wouldn’t be surprised. If I said the correlation was .35, on a scale of 0 to 1, that would seem like a strong relationship. And it is. That’s what I got using the National Longitudinal Survey of Youth. I compared the Armed Forces Qualifying Test scores, taken in 1999, when the respondents were ages 15-19 with their household income in 2011, when they were 27-31.*

Here is the linear fit between between these two measures, with the 95% confidence interval shaded, showing just how confident we can be in this incredibly strong relationship:


That’s definitely enough for a screaming headline, “How your kids’ test scores tell you whether they will be rich or poor.” And it is a very strong relationship – that correlation of .35 means AFQT explains 12% of the variation in household income.

But take heart, ye parents in the age of uncertainty: 12% of the variation leaves a lot left over. This variable can’t account for how creative your children are, how sociable, how attractive, how driven, how entitled, how connected, or how White they may be. To get a sense of all the other things that matter, here is the same data, with the same regression line, but now with all 5,248 individual points plotted as well (which means we have to rescale the y-axis):


Each dot is a person’s life — or two aspects of it, anyway — with the virtually infinite sources of variability that make up the wonder of social existence. All of a sudden that strong relationship doesn’t feel like something you can bank on with any given individual. Yes, there are very few people from the bottom of the test-score distribution who are now in the richest households (those clipped by the survey’s topcode and pegged at 3 on my scale), and hardly anyone from the top of the test-score distribution who is now completely broke.

But I would guess that for most kids a better predictor of future income would be spending an hour interviewing their parents and high school teachers, or spending a day getting to know them as a teenager. But that’s just a guess (and that’s an inefficient way to capture large-scale patterns).

I’m not here to argue about how much various measures matter for future income, or whether there is such a thing as general intelligence, or how heritable it is (my opinion is that a test such as this, at this age, measures what people have learned much more than a disposition toward learning inherent at birth). I just want to give a visual example of how even a very strong relationship in social science usually represents a very messy reality.

Post-publication addendums

1. Prediction intervals

I probably first wrote about this difference between the slope and the variation around the slope two years ago, in a futile argument against the use of second-person headlines such as “Homophobic? Maybe You’re Gay.” Those headlines always try to turn research into personal advice, and are almost always wrong.

Carter Butts, in personal correspondence, offered an explanation that helps make this clear. The “you” type headline presents a situation in which you – the reader — are offered the chance to add yourself to the study. In that case, your outcome (the “new response” in his note) is determined by the both the line and the variation around the line. Carter writes:

the prediction interval for a new response has to take into account not only the (predicted) expectation, but also the (predicted) variation around that expectation. A typical example is attached; I generated simulated data (N=1000) via the indicated formula, and then just regressed y on x. As you’d expect, the confidence bands (red) are quite narrow, but the prediction bands (green) are large – in the true model, they would have a total width of approximately 1, and the estimated model is quite close to that. Your post nicely illustrated that the precision with which we can estimate a mean effect is not equivalent to the variation accounted for by that mean effect; a complementary observation is that the precision with which we can estimate a mean effect is not equivalent to the accuracy with which we can predict a new observation. Nothing deep about that … just the practical points that (1) when people are looking at an interval, they need to be wary of whether it is a confidence interval or a prediction interval; and (2) prediction interval can (and often should be) wide, even if the model is “good” in the sense of being well-estimated.

And here is his figure. “You” are very likely to be between the green lines, but not so likely to be between the red ones.


2. Random other variables

I didn’t get into the substantive issues, which are outside my expertise. However, one suggestion I got was interesting: What about happiness? Without endorsing the concept of “life satisfaction” as measured by a single question, I still think this is a nice addition because it underscores the point of wide variation in how this relationship between test scores and income might be experienced.

So here is the same figure, but with the individuals coded according to how they answered the following question in 2008, when they were age 24-28, “All things considered, how satisfied are you with your life as a whole these days? Please give me an answer from 1 to 10, where 1 means extremely dissatisfied and 10 means extremely satisfied.” In the figure, Blue is least satisfied (1-6; 21%), Orange is moderately satisfied (7-8; 46%), and Green is most satisfied (9-10; 32%)


Even if you squint you probably can’t discern the pattern. Life satisfaction is positively correlated with income at .16, and less so with test scores (.07). Again, significant correlation — not helpful for planning your life.

* I actually used something similar to AFQT: the variable ASVAB, which combines tests of mathematical knowledge, arithmetic reasoning, word knowledge, and paragraph comprehension, and scales them from 0 to 100. For household income, I used a measure of household income relative to the poverty line (adjusted for household size), plus one, and transformed by natural log. I used household income because some good test-takers might marry someone with a high income, or have fewer people in their households — good decisions if your goal is maximizing household income per person.


Filed under Me @ work

How many WWII war brides are still living?

Maybe a couple thousand.

European war brides arriving in New York, 1945

European war brides arriving in New York, 1945

Someone should do some new interviews with the World War II “war brides,” because there aren’t very many still living.

I count 1,195 still married and living with their husbands. That means there might be something like 2,000 living if you count widows and those who have remarried. We don’t know exactly how many there were, but various sources put the number at 60,000 or more.

Here’s how I got that current number, using the American Community Survey three-year file, 2010-2012. It’s all the couples who met the following conditions:

  • Married, spouse-present
  • She was born outside the U.S.
  • He was born in the U.S.
  • He is a WWII-era veteran
  • They were married in the years 1941-1945
  • She immigrated in or after the year of their marriage

It’s a pretty simple set of rules.

Some caveats: This doesn’t include any widows or widowers, just those still married (otherwise the ACS doesn’t have any spouse information). I didn’t set a requirement that she be born in a place where American soldiers were during the war (I don’t know all the places they were). I don’t know that all of the WWII-era veterans served outside the U.S. So some of these might not be real war brides, in the sense of women who met and married American military men outside the U.S. during a war.

Still, I think the formula works well. These are the women it turned up:

  • 84% immigrated in 1945 or 1946
  • The age range is 82-94, with a median of 85
  • About two-thirds were under age 20 when they married
  • 61% from the United Kingdom (mostly England)
  • 11% from elsewhere in Western Europe (France, Belgium, Italy)
  • 7% from Eastern Europe (Czechoslovakia, Yugoslavia)
  • The remaining 20% from Canada, Australia/New Zealand, Israel/Palestine, Japan, other)

If you follow my suggestion of finding and interviewing these women or their husbands, here are some other sources you might use:


Leave a comment

Filed under Uncategorized

What do doctors, lawyers, police, and librarians Google?

Now with college teachers!

What do doctors, lawyers, police, and librarians Google? I’ll tell you. But first — if you are going to take this too seriously, please stop now.

Data and Method

Using IPUMS to extract data from the 2010-2012 American Community Survey, I count the number of people ages 25-64, currently employed, in a given occupation. I divide that by each state’s population in that age range (excluding Washington DC from all analyses). I enter those numbers into the Google Correlate tool to see which searches are most highly correlated with the distribution of each occupation across states (the tool reports the top 100 most correlated searches). In other words, these are searches that maximize the difference between, for example, high-lawyer and low-lawyer states — searches that are relatively popular where there are a lot of lawyers, and relatively unpopular where there are not a lot of lawyers.

Is this what lawyers actually Google? We can’t know. But I think so. Or maybe what people who work in law firms do, or people who live with lawyers. It’s a very sensitive tool. I made this case first in the post, Stuff White People Google. Check that out if you’re skeptical.

For each occupation, I first offer a few highly correlated searches that support the idea that the data are capturing what these people search for. Then I list some of the interesting other hits from each list.



Police per adult

Police per adult

The map of police per adult looks pretty random, but the list of correlated search terms doesn’t. On the list are “security training,” “tsa jobs,” “waist belt,” “weight vest,” and “air marshals.”

After all the security stuff, the only major category left in the 100 searches most correlated with police in the population is women. Specifically, their search taste includes tough actress Rachel Ticotin, body builder Denise Masino, Brazilian actress Alice Braga, actress Rosario Dawson, and, “israeli women.” (Remember, Google suppresses known porn terms, so this is just what got through the filter.) It’s a leap from this data to the statement, “police search for images of these women,” but this is who they would find if that were the case (is this a “type”?):



Librarians per adult

Librarians per adult

On the other hand, librarians. They are the smallest occupation I tried: the average state population aged 25-64 is only one tenth of one percent librarians. Yet, their distribution leaves an unmistakable trace in the Google search patterns. It especially seems to pick up terms associated with public libraries. Correlated terms include, “cataloguing,” and “quiet hours.” And then there are terms one might ask a librarian about, classic reference-desk questions such as, “which vs that,” “turn off track changes,” “think tanks,” “9/11 commission,” and “irs form 6251″; and term paper topics like Shakespeare titles or “human development report.”

What about the librarians themselves, or those close to them? Could it be they who are searching for Ann Taylor dresses, Garnet Hill free shipping, Lands End home, and textile museums? We can’t know for sure. Of course, if anyone knows how to cover their search tracks, it might be this crowd.


Doctors per adult

Doctors per adult

You know they’re doctors, because the search terms most correlated the map include “md, mph,” “md, phd,” “nejm,” “journal medicine,” “tedmed,” and “groopman.” What else do they like? Chic Corea, Tina Fey, Larry David, Mad Men (season 1) and The West Wing, Laura Linney, John Oliver, Scrabble 2-letter words, and a bunch of Jewish stuff.


Lawyers per adult

Lawyers per adult

That’s the map of lawyers per adult across states. Is it really lawyers? The top 100 searches correlated with the distribution shown above include “general counsel,” and then a lot of financial terms like, “world economic forum,” “international finance corporation,” and “economist intelligence.” Then there are international travel terms, like, “rate euro dollar,” “royal air,” and “swiss embassy.”

Looks like lawyers in lawyer-land are richer and more finance-oriented than lawyers in general. On the cultural side, they search for clothing terms Massimo Dutti, Hugo Boss, and Benetton. They apparently like to eat at Zafferano in London, and drink Caipirinhas. Also, they like “vissi,” which is an aria from Tosca but also a Cypriot celebrity; I lean toward the latter, because Queen Rania is also on the list. Finally, they combine their interests in law, finance, and wealthy attractive women by searching for Debrahlee Lorenzana, the “too-hot-for-work” banker.

By popular demand: Post-secondary teachers


Finally, here without comment are the results for “post-secondary teachers,” which includes any college teacher who didn’t instead specify a specialty, such as “psychologist” or “economist.” (It’s hard to see on the map, but Rhode Island is the highest.) I broke the results into four rough categories:


bmi index
body image
citation style
critical theory
debt to equity
debt to equity ratio
democracy in america
economic inequality
economic statistics
edward elgar
effect size
email forward
equals sign
google scholar
growth rates
inflation rate
inflation rates
international study
journal of
journal of nutrition
marginal propensity
marginal propensity to consume
meters per second
piano sonata
prefrontal cortex
profile of
psychology studies
quick ratio
rejection letter
returns to scale
ways to end a letter


1% milk
2006 olympics
best pump up songs
crib safety
easy halloween costume
graco snug
ipod history
jackson superbowl
janet jackson superbowl
mastermind game
maxim online
most popular names
national sleep foundation
olympic figure skating
olympics 2006
pairs figure skating
sandra boynton
senior hockey
snl clips
stuff magazine
stumbled upon
toilet training


1812 overture
acapella group
acapella groups
africa toto
ave verum
for the longest time
it breaks my heart
pdq bach
taylor swift

Birth control

apri birth control


Poor social scientists, generations of them spending their lives raising a few thousand dollars to ask a few thousand people a few hundred stilted, arbitrary survey questions. Meanwhile, coursing through the cable wires below their feet, and through the air around them, billions of data bits carry so much more potential information about so many more people, in so many intimate aspects of their lives, then we could even dream of getting our hands on. Just think of the power!

RingfrodoNote: I’ve done many posts like this. Some use time series instead of geographic variation, some use terms from Google Books ngrams. Browse the series under the Google tag, or check out this selection:




Filed under Me @ work

Response from Supporting Healthy Marriage supporters, with responses

In response to yesterday’s post, “This ‘Supporting Healthy Marriage,’ I do not think it means what you think it means,” Phil and Carolyn Cowan posted a comment, which I thought I should elevate to a new post.

Photo by Ben Francis from Flickr Creative Commons

Photo by Ben Francis from Flickr Creative Commons

Here is their comment, in full, with my responses.

Since the issue here is one of perspective in reporting, we (Phil Cowan and Carolyn Cowan) need to say that we were two of a group of academic consultants to the Supporting Healthy Marriage Project.

Thank you for acknowledging that. I noticed that Alan Hawkins, in his comment on the new study for Brad Wilcox’s blog, says he has “published widely on the effectiveness of marriage and relationship education programs,” but doesn’t say who paid for that voluminous research (with its oddly consistent positive findings). More about his Hawkins below.

Social scientists who want to inform the public about the results of an important study should actually inform the public about the results, not just give examples that support the author’s point of view.

Naturally, which is why I publicized the study, provided a link to it in full, and provided the examples quoted below.

It’s true as you report that there were no differences in the divorce rate between group participants and controls (we can debate whether affecting the divorce rate would be a good outcome), and that… [quoting from the original post]

“…there were no differences in the divorce rate between group participants and controls and “there were small but sustained improvements in subjectively-measured psychological indicators. How small? For relationship quality, the effect of the program was .13 standard deviations, equivalent to moving 15% of the couples one point on a 7-point scale from “completely unhappy” to “completely happy.” So that’s something. Further, after 30 months, 43% of the program couples thought their marriage was “in trouble” (according to either partner) compared with 47% of the control group. That was an effect size of .09 standard deviations. So that’s something, too. Many other indicators showed no effect. However, I discount even these small effects since it seems plausible that program participants just learned to say better things about their marriages. Without something beyond a purely subjective report — for example, domestic violence reports or kids’ test scores — I wouldn’t be convinced even if these results weren’t so weak.”

1. A slight uptick in marital satisfaction. The program moved 15% of the couples up one point. But more than 50 studies show that without intervention, marital quality, on the average goes down. And, it isn’t simply that 15% of the couples moved up one point. Since this is the mean result, some moved less (or down) but some moved up. Some also moved up from the lower point to relationship tolerability.

It is interesting that, with so many studies showing that marital quality goes down without intervention, this is not one of them. That is important because of what it implies about the sample. Quoting from the report now (p. 32):

At study entry, a fairly high percentage (66 percent) of both program and control group couples said that they had recently thought their marriage was in trouble. This percentage dropped across both research groups over time. This finding is contrary to much of the literature in the area, which generally suggests that marital distress tends to increase and that marital quality tends to decline over time. The decline in marital distress was initially steeper for program group members, and the difference between the program and control groups was sustained over time. This suggests that couples may have entered the program at low points in their relationships.

Back to the Cowans:

While the effects were small (but statistically reliable), they were hardly trivial. For instance, two years after the program, about 42% of SHM couples reported that their marriage had been in trouble recently compared to about 47% of control-group couples. That 5% difference means nearly 150 more SHM couples than control-group couples felt that their marriage was solid.

There are several problems here.

First, this paragraph appears verbatim in Hawkins’ post as well. I’m not going to speculate about how the same paragraph ended up in two places — there are some obvious possibilities — but clearly someone has not communicated the origin of this passage.

Second, this is not the right way to use “for instance.” This “for instance” refers to the only outcome of any substantial size in the entire study. It is not an “instance” of some larger pool of non-trivial results, it is the outlier. (And “solid” is not the same as not saying the marriage is “in trouble.”)

Anyway, third, this phrase is just wrong: “small (but statistically reliable)… hardly trivial.” For most of the positive outcomes they were exactly so small as to be trivial, and exactly not statistically reliable. Quoting from the report again, on coparenting and parenting (p. 39):

Table 9 shows that, of the 10 outcomes examined, only three impacts are statistically significant. The magnitudes of these impact estimates are also very small, with the largest one having an effect size of 0.07. These findings did not remain statistically significant after additional statistical tests were conducted to adjust for the number of outcomes examined. In essence, the findings suggest that there is a greater than 10 percent chance that this pattern of findings could have occurred if SHM had no effect on coparenting and parenting.

And quoting from the report again, on child outcomes (p. 41):

Table 10 shows that the SHM program had statistically significant impacts on two out of four child outcomes, but the impacts are extremely small. SHM improved children’s self-regulatory skills by 0.03 standard deviation, and it reduced children’s externalizing behavior problems by 0.04 standard deviation. … The evidence of impacts on child outcomes is further weakened by the results of subsequent analyses that were conducted to adjust for the number of outcomes examined. These findings suggest that there is a greater than 10 percent chance that this pattern could have occurred if SHM had no effect on child outcomes.

In other words, trivial effects, and not statistically reliable.

2. You say that “Without something beyond a purely subjective report…I wouldn’t be convinced even if these results weren’t so weak.” You were content to focus on two self-report measures. At the 18 month follow-up, program group members reported higher levels of marital happiness, lower levels of marital distress, greater warmth and support, more positive communication skills, and fewer negative behaviors and emotions in their interactions with their spouses, relative to control group members. They also reported less psychological abuse (though not less physical abuse). These effects continued at the 36 month follow-up [should be 30-month -pnc]. Observations of couple interaction (done only at 18 months) indicated that the program couples, on average, showed more positive communication skills and less anger and hostility than the control group. Because the quality of these interactions of the partners, the effects, though small, were coded by observers blind to experimental status of the participants, meaning that not only the self-reports suggest some positive effects but observers could identify some differences between couples in the intervention and control groups that we know are important to couple and child well-being.

I am confused by this. The description of the variables for communication skills and warmth (p. 67) describes them as answers to survey questions, not observations (e.g., “We are good at working out our differences”). I’m looking pretty hard and not seeing what is described here. The word “anger” is not in the report, and the word “hostility” only occurs with regard to parents’ behavior toward children. Someone please point me to the passage that contradicts me, if there is one.

3. When all the children were considered as one group, regardless of age, there were no effects on child outcomes, but there WERE significant effects on younger children (age 2-4), compared with children 5 to 8.5 and children 8.5 to 17. The behaviors of the younger children of group participants were reported to be – and observed to be — more self- regulated, less internalizing (anxious, depressed, withdrawn), and less externalizing (aggressive, non-cooperative, hyperactive). It seems reasonable to us that a 16 week intervention for parents might not be sufficient to reduce negative behavior in older children.

On the younger children, I discounted that because the report said (p. 42): “While the findings for the youngest children are promising, there is some uncertainty because the pattern of results is not strong enough to remain statistically significant once adjustments are made to account for the number of outcomes examined.”

4. For every positive outcome we have cited, you or any critic can find another measure that shows that the intervention had no effect. That’s part of our point here. Rather than yes or no, what we have is a complicated series of findings that lead to a complicated series of decisions about how best to be helpful to families.

That’s just not an accurate description. There are many null findings for each positive finding, and the positive findings themselves are either small, trivially small, or not statistically reliable.

4. Several times you suggest that giving couples the $9,000 per family (the program costs) would do better. Do you have evidence that giving families money increases, or at least maintains, family relationship quality? Is $9,000 a lot? Compared to what? According to the Associated Press, New York city’s annual cost per jail inmate was $167,731 last year. In other words, we are already spending billions to serve families when things go wrong, and some of the small effects of the marital could be thought of as preventive – especially at earlier stages of children’s development.

At the end of your blog, you rightly suggest a study in which giving families money is pitted in a random trial against relationship interventions. That’s a good idea, but that suggests more research. Furthermore, why must we always discuss programs in terms of yes or no, good or bad? What if we gave families $9,000 AND provided help with their relationships – and tested for the effects of a combined relationship and cash assistance.

We have lots of evidence that richer couples are less likely to divorce, of course. I don’t know that giving someone $9,000 would help with relationship quality, but I’m guessing it would at least help pay the rent or pay for some daycare.

It’s important to acknowledge that we’re not talking about research. The marriage promotion program is coming out of the welfare budget, not NIH or NSF. This study is a small part of it. Hundreds of millions of dollars have been spent on this, of which the studies account for a small amount. If this boondoggle continues, and they continue to study it, then they should include the cash-control group.

5. It seems to us that as a social scientist, you would want to ask “what have we learned about helping families from this study and from other research on couple relationship education?” We would suggest that we’ve learned that the earlier Building Strong Families program for unmarried low-income families had low attendance and no positive effects. A closer reading of those reports suggest that many of the unmarried partners were not in long-term relationships and were not doing very well at the outset. Perhaps it was a long-shot to offer some of them relationship help. We’ve also learned that the Strengthening Healthy Marriage program for married low-income families had some small but lasting effects on both self-reported and observed measures of their relationship quality (we think that the researchers learned something from the earlier study). And, notably, we’ve learned that there seemed to be some benefits for younger children when their parents took advantage of relationship strengthening behaviors.

We always learn something. See my comments above for why this is a stretch. I would be happy to see, and even pay for, research on what helps poor families. We already do some of that, through scientific agencies. My objection is not to the research, but to the program that it is studying, which takes money away from things we know are good.

Here is their last word — as good a defense as any for this program.

We know from many correlational studies that when parents are involved in unresolvable high level conflict, or are cold and withdrawn from each other, parenting is likely to be less effective, and their children fare less well in their cognitive, emotional, and social development. It was not some wild government idea that improving couple relationships could have benefits for children. Evidence in many studies and meta-analyses of studies of couple relationship interventions in middle-class families, and more recently for low-income families, have also been shown to produce benefits for the couples themselves — and for their kids. This was not a government program to force marriage on poor families. The participants were already married. It was a program that offered free help because maintaining good relationships is hard for couples at any level, but low-income folks have fewer financial resources to get all kinds of help that every family needs.

We are not suggesting that strengthening family relationships alone is a magic bullet for improving the lot of poor families. But, in our experience over the past many years, it gives the parents some tools for building more productive couple and parent-child relationships, which gives both the parents and their children more confidence and hope.

What we need to learn is how to do family relationship strengthening more effectively, and how to combine that activity with other approaches, now being tried in isolated silos of government, foundations, and private agencies, in order to make life better for parents and their kids.
In our view, trumpeting the failure of Supporting Healthy Marriage by focusing on a few of the negative findings doesn’t help move us toward that goal.


Filed under Research reports

This ‘Supporting Healthy Marriage,’ I do not think it means what you think it means

New results are in from the unrelenting efforts to redirect welfare spending to marriage promotion. By my unsophisticated calculations we’re more than $1 billion into this program, without a single, proven healthy marriage yet to show for it.

The latest report is a study of the Supporting Healthy Marriage program, in which half of 6,298 couples were offered an extensive relationship support and education program. Short version: Fail.

Photo by Marlin Keesley from Flickr Creative Commons

Photo by Marlin Keesley from Flickr Creative Commons

Supporting Healthy Marriage is a federal program called “the first large-scale, multisite, multiyear, rigorous test of marriage education programs for low-income married couples.” The program evaluation used eight locations, with married, low- or modest-income parents (or expectant couples) offered a year-long program. Those in the program group had a four- to five-month series of workshops, followed by educational and social events to reinforce the curriculum.

Longer than most marriage education services and based on structured curricula shown to be effective with middle-income couples, the workshops were designed to help couples enhance the quality of their relationships by teaching strategies for managing conflict, communicating effectively, increasing supportive behaviors, and building closeness and friendship. Workshops also wove in strategies for managing stressful circumstances commonly faced by lower-income families (such as job loss, financial stress, or housing instability), and they encouraged couples to build positive support networks in their communities.

This was a good program, with a good quality evaluation. To avoid selection biases, for example, the study included those who did not participate despite being offered the program. But participation rates were good:

According to program information data, on average, 83% of program group couples attended at least one workshop; 66% attended at least one supplemental activity; and 88% attended at least one meeting with their family support workers. Overall, program group couples participated in an average of 27 hours of services across the three components, including an average of 17 hours of curricula, nearly 6 hours of supplemental activities, and 4 hours of in-person family support meetings.

The couples had been together an average of 6 years; 82% had incomes below twice the poverty level. More than half thought their marriage was in trouble when they started.

But the treatment and control groups followed the exact same trajectory. At 12 months, 90% of both groups were still married or in a committed relationship, after 30 months it was 81.5% for both groups.


The study team also broke down the very diverse population, but could not find a race/ethnic or income group that showed noteworthy different results. A complete failure.

But wait. There were some “small but sustained” improvements in subjectively-measured psychological indicators. How small? For relationship quality, the effect of the program was .13 standard deviations, equivalent to moving 15% of the couples one point on a 7-point scale from “completely unhappy” to “completely happy.” So that’s something. Further, after 30 months, 43% of the program couples thought their marriage was “in trouble” (according to either partner) compared with 47% of the control group. That was an effect size of .09 standard deviations. So that’s something, too. Many other indicators showed no effect.

However, I discount even these small effects since it seems plausible that program participants just learned to say better things about their marriages. Without something beyond a purely subjective report — for example, domestic violence reports or kids’ test scores — I wouldn’t be convinced even if these results weren’t so weak.

What did this cost? Round numbers: $9,100 per couple, not including evaluation or start-up costs. That would be $29 million for half the 6,298 couples. The program staff and evaluators should have thanked the poor families that involuntarily gave up that money from the welfare budget in the service of the marriage-promotion agenda. We know that cash would have come in handy – so thanks, welfare!

The mild-mannered researchers, realizing (one can only hope) that their work on this boondoggle is coming to an end, conclude:

It is worthwhile considering whether this amount of money could be spent in ways that bring about more substantial effects on families and children.

For example, giving the poor couples $9,000.

Trail of program evaluation tears

We have seen results this bad before. The Building Strong Families (BSF) program, also thoroughly evaluated, was a complete bust as well:

Some of the people trying to bolster these programs — researchers, it must be said, who are supported by the programs — have produced almost comically bad research, such as this disaster of an analysis I reported on earlier.

Now it’s time to prepare ourselves for the rebuttals of the marriage promoters, who are by now quite used to responding to this kind of news.

  • We shouldn’t expect government programs to work. Just look at Head Start. Of course, lots of programs fail. And, specifically, some large studies have failed to show that kids whose parents were offered Head Start programs do better than those whose parents were not. But Head Start is offering a service to parents who want it, that most of them would buy on their own if it were not offered. Head Start might fail at lifting children out of poverty while succeeding at providing a valuable, need-based service to low-income families.
  • Rich people get marriage counseling, so why shouldn’t poor people? As you can imagine, I am all for giving poor people all the free goods and services they can carry. Just make it totally voluntary, don’t do it to change their behavior to fit your moral standards, and don’t pay for it by taking cash out of the pockets of single-parent families. I really am all in favor of marriage counseling for people who want it, but this is not the policy platform to get that done.
  • These small subjectively-measured benefits are actually very important, and were really the point anyway. No, the point was to promote marriage, from the welfare law itself (described here) to the Healthy Marriage Initiative. If the point was to make poor people happier Congress never would have gone for it.
  • We have to keep trying. We need more programs and more research. If you want to promote marriage, here’s a research plan: have a third group in the study — in addition to the program and control group — who get cash equivalent to the cost of the service. See how well the cash group does, because that’s the outcome you need to surpass to prove this policy a success.

Everyone loves marriage these days. But a lot of people like to think of promoting marriage as a way to reduce poverty, and with that they believe poor people are that way because they’re not married. That’s mostly backwards.


Filed under In the news, Research reports

Obama economic adviser on marriage and the gender gap

Two parts of this interview with Obama economic adviser Betsey Stevensen stood out to me. I’m just surprised to hear such straightforward social science from someone in such a powerful position.


This, on marriage and poverty:

What’s your reaction when you hear conservatives talk about marriage as a poverty reduction tool?

My research found that actually, if you want to increase marriage, you need to increase the minimum wage and strengthen the middle class so that people can enjoy the fruits of marriage from those more comfortable positions. I do think that conservatives don’t understand that the dynamics of marriage have changed in such a way that income supports marriage, rather than the marriage supporting having a higher income or supporting getting people out of poverty. There’s also the fact that they seem to really believe that if you push young people to marriage you can alleviate poverty, but then you see enormously high divorce rates, which actually makes things even worse because divorce is very expensive. The big differences in divorce come from, if you’re a 20-year-old high school dropout, you have [approximately] a 60% chance of divorcing within 10 years of marriage, but if you’re a 35-year-old with a college degree, you have [approximately] a 5% chance of divorcing. If what you think is that marriage is important for having a strong middle class, what you do is actually encourage people to wait before settling down.

The conservative view is, we should smush you together, then you’ll have more money, then there’ll be less tension. But actually, when you get two people making $7.25, there’s a lot of tension because you’re both still struggling. That tension leads to family conflict.

And this, on the gender wage gap:

Every time the president comes out and says, women should have equal pay for equal work, you have folks, including economists, come out and say, that’s a misleading number, that’s not for the same job, that’s year-round full-time wages, and a big part of it is women’s choices. What’s your response to that, and what’s a good way to understand these numbers?

When people come out and say that’s not a fair number, well, what really is a fair number? You brought up “women’s choices.” Well, some women’s choices come about because they’re being discriminated against. Some of women’s choices come because they experience sexism. Some of women’s choices come because they are disproportionately balancing the needs of work and family. Which of these choices should we consider legitimate choices, and which of them should we consider things that we have a societal obligation to try to mitigate, to alleviate some of these constraints so that they can make different choices? A lot of people will say things like, let’s control for occupational choices. But the research is showing us that women are choosing occupations which penalize them the least for taking time out of work.

If there was less discrimination, if there was more flexibility in work, you wouldn’t see women necessarily choosing the same occupations. So why should I take the wage gap holding occupation constant? If we change society, we reduce discrimination, we’re not going to hold occupational choice constant – women are going to choose different occupations.

I agree that the 77 cents on the dollar is not all due to discrimination. No one is trying to say that it is. But you have to point to some number in order for people to understand the facts. And what it represents is the fact that women on average are put in situations every day that for a variety of reasons mean they earn less. Much of what we need to do to close that gap is to change the constraints that women face. And there are things we haven’t tried.

I wouldn’t expect Obama to say things like this, but I’m impressed that someone in his near proximity would.

For more, follow the tags for marriage promotion and the gender inequality.


Filed under Uncategorized

Different divorce rates

Deadline crush, not getting out the posts I want to. So here instead is one thing I was planning to write about but didn’t really yet.

Photo by Dan Bluestein from Flickr Creative Commons

What’s the rate? Photo by Dan Bluestein from Flickr Creative Commons

I’ve written about divorce quite a bit on here, including on the mess of our official statistics. Now Sheela Kennedy and Steven Ruggles have a (paywalled) paper in the January issue of Demography called, “Breaking Up Is Hard to Count: The Rise of Divorce in the United States, 1980–2010.” Because of the paywall and the obscure academic journal, I thought I had some time to write about it, but it’s been reported on Wonkblog and and other places, so no point in waiting.

The headline is, “divorce is actually on the rise.” It’s risen when they age-standardize the trend, but it’s complicated: “Divorce rates have doubled over the past two decades among persons over age 35. Among the youngest couples, however, divorce rates are stable or declining.” The interpretation is not as simple as, “they have a better measure.”

Meanwhile, I was quoted in a Wall Street Journal story about some TV show, and I let slip my multiple-decrement lifetable version of the current divorce rate. This hasn’t been finished, much less peer-reveiwed, but I’m pretty confident about the basic result. I wrote to the reporter, who asked me for the divorce rate:

As for divorce rates, it’s hard to be definitive because there is no one answer. One answer is: In 2012 there were 19 divorces for every 1000 people who were married (my calculation from the 2012 ACS).

However, what most people want to know is what percentage of people who get married will end up getting divorced. There is no official estimate of this because it involves a guess about the future. We can estimate divorce like we estimate life expectancy — it’s not the actual prediction of how long people will live, it’s how long they would live on average if they lived through the risks of most recent year over and over again their whole lives. (Technically, it’s a projection, not a prediction.) Anyway, using that method, I estimate that about 50% of couples who married in 2012 would eventually divorce (with the rest of the marriages ending with someone’s death).

In her story, of course, that became, simply, “And about half of those who married in 2012 will eventually divorce.”

This method, which I got from this old Sam Preston paper, combines mortality rates and death rates to project how many people are lucky enough to die before divorcing at current rates. (Hence “multiple-decrement,” the demographers’ dry way of saying, “there are only two ways out of this.”) When he applied the method, with much cruder data from 1973, incidentally, he got a 43% divorce rate, which was much higher than the rates floating around at the time, and would have made big news in the blogosphere if there had been one.

More on this eventually.


Filed under In the news

What was I supposed to do, not report the results?

In case you haven’t been following the research on this, my understanding is that there is some evidence that women in several cultures are more likely to wear red-related colors when they are trying to look sexually attractive. We know that from the article “Women Use Red in Order to Attract Mates” in the journal Ethos. That’s all well and good, but to make it really interesting, we’d like to know that women are especially likely to do that when they are in the most fertile time in their menstrual cycle. Because, you know:

Photo by D. Gordon E. Robertson from Wikimedia Commons

Photo by D. Gordon E. Robertson from Wikimedia Commons

Unfortunately, that paper from Ethos did not find that red-wearing was associated with menstrual cycles. But, Beall and Tracy were able to find that link. Their conclusion:

Our results thus suggest that red and pink adornment in women is reliably associated with fertility and that female ovulation, long assumed to be hidden, is associated with a salient visual cue.

As Kim Weeden pointed out when I mentioned this on Twitter, Andrew Gelman used that paper as an example of how researchers have many opportunities to slice findings before settling on those that support their hypotheses.

Fortunately, Beall and Tracy set out to replicate their finding. Unfortunately, when they attempted to replicate the results, they were not successful. Fortunately, they realized it was because they were being confounded by the weather. As they have now reported, this is important because in warm weather female humans don’t need to resort to red because they can manage their attractiveness by reducing the amount of clothing they wear (and then, who cares what color it is?). Thus:

If the red-dress effect is driven by a desire to increase one’s sexual appeal, then it should emerge most reliably when peak-fertility women have few alternative options for accomplishing this goal (e.g., wearing minimal clothing). Results from re-analyses of our previously collected data and a new experiment support this account, by demonstrating that the link between fertility and red/pink dress emerges robustly in cold, but not warm, weather.

And here it is. Happy, Gelman?


Confirmatory classroom exercise

Since I am teaching love and romance in my family course this week, I thought we should add something to the conversation. I only did one exercise, and I am reporting the full results here. Nothing hidden, no tricky recodes, no other questions on the survey, no priming of the respondents (it was at the start of the lecture).

I have 80 students in the class, which means 53 were there in time for the exercise, 29 men and 24 women. I gave them this two-part question:


Because red and pink are both associated with fertility (see the baboon), I combined them in the analysis (but it works if you just use red, too). And these were the results:


The statistical test for the difference between date and family event for women is significant at the level of p<.035. This is not research, it’s just a classroom exercise (which means no IRB, no real publication). But if it were research, it would be consistent with the women-wear-reddish-to-attract-mates theory (although without the menstrual cycle question, its contribution would be limited).

Most sociologists might not go for this kind of stuff. Maybe it’s a slippery slope that leads to unattractive conclusions about gender inequality in the “natural” order. My perspective is that I don’t care. Of course this is not really evidence that evolution determines what American (or, in the case of the Ethos paper, Slovak) students wear on dates. But it doesn’t refute the theory, either.

More importantly, I am confident that we could, if desired, through concentrated social engineering, eliminate the practice of women wearing reddish on dates if we thought it was harmful — just as we have (almost) engineered away a lot of harmful behaviors that emerged from the primordial past, such as random murder, cannibalism, and hotmail. After all, they did it in China:


Sorry, wrong picture:


For previous posts in the series, follow the color tag.




Filed under Uncategorized