Category Archives: Me @ work

Marriage matters

A critical post describes me this way: “progressives, like sociologist Philip Cohen, who seek to minimize or deny the importance of family structure.” It’s not an accurate way to frame this debate. The question is much more about goals. If your goal is reducing child poverty, for example, we know how to do that, more or less — something like giving their parents money. But the ostensible goal of the marriage promotion people is increasing marriage because they believe there are things that come from marriage that are both good and not producible through any other means. I say ostensible because, given the obvious futility of all marriage promotion policies, one must suspect other motives as well. (Some of those motives are religious, of course, but I set those aside for now).

No doubt there are some things that only marriage can truly provide to children under present conditions. But that doesn’t imply a particular policy goal, much less a specific approach. I have a few problems with assuming we need more marriage.

What’s wrong with that?

Demographic barriers. First, not everyone can be married, or have married parents, no matter what we do with policy. So we need to figure out how to minimize the harms associated with not being married anyway. The steeper the penalty for non-marriage (what most people call the benefits of marriage) the less we are responding to the needs of people who suffer not only the losses that flow from non-marriage, but also the deficits that cause non-marriage.

This is one thing that’s so powerful about Chinese history for me — as told by Lee and Wang — in which marriage was absolutely essential for social success and also mathematically denied to a large portion of the population (because of female infanticide and polygyny). In that case you’d find a very strong benefit to marriage — in a system of marriage that exacted tremendous costs. If we have a strong marriage benefit (that is, non-marriage penalty) we have the same situation as feudal China. The demography of inequality means lots of people can’t be married. Here are the sex ratios for non-Hispanic Blacks and Whites by age (this is the whole U.S. population, not just single people, or people who aren’t in prison):

black white sex ratio.xlsx

This is just an example of a demographic dead end for marriage promoters. What’s the plan? The plan is that some large portion of poor and/or Black women — because of circumstances beyond their control — should not have children. (In their heart of hearts, they believe they shouldn’t have sex at all, but that’s not spoken in polite circles.) Life is hard. Try going to church instead.

Marriage is bad, too. Second, marriage also causes harms. This should be obvious, but I guess in a society where people are so ready to believe that racial discrimination isn’t a problem anymore or that women have actually overturned patriarchy, we have to keep pointing that out. The married-couple family is still an extremely dangerous place for a lot of people — mostly women and children — whose abuse and exploitation is made more difficult to root out the more we make marriage an exalted status and indicator of social competence. It’s not an accident that marital rape was protected by law for all those centuries — the men who did it were (are) engaged in the successful performance of admirable patriarchal roles.

Outside of violent rape and abuse, there are also lots of people who aren’t happy in their marriages, or who don’t want to be married at all. In 1960, 94% of women were married before reaching age 40. Now that number is 78%. You know if 94% of people were married a lot of them weren’t happy. So, what’s the right number, marriage promoters? Is 78% participation in your favorite institution too low? To the people who aren’t happy in marriage, or don’t want to be married, the marriage promoters offer two choices: be celibate, or shame on you. (Actually, there is a third choice: date night!) For people who want to have children, too, their offer is to punish your children by withholding a child tax credit.

Signification. Third, the more we support marriage through politics, policy, and “the culture,” the more we increase its value as a conformity signifier, the more we constrain people’s family options — and that exacerbates the non-marriage penalty, which starts to look like other penalties people dole out for non-conformity, like gender non-conformity for example, or employer discrimination in favor of married men.

Bologna. Fourth — and less important, given the first three — of course there is no evidence that policy can increase marriage in any meaningful way, making the whole exercise deeply cynical. It’s largely a conversation between people validating the importance of marriage in their own social (and political and religious) circles, rather than a serious attempt to reverse the course of modern demographic evolution.

Marriage, advantage

I have been dealing with the paradox of marriage advantages for many years. Here’s a little greatest-hits history on this argument, with links. (I’m assembling this material for a book project anyway).

In a 2001 review of Linda Waite et al.’s edited volume, The Ties That Bind, I wrote, in reference to a chapter by Paula England, who influenced me fundamentally:

England paraphrases a quip about another tie that binds — capitalism — when she says, “The only thing worse than being dominated by a husband is not being dominated by a husband.” Given that women in the United States still have children, and are still largely responsible for their care, and given that men still dominate economically, married mothers and their children are advantaged.

So, what to do? Here’s me on Huffington Post in 2009:

…when a condition … yields benefits compared to being in some other condition — that also means it contributes to inequality. That’s because not everyone gets to experience it. If children of married couples are more likely to finish high school than those who grow up with single mothers, for example, then there is inequality between those two groups. One policy approach to that inequality is to make the condition more common — for example, encourage or coerce people to marry (or discourage people from having children when single). Another approach is to improve outcomes for people in the disadvantaged group. For example, because resource scarcity is a big part of the problem for single-parent families, we could support a public school system that educates all children effectively, or provide income support to poor families.

I carried on that argument in 2011, writing:

It’s obvious empirically that adults and children in married-couple families, on average, are doing better on many measures than those not in such families. The logical problem is when people conclude from this pattern that the obvious response is to “strengthen marriage and family life.” But, why not try to reduce that disparity instead? This is the logical equivalent of the Republican mantra that “We don’t have a revenue problem in Washington; we have a spending problem.” That’s only true if you’re doing one-handed math.

I made the case more formally in this 2014 chapter, responding to a paper by Sara McLanahan:

Cross-sectional comparisons show that children of married parents are less likely to suffer material deprivation. To reduce hardships for children, therefore, some analysts advocate policies that would increase marriage rates. I argue that alternative approaches offer more chance of success: increasing education levels and reducing the penalty for single parenthood.

And I concluded:

The rise of women’s independence, along with the decline in marriage and fertility, are interrelated parts of modern social development. And the overall consequence of these trends must be deemed positive – as life expectancies have increased, absolute poverty has decreased, and gender inequality has receded. The delay in age at marriage and the extension of divorce rights have no doubt prevented or ended many unhappy or unsafe marriages, even as they have carried risks. But the advocates for marriage offer no attempt to specify the ideal marriage rate. How are we to know that the decline in marriage has gone too far? The unwavering advocacy for more marriage, in the face of its continued inefficacy and impracticality, dissolves into ideology and distracts from the important challenges we face in attempting to improve the quality of life for poor families and their children.

Finally, I also addressed the contribution of marriage trends to economic inequality:

Falling marriage does contribute to rising inequality in the USA, because of how it’s manifesting: increasing selectivity in marriage, so that richer people are getting and staying married more; and increasing social class endogamy, so that there are more two-high-income families lording over more one-low-income families. And all of that is exacerbated by widening underlying inequality, with high-end incomes pulling away from low-end incomes, relatively unchecked by income redistribution.

One obvious solution is to take money away from married high-income people and give it to single low-income people. With all the benefits that married people get — many of them through no special effort of their own, but rather as a result of their social status at birth, race, health, good looks, legal perks, or lucky breaks — it seems reasonable to tax marriage, like a windfall profits tax, or an inheritance tax, or a progressive income tax. But, if you’re squeamish about taxing something “good” like marriage, then just taxing wealth a little more would accomplish much the same thing. This elegant solution would decrease inequality, increase well-being for poor people, and equalize life chances for children (who are the future, I believe). In other words, it’s out of the question.

Far from minimizing or denying the importance of family structure, I have written about it extensively — and argued we should find ways to reduce it.  On the other hand, marriage promotion as an approach to poverty and inequality is both completely ineffective at achieving its goals and harmful in other ways.


Filed under Me @ work

I overspoke myself on Twitter

Possibly not the only time.

A blog called Random Critical Analysis (RCA) has posted, “On Philip Cohen’s knee-jerk response to Chetty’s “causal mobility” data and its association with single-motherhood.” I now must admit that I overspoke myself on Twitter.

But I think the blog post I wrote holds up OK. I complained in the post that the now-famous Chetty et al. analysis of intergenerational mobility had mishandled race, leading to people like David Leonhardt (and rightward from there) to conclude that the big story of hampered social mobility is family structure. It’s part of the overall pattern of polite society embracing the issue of economic inequality but also using that as a foil to avoid the issue of race inequality.

Brad Wilcox has seized on the Chetty analysis, repeating ad nauseum the quote that single parenthood is the “single strongest correlate of upward mobility.” My beef was, and is, that the analysis that was based on — which used the rate of single parenthood at the labor market level to predict intergenerational mobility — did not control for the racial composition of the labor market. That’s an obvious problem when your map of mobility looks like this:


When your analysis is ecological, that is, based only on aggregate characteristics, you have to be very cautious about drawing conclusions. It’s especially dicey in the Chetty case because the basic data, from tax, returns, includes family structure (because of parents’ marital status) but not race (which doesn’t go on your tax form). And that’s even more dicey because we know that at the individual level single parenthood is definitely not the “single strongest correlate of upward mobility.” I’ve been writing about this for years (follow the single-mother tag), but this figure from 2012 sums it up nicely (details in the old post):

You just have to keep that in perspective when you jump to an aggregate-level analysis. The difference between averages in Atlanta versus Salt Lake City — important as it is — is never going to be as big as the difference between a rich family and a poor family. Social parents’ class matters much more for determining children’s social class than does family structure.

Anyway, RCA is reworking my very simple analysis showing the effect of single motherhood rates was reduced by two-thirds when a single control for racial composition (percent Black) was added. That’s making the obvious point that, because single parenthood and percent Black in the local area are so strongly correlated, if you don’t take percent Black into account it looks like single parenthood has a huge, independent effect — which incorporates the effects of racism or other community factors associated with historical race composition. The new RCA post goes much further in the analysis, and concludes:

It ought to be pretty clear by now single-motherhood is capturing something quite powerful and that, contrary to Cohen’s strong assertions, it is not well explained by race.  If anything, single-motherhood mediates the black association much better than the reverse.

I’m not persuaded by the conclusion; you can evaluate it yourself. But the premise of the RCA post is actually not my blog post, but my tweets. As time went by I apparently became frustrated at the continued repetitions of the single mother thing by people who were ignoring my very clever post, and with the carelessness that distance allows I overstated my own claim, so I tweeted this,

The table and the highlighting are mine. What I should have paid attention to was my own next sentence after the underlined part: “That’s not an analysis, it’s just an argument for keeping percent Black in the more complex models.” I didn’t do a serious analysis — I just did enough to prove the point that racial composition should be in the model. Without that, you shouldn’t run around saying single parenthood is the most important factor. (RCA also believes I shouldn’t have said in the post that “Percent Black statistically explains the relationship between single motherhood and intergenerational immobility.”  I think “explains” is defensible, in that the effect is no longer statistically distinguishable from zero at the conventional level, but it’s clearly not the same as proving there is no effect, so I’ll take the criticism, too.)

I actually first did the little analysis in an earlier post, debunking a univariate analysis by Scott Winship and Donald Schneider. In that case I concluded: “This [my analysis] is not a rigorous examination of the cause of intergenerational immobility. It is just debunking one bivariate story that is too easily picked up by the forces of bad.” That seems about right.

Anyway, in conclusion, it was incorrect based on what I did for me to tweet, “the single mother effect in Chetty is all in the % Black effect.” I should just say single parenthood hasn’t been proven to matter as much as its partisans say it has. Even if it’s less effective in a tweet. This is a common frustration, that it takes more work to debunk something than to bunk it in the first place. But that’s not a good excuse.

Finally, I’m grateful that what I write matters enough that someone would go to the trouble of testing my claims to hold me accountable.


Filed under Me @ work

Shine a light on journal self-citation inflation

Photo by pnc.

Photo by pnc.

Note: Welcome Inside Higher Ed readers. I’d be happy to hear accounts from disciplines other than sociology. Email me at

In my post on peer review the other day, I mentioned that a journal editor made this request — before she agreed to send the paper out for review:

“If possible, either in this section or later in the Introduction, note how your work builds on other studies published in our journal.”

A large survey on “coercive citation” practices, published in Science in 2012 (paywalled; bootlegged PDF) found that 20% of researchers had, in the previous five years, “received a request from an editor to add more citations
from the editor’s journal for reasons that were not based on content.” The survey, which was sent to email lists for academic associations, including the American Sociological Association, found sociologists and psychologists were less likely to report having experienced this practice than were economists and those in business-related disciplines.

The journal I named, Sex Roles, is high on the list of those most frequently mentioned — cited by four respondents, more than any journal outside of business, marketing, or economics. But there are a lot of other journals you know on the list.

Although I made the assumption that the Sex Roles editor was trying to increase the impact factor — the citation rate — for her journal, one could defend this practice as being motivated by other interests (I’ll leave that to you). It also seems likely that some requests are open to interpretation — for example, mixing in citations from different journals, or offering specific reasons for including particular citations.

Tell me about it

To look into this a little more, I’m asking you to send me requests for journal self-citation that you have received. I’ll keep them confidential, but if I get enough to make it interesting, I will post: (1) journal name, (2) the type of request, (3) the date (month and year), and (4) the stage in the publication process. Feel free to include extenuating details or other information you would like to share, and let me know if you want it disclosed. I assume most of you are sociologists, but I’ll include items from any discipline.

To be included on the list, I’ll need to see copies of the letter or email you received. I will not disclose your identity or information about you, or the specific article under review. I won’t use quotes that might identify the author or article under review.

I will also send the list to the current editors of journals named and give them an opportunity to respond.

My contact information is here.

Maybe there’s not enough here to go on, but if there is, I think shining a light on it would be a good thing, and might deter the practice in the future.


Filed under Me @ work

Our broken peer review system, in one saga

When at last Odysseus returns.

When at last Odysseus returns.

Everybody’s got a story. This is the story of publishing a peer-reviewed journal article called, “The Widening Gender Gap in Opposition to Pornography, 1975–2012.” The paper has now been published, and is available here in preprint, or here if you’re on a campus that subscribes to Social Currents through Sage.

Lucia Lykke, a graduate student in our program, and I began this project in the fall of 2012. We came up with the idea together. I did the coding and she wrote the text. Over the course of two years we sent the paper to four journals – once to Gender & Society, four times to Sex Roles, once to Social Forces, and twice to Social Currents, which finally accepted it in July 2015 and published it online on September 21.*

This story illustrates some endemic problems with our system of scholarly communication, both generally and in the discipline of sociology specifically. I discuss the problems after the story.


The gist of our paper is this: Opposition to pornography has declined in the U.S. since 1975, but faster for men than for women. As a result, the gender gap in opposition – with women more likely to oppose pornography – has widened.

That’s the finding. Our interpretation – which is independent of the veracity of our finding – is that opposition has declined as porn became more ubiquitous, but that women have been slower to drop their opposition because at the same time mainstream porn has become more violent and degrading to women. We see all this reflecting two trends: pornographication (more things in popular culture becoming more pornographic) and post-feminism (less acceptance of speaking up against the sexist nature of popular media, including porn). We could be wrong in our interpretation, and there is no way to test it, but the empirical analysis is pretty straightforward and we should accept it as a description of the trend in attitudes toward pornography. And for doing that empirical work we beg permission to tell you our interpretation.

The analysis is possible because the General Social Survey has, since 1975, asked a large sample of U.S. adults this question about every two years:

Which of these statements comes closest to your feelings about pornography laws: 1. There should be laws against the distribution of pornography whatever the age. 2. There should be laws against the distribution of pornography to persons under 18. 3. There should be no laws forbidding the distribution of pornography.

We tracked the rate at which people selected the first choice versus the others. It’s not very complicated (although we tried it half a dozen other ways, of course). Also of course it’s not perfect – it’s not a great question for today’s social reality, but it’s the only thing like it asked over such a long period. This is what’s great and what’s limiting about the General Social Survey. So, let’s agree to collect better data, and also use this. There, was that so hard?

Here is supporting detail on our particular saga. (We have left the typos from reviewers intact, because it makes us look smarter than they are. And these are selective excerpts to make various points – there was a lot, lot, more.)

Before and after

Just to be clear what the world gained by 13 reviews and two years of waiting, you can compare the abstract at the beginning to the one at the end. This was the original abstract:

In the last several decades pornography in the U.S. has become more mainstream, more accessible, and more phallocentric and degrading to women. Yet research has not addressed how opposition to pornography has changed over the past several decades. Here, we examine opposition to pornography and gender differences in anti-pornography attitudes, using the 1975-2012 General Social Survey. Our findings show that both men’s and women’s opposition to pornography have decreased significantly over the past 40 years, but men’s opposition has declined faster and women remain more opposed to pornography. This is consistent with both the growing normative nature of pornography consumption for men and its increasingly degrading content. We situate these trends within a cultural climate in which women are caught between postfeminism and pornographication – between cultural messages that signal the social acceptability of pornography and compel women’s acquiescence, on the one hand, and the increased presence of pornography many women consider offensive and harmful on the other.

And this was the abstract we ended up with:

In the last several decades pornography in the U.S. has become more mainstream, more accessible, more phallocentric and more degrading to women. Further, consumption of pornography remains a major difference in the sexual experiences of men and women. Yet research has not addressed how opposition to pornography has changed over the this period, despite shifts in the accessibility and visibility of pornography as well as new cultural and legal issues presented by the advent of Internet pornography. We examine gender differences in opposition to pornography from 1975 to 2012, measured by support for legal censorship of pornography in the General Social Survey. Results show that both men’s and women’s opposition to pornography have decreased significantly over the past 40 years, suggesting a cultural shift toward “pornographication” affecting attitudes. However, women remain more opposed to pornography than men, and men’s opposition has declined faster, so the gender gap in opposition to pornography has widened, indicating further divergence of men’s and women’s sexual attitudes over time. This is consistent with the increasingly normative nature of pornography consumption for men, increases over time in men’s actual consumption of pornography, and its increasingly degrading depiction of women.

The regression model we started with in 2013 had logistic regression coefficients showing a decline of .012 per year in the log odds of women favoring laws against the distribution of pornography, versus .022 for men. (That is, the decline has been almost twice as fast for men.) After all we went through with the other variables, we ended up with .012 and .023.


August 6, 2013: Submitted to Gender & Society

September 23, 2013: Rejected, with four reviews

Reviewer A was concerned about framing, and about the dependent variable.

if one takes this more complex and nuanced definition of postfeminism into account, the theoretical frame of does not work well for the paper … I also thought that the authors could have gone further in discussing broader cultural changes in sexuality in the media, especially the increasing sexualization and pornification in advertising and the media. …

an analysis of a GSS question concerning laws regarding the restriction of pornography, seem limited. In particular, that GSS question does not seem to get at the historical changes that have occurred in pornography distribution and consumption given its widespread internet usage.

Reviewer B was all about framing:

[I] appreciate your analysis of anti-pornography research and the effects of post-feminism on attitudes towards pornography … [but] I think the literature review needs to spend at least some time outlining feminist pro-pornography arguments. …

doesn’t it make sense to incorporate a discussion of the history of pornography regulation since the 1970s in the U.S? [… and …] While you bring up race in the analysis of your data, the literature review is surprisingly devoid of anything having to do with pornographic representations of gender and race.

Reviewer C thought we should have included a content analysis of pornography over time – done a different study, that is — and framed it differently:

Pornography needs to be defined … Cost, images and rejection of feminist view would clearly support a content analysis on pornography … The provided discussion of pornographication seems to more support the use of images and actual study of pornography, more so than people’s attitudes toward it … more justification to the existing literature needs to be added … Some legal gender studies should be included here … Gender is not one sided and the author should consider adding some agency to [men’s] role in the study and discussion.

Reviewer D concluded that the data weren’t good enough to support our interpretation:

The author, however, does not empirically demonstrate that the found decline in opposition is the result of either postfeminism or pornographication. … The General Social Survey is convenient, easy to access, and quick to run. This, however, does not necessarily make for good empirical evidence. … If the author wanted to investigate postfeminism and pornagraphication and the relationship to pornography, a much more nuanced empirical study would have needed to have been designed.

In a world with limited space for publishing research – which is not our world – this would be a good reason to reject the article.

October 7, 2013 (approximate): Submitted to Sex Roles

October 9, 2013: Returned by the editor

The editor, Irene Frieze, returned the paper almost immediately, saying: “major revisions are needed before we can move ahead in the review process.”

Some of what she asked for reflects the competitive climate of contemporary academic journals. For example, she asked us to pad the journal’s citation count: “If possible, either in this section or later in the Introduction, note how your work builds on other studies published in our journal.”

And she tried to make the journal seem more international:

Explain why your study is important to readers from many countries with a sentence or two. … Note what country each empirical study you cite was done in and explain how any cited studies done in other countries are relevant in understanding your sample.

She also asked for what appear to be standard requirements for the journal:

Add demographic information about the sample and explain more about how they were recruited. Add a table showing the demographic characteristics of the women as compared to the men in the sample in different time periods. … Add correlations computed separately for women and men as well.

And, the dreaded memo requirement: “Assuming you do wish to submit a revision, I would need a revised manuscript and a detailed list outlining the changes you have made in response to these comments.”

November 9, 2013: Resubmitted to Sex Roles, first revision

February 17, 2014: Revise and resubmit, based on one review (“major revisions”)

The reviewer had trouble with our statistical presentation:

I see that on Table 2, the difference between the women’s and men’s regression effect for year shows both women’s and men’s significant (-.012 and -.022). This suggests that for both female and male respondents the year is significant, but it doesn’t show statistically that men’s decline in opposition is steeper than is women’s. Where is the statistic showing a significant difference in slope? [The table had a superscript b next to the men’s coefficient, with the note, “Gender difference significant at p<.05.” Although we didn’t provide the details, that test came from a separate, “fully-interacted” model in which every variable is allowed to have a separate effect by gender.]

This reviewer – who stuck with this complaint for three rounds – also had trouble with the smallness of the coefficients:

Although the coefficient is twice as large for year among men than among women, it’s a very small percentage. With such a large sample size, almost anything will be significant. I’d like to see an effect size statistic.

She might have been confused because the variable here is “year” – a continuous variable ranging from 0 in 1975 to 37 in 2012, so the coefficient reflects the size of the average one-year change, which makes it look “very small.”

A common problem for authors responding to reviewers is the simultaneous demands for less and more. Sometimes that’s good – a healthy revision process. Here is a funny example of that: “There seems to be a much longer introduction than is needed for the findings, especially since what would be interesting to me is omitted.”

However she grasped the concise nature of the findings, which she somehow took as a weakness:

I would like to see how each of these control variables interacts with the changes over years. I believe that analysis is possible using time series analyses. The reader is left with only a few main conclusions: both men and women indicate less opposition over time to pornography, and that men’s opposition declines more than female’s, and men show less opposition to pornography control overall.

Exactly. Oh well.

May 17, 2014: Resubmitted to Sex Roles, second revision

July 8, 2014: Revise and resubmit, with two reviews

The editor now told us: “We were able to find a second reviewer, this time. We won’t continue to add new reviewers for additional drafts.” (This promise, sadly, did not hold.)

The dependent variable – that three-response question about laws regulating pornography – caused continuing consternation. The editor wrote:

none of us feels that the combining of the three categories of responses for the pornography acceptance variable is appropriate. You either need to omit one of the 3 categories from the analysis, or do something like a discriminant analysis to look at differences in those responding to each of the three categories.

And then this bad signal that the editor and reviewers did not understand the basic structure of the analysis:

Another issue that all of us agree on is that you have failed to provide statistical evidence supporting your assertion of evidence of a linear trend in support over time. Either do a real trend analysis, for women and men separately, or compare the data over several specific years using something like ANOVA by year and gender. This would also allow you to see if these is really the interaction you assert is present.

As you can see in the final paper – which was the case in this revision as well – we did a “real trend analysis, for women and men separately.”

We tried to make this as clear as possible, writing in the paper:

We use logistic regression models to test for differences on this measure between men and women across the 23 administrations of the GSS since 1975. We test time effects with a continuous variable for year, which ranges from 0 in 1975 to 37 in 2012. This coding allows for an intuitive interpretation of the intercept and produces coefficients equal to the predicted change in the odds of opposing pornography associated with a one year change in the independent variable (non-linear specifications did not improve the model fit). … The first model combines men and women, while models 2 and 3 analyze men and women separately, after tests showed differences in the coefficients by gender on six of the variables (marked with superscript ‘b’). … Comparison of Model 2 and Model 3 confirms that the decline in opposition to pornography has been more pronounced for men than for women, as the coefficient for the year variable is almost twice as large.

We thought that Reviewer 1, back from the previous round, was doubling down on misunderstanding what we did, and the editor thought this as well. The reviewer wrote: “I don’t agree that the years need collapsing in the analyses. I believe it is better to see the linear trend. Also, I don’t like to see data left out, in this case data from the individual years.”

In fact, we found out in the next round of reviews we found the she meant this is a disagreement with the editor! (“The authors misread my statement about collapsing the years. I was disagreeing with the editor who suggested collapsing the years. I did not suggest myself that the years should be collapsed. I agree that the years should not be collapsed. It’s not me who misread the paper, it’s the authors who misread my statement.”)

That said, she still did not grasp the analysis:

You state that ‘This coding allows for an intuitive interpretation of the intercept and produces coefficients equal to the predicted change in the odds of opposing pornography associated with a one year change in the indepenjdent variable.’ In the results section, please describe how your data fit an ‘intuitive’ interpretation and how the coefficients that are produced explain the one year change. There is a disconnect for me from this statement and the description of the data.

And she added:

Please carefully describe the statistical analysis and statistical findings that describe the difference between the declines in opposition for women vs. men. Is the beta for gender .78 and for year -.02, and how did you test for the difference in betas of -.01 vs. -.02? Mention the test you used to assess this. This doesn’t seem like much of a difference in slope. That one is twice as large as the other is fairly meaningless when it is .01 vs. .02.

And added again later: “P. 22, agvain when you say a coefficient for the year variables is “amost twice as large,” you are talking about .01 vs .02.”


The editor and Reviewer 1 had a long-running dispute about how to handle all of our control variables. The editor was sticking to the policy that we needed a table showing complete correlations of all variables separately by gender. And a discussion of every variable, with references, justifying its inclusion. The editor said in the first round:

You also need to explain each of the control variables you include in your regressions in the Introduction. Add at least a sentence for each variable explaining why it is important to the issues you are testing.

In response, we included a long section beginning with, “Various social and demographic characteristics are associated with pornography use and attitudes toward pornography, and we account for these characteristics in our empirical analysis below.”

But then Reviewer 1 said of that passage: “Much of the material in “Attitudes Toward Pornography” is not relevant. … Gender and gender differences are what you are studying.”

And in response to our gigantic correlation table of all variables separately by gender, Reviewer wrote: “I … strongly recommend deletion of Table 3. This is not a study of the correlates of attitudes toward pornography, and the intercorrelations of all the control variables are outside the range of your focus.”

Never mind.

Reviewer 2, the new reviewer, had some reasonable questions and suggestions. For example, s/he recommended analyzing the outcome with a multinomial logistic regresstion, which we did but it didn’t matter; and controlling for pornography consumption (“watched an x-rated movie in the past year”), which we did and it didn’t matter (in fact, basically none of the control variables affect the basic story much, but reviewers have a hard time believing this). S/he also had lots of objections to how we characterized various feminist authors and terms in the framing, and really didn’t like “pornographication” as a term, listing as a “major” objection:

the term ‘pornographication’ is problematic and should be removed from the paper in favor of a more academic description of increased access to sexualized media.

September 10, 2014 (approximate): Resubmitted to Sex Roles, third revision

October 11, 2014: Revise and resubmit, with one review

The editor now informed us that one reviewer just recommended rejecting the paper because we didn’t address her concerns, while the other called for “major revisions.”

Given this type of feedback, I would normally reject a paper already in its third revision. However, I would like to offer one more opportunity for you to make the requested changes. If you do resubmit, I may seek new reviewers and essentially begin the review process anew, unless it is clear that my earlier concerns are fully addressed.

Despite three drafts and as many memos, the editor still did not seem to understand that our outcome variable was a single question with three options. She wrote:

One of my basic requests has been that you consider the question about exposure of pornography to those under 18 as a separate dependent variable, or omit this entirely from the study. Conceptually, I feel this is quite different from the other two survey items and cannot be combined with them. This will require major changes in the analysis and rationale for predictions relating to each of these measures.

The reviewer, however, disagreed, voicing approval for our choice. The editor clarified, “If my requests conflict with those of the reviewer, it is my requests you need to follow, not those of the reviewer.”

They had no trouble agreeing, however, that they did not understand the linear time trend we were testing: “As the reviewer explains, we do need a clearer discussion of how the linear trend is being tested.”

Reviewer 1 wrote:

Regarding the analysis of the time trend, although the authors state [in the memo] that the starring of the coefficients on Table 4 demonstrate a significant linear trend, it was not apparent to the editor and reviewers. As one of the main points of the study, it should be made very obvious that there is a significant linear trend via statistics. If this means being more explicit in the text of the results section, it would be important to do. If there’s this much confusion, the statistical analysis needs clarification.

You can look at the table in the final publication for yourself to see if this remains unclear. And then the reviewer added:

As I previously mentioned, though significant, a change of -.02 vs .-.01 is not substantial. Thus, the authors should refrain from concluding one is twice as large as the other.

We decided to take our business elsewhere rather than submit another revision.

November 4, 2014: Submitted to Social Forces

December 29, 2014: Rejected, with two reviews

Reviewer 1 only had concerns about framing, such as, “expand their discussion of the broader cultural changes in sexuality in the culture,” and discuss “changes in gay and lesbian identities and visibility during this period.”

Reviewer 2 simply thought we couldn’t answer the questions we posed with the data we had:

The paper is motivated by a largely assumed cultural ‘pornographication’ process linked to post-feminism. Neither concept seems well-suited to explain public opinion formation or change, and greater specificity about these concepts would likely outstrip the operational capacity of the GSS to model how gender and sexuality attitudes may influence shifts in beliefs about pornography.

There were some other technical issues about specific variables that aren’t very important. Again, this is very reasonable basis for making the ridiculous judgment forced by the system of publishing in the limited pages of a print journal.

January 16, 2015: Submitted to Social Currents

April 9, 2015: Revise and resubmit, based on three reviews

The editors, Toni Calasanti and Vincent Roscigno, wrote:

While stated differently in each case, the overriding sentiment across the reviewers is that the paper needs better framing. … the potential contribution of this study is not realized because the theoretical framework is lacking, limiting your ability to discuss the implications of your findings.

Reviewer 1 wanted the “post-feminism” discussion put back in the front: “It’s not until the conclusion of the manuscript that we learn about a potential contribution to ‘postfeminism’ and current work there.”

Reviewer 1 also attempted to lead us into a common trap. S/he wrote:

The hypotheses don’t necessarily derive from a particular theory in sociology or test a specific argument about gender, public opinion theories, and pornography per se. Rather, the project is descriptive (divergence of male/female support for legal control, rate of change over time, etc.). That isn’t fatal. But a project that makes a more direct connection to advancing current theoretical work in feminism and sexuality studies, or current theorizing about the importance of public opinion and values about pornography, would strengthen the overall contribution of this research.

Making the paper more theoretical is not a bad suggestion, but in this context – since the data are so limited – it’s a sure setup for a future reviewer to complain that you have asked questions you can’t sufficiently answer with your data.

The three reviewers’ other concerns by this point were quite familiar to us. For example, “perhaps a line or two to strengthen the validity of measure could be added based on some of the studies cited.” And a worry about about collapsing the dependent variable into two categories. And the need to acknowledge debates within feminism about the meaning of “pornographication.” We dutifully beefed up, clarified, and strengthened. And wrote a memo.

May 20, 2015: Resubmitted to Social Currents, first revision

July 18, 2015: Accepted


Some of the problems apparent in this story are common to sociology, some are more general.

Sociologists care way too much about framing. Most (or all) of the reviewers were sociologists, and most of what they suggested, complained about, or objected was about the way the paper was “framed,” that is, how we establish the importance of the question and interpret the results. Of course framing is important – it’s why you’re asking your question, and why readers should care (see Mark Granovetter’s note on the rejected version of “the Strength of Weak Ties”). But it takes on elevated importance when we’re scrapping over limited slots in academic journals, so that to get published you have to successfully “frame” your paper as more important than some other poor slob’s.

The journal system gets in the way. When journals reject you they report the low percentage of papers they accept. This is supposed to make the rejected authors feel better, but it also shows the gross inefficiency of the system: why should you bounce from journal to journal with low acceptance rates – in our case, asking our colleagues to write 13 reviews – instead of being vetted once by a centralized system with reviewers who work to a common standard? The answer is because that’s the way they did it in the Dark Ages, when physically printing research papers at high cost was the only way of distributing scholarly output.

The system is slow. As a result of these and other systemic problems, we do a terrible job of advancing knowledge. From the time of our first submission to the publication date was 776 days. For 281 of those days it was in our hands, but for the other 495 days it was in the hands of editors, reviewers, and the publisher. Despite responding to 13 reviews, with a lot of tinkering, the basic result did not change from our first submission in August 2013 to our last submission in May 2015. The new knowledge was all created two years before it was published.

The system is arbitrary I don’t want to make Social Currents look bad here, with the implication that they are a lower quality journal because they published something rejected by three journals before. After all, Granovetter’s paper was rejected by American Sociological Review before getting 35,000 citations as an American Journal of Sociology paper. I also like the example of Liana Sayer and Suzanne Bianchi’s paper on economic independence and divorce, which was rejected by the Journal of Marriage and Family, the flagship journal of the National Council on Family Relations (NCFR), before promptly winning NCFR’s best-paper award after it was published in the Journal of Family Issues. That is, one small group of reviewers deemed it unpublishable in a top journal, and the next declared it the best article of the year. That’s a very wide spread. The arbitrariness of the review system we have now creates cases like this – and who knows how many others. It’s not a systemic problem that Sex Roles has a reviewer that won’t let you say .02 is twice as large as .01. The problem is that could happen anywhere – and cost people their careers – at the same time that bad stuff gets through for arbitrary (or pernicious) reasons. There is too much noise in the current peer-review system to trust it for quality control.


Consider an alternative system, for example, in which the paper – having passed a very low bar of basic quality – had been published after the first set of reviews and then subjected to post-publication review and discussion in the field. Another alternative is publishing it before any formal review process, and allowing post-publication review to do the whole vetting process.

Models exist. Sociology doesn’t have a central working paper system, but there are smaller systems. In my neck of the woods, the California Center for Population Research has a working paper archive, which houses papers from six population centers. Math types have arXiv, which has more than a million papers, with each new one “reviewed by expert moderators to verify that they are topical and refereeable scientific contributions that follow accepted standards of scholarly communication.” They also use a system of member endorsement to cut down on junk submissions. If papers are subsequently published the arXiv version is updated to link to the published version. Sociology should make something like this.

Another step in the right direction is rapid-response, open-access peer-review, with quick up-or-down decisions. In sociology this includes Sociological Science, run by an independent team and supported by author fees (often paid by university libraries or grants); and Socious, run by the American Sociological Association and subsidized by the for-profit publisher Sage in an attempt to pacify open-access advocates. These work more or less like PLOS One, which “accepts scientifically rigorous research, regardless of novelty.”

I’m happy to publish in such outlets, but many of us worry about the career implications for our students who risk having their CVs seen as sketchy by old-fashioned types. We need them to be institutionalized.

In the meantime, those of us in position to conduct peer review can do our part to be better reviewers (see this excellent advice). And we can make explicit decisions about which journals we will review for. The system runs off our discretionary contributions, and we shape it through our actions. That argument is for a separate post.

* We did the research together — and Lucia did most of the work — but blame me for the content of this post.


Filed under Me @ work

Weathering and delayed births, get your norms off my body edition

You can skip down to the new data and analysis — or go straight to my new working paper — if you don’t need the preamble diatribe.

I have complained recently about the edict from above that poor (implying Black) women should delay their births until they are “financially ready” — especially in light of the evidence on their odds of marriage during the childbearing years. And then we saw what seemed like a friendly suggestion that poor women use more birth control lead to some nut on Fox News telling Rebecca Vallas, who spoke up for raising the minimum wage:

A family of three is not supposed to be living on the minimum wage. If you’re making minimum wage you shouldn’t be having children and trying to raise a family on it.

As if minimum wage is just a phase poor people can expect to pass through only briefly, on their way to middle class stability — provided they don’t piss it away by having children they can’t “afford.” This was a wonderful illustration of the point Arline Geronimus makes in this excellent (paywalled) paper from 2003, aptly titled, “Damned if you do: culture, identity, privilege, and teenage childbearing in the United States.” Geronimus has been pointing out for several decades that Black women face increased health risks and other problems when they delay their childbearing, even as White women have the best health outcomes when they delay theirs. This has been termed “the weathering hypothesis.” In that 2003 paper, she explores the cultural dynamic of dominance and subordination that this debate over birth timing entails. Here’s a free passage (where dominant is White and marginal is Black):

In sum, a danger of social inequality is that dominant groups will be motivated to promote their own cultural goals, at least in part, by holding aspects of the behavior of specific marginal groups in public contempt. This is especially true when this behavior is viewed as antithetical or threatening to social control messages aimed at the youth in the dominant group. An acknowledgment that teen childbearing might have benefits for some groups undermines social control messages intended to convince dominant group youth to postpone childbearing by extolling the absolute hazards of early fertility. Moreover, to acknowledge cultural variability in the costs and consequences of early childbearing requires public admission of structural inequality and the benefits members of dominant groups derive from socially excluding others. One cannot explain why the benefits of early childbearing may outweigh the costs for many African Americans without noting that African American youth do not enjoy the same access to advanced education or career security enjoyed by most Americans; that their parents are compelled to be more focused on imperatives of survival and subsistence than on encouraging their children to engage in extended and expensive preparation for the competitive labor market; indeed, that African Americans cannot even take their health or longevity for granted through middle age (Geronimus, 1994; Geronimus et al., 2001). And one cannot explain why these social and health inequalities exist without recognizing that structural barriers to full participation in American society impede the success of marginalized groups (Dressler, 1995; Geronimus, 2000; James, 1994). To acknowledge these circumstances would be to contradict the broader societal ethic that denies the existence of social inequality and is conflicted about cultural diversity. And it would undermine the ability the dominant group currently enjoys to interpret their privilege as earned, the just reward for their exercise of personal responsibility.

But the failure to acknowledge these circumstances results in a disastrous misunderstanding. As a society, we have become caught in an endless loop that rationalizes, perhaps guarantees, the continued marginalization of urban African Americans. In the case at hand, by misunderstanding the motivation, context, and outcomes of early childbearing among African Americans, and by implementing social welfare and public health policies that follow from this misunderstanding, the dominant European American culture reinforces material hardship for and stigmatization of African Americans. Faced with these hardships, early fertility timing will continue to be adaptive practice for African Americans. And, reliably, these fertility and related family “behaviors” will again be unfairly derided as antisocial. And so on.

Whoever said demography isn’t theoretical and political?

A simple illustration

In Geronimus’s classic weathering work, she documented disparities in healthy life expectancy, which is the expectation of healthy, or disability-free, years of life ahead. When a poor 18-year-old Black woman considers whether or not to have a child, she might take into account her expectation of healthy life expectancy — how long can she count on remaining healthy and active? — as well as, and this is crucial, that of her 40-year-old mother, who is expected to help out with the child-rearing (they’re poor, remember). Here’s a simple illustration: the percentage of Black and White mothers (women living in their own households, with their own children) who have a work-limiting disability, by age and education:


Not too many disabilities at age 20, but race and class kick in hard over these parenting years, till by their 50s one-in-five Black mothers with high school education or less has a disability, compared with one-in-twenty White mothers who’ve gone on to more education. That looming health trajectory is enough — Geronimus reasonably argues — to affect women’s decisions on whether or not to have a child (or go through with an accidental pregnancy). But for the group (say, Whites who aren’t that poor) who have a reasonable chance of getting higher education, and making it through their intensive parenting years disability-free, the economic consequence of an early birth weighs much more heavily.

Some new analysis

As I was thinking about all this the other day, I went to check on the latest infant mortality statistics, since that’s where Geronimus started this thread — with the observation that White women’s chance of a baby dying decline with age, while Black women’s don’t. And I noticed there is a new Period Linked Birth-Infant Death Data File for 2013. This is a giant database of all the births — with information from their birth certificates — linked to all the infant deaths from the same year. These records have been used for analyzing infant mortality dozens of times, including in pursuit of the weathering hypothesis, but I didn’t see any new analyses of the 2013 files, except the basic report the National Center for Health Statistics put out. The outcome is now a working paper at the Maryland Population Research Center.

The gist of the result is, to me, kind of shocking. Once you control for some basic health, birth, and socioeconomic conditions (plurality, parity, prenatal care, education, health insurance type, and smoking during pregnancy), the risk of infant mortality for Black mothers increases linearly with age: the longer they wait, the greater the risk. For White women the risk follows the familiar (and culturally lionized) U-shape, with the lowest risk in the early 30s. Mexican women (the largest Hispanic group I could include) are somewhere in between, with a sharp rise in risk at older ages, but no real advantage to waiting from 18 to 30.

I’ll show you (and these rates will differ a little from official rates for various technical reasons). First, the unadjusted infant mortality rates by maternal age:

Infant Death Rates, by Maternal Age: White, Black, and Mexican Mothers, U.S., 2013. Infant death rates per 1,000 live births for non-Hispanic white (N = 1,925,847), non-Hispanic black (N = 533,341), and Mexican origin (N = 501,390) mothers. Data source: 2013 Period Linked Birth/Infant Death Public Use File, Centers for Disease Control.

Infant Death Rates, by Maternal Age: White, Black, and Mexican Mothers, U.S., 2013. Infant death rates per 1,000 live births for non-Hispanic white (N = 1,925,847), non-Hispanic black (N = 533,341), and Mexican origin (N = 501,390) mothers. Data source: 2013 Period Linked Birth/Infant Death Public Use File, Centers for Disease Control.

These raw rates show the big health benefit to delay for White women, a smaller benefit for Mexican mothers, and no benefit for Black mothers. But when you control for those factors I mentioned, the infant mortality rates for young Black and Mexican mothers are lower — those are the mothers with low education and bad health care. Controlling for those things sort of simulates the decisions women face: given these things about me, what is the health effect of delay? (Of course, delaying could contribute to improving things, which is also part of the calculus.) Here are the adjusted age patterns:

Adjusted Probability of Infant Death, by Maternal Age: White, Black, and Mexican Mothers, U.S., 2013 Predicted probabilities of infant death generated by Stata margins command, adjusted for plurality, birth order, maternal education, prenatal care, payment source, and cigarette smoking during pregnancy; models estimated separately for white (A), black (B), and Mexican (C) mothers (see Tab. 1). Error bars are 95% confidence intervals. Data source: 2013 Period Linked Birth/Infant Death Public Use File, Centers for Disease Control.

Adjusted Probability of Infant Death, by Maternal Age: White, Black, and Mexican Mothers, U.S., 2013. Predicted probabilities of infant death generated by Stata margins command, adjusted for plurality, birth order, maternal education, prenatal care, payment source, and cigarette smoking during pregnancy; models estimated separately for white (A), black (B), and Mexican (C) mothers (see Tab. 1). Error bars are 95% confidence intervals. (A separate test showed the linear trend for Black women is statistically significant.) Data source: 2013 Period Linked Birth/Infant Death Public Use File, Centers for Disease Control.

My jaw kind of dropped. Infant mortality is mostly a measure of mothers’ health. Early childbearing looks a lot crazier for White women than for Black and Mexican women, and you can see why the messaging around delaying till your “ready” seems so out of tune to the less privileged (and that really means race more than class, in this case). Why wait? If women knew they had higher education, a good job, and decent health care awaiting them throughout their childbearing years, I think the decision tree would look a lot different.

Of course, I have often said that delayed marriage is good for women. And delayed childbearing would be — should be — too, as long as it doesn’t put the health of the mother and her children at risk (and squander the healthy rearing years of their grandparents).

Please check out the working paper for more background and references, and details about my analysis.


Filed under Me @ work

Lifetime chance of marrying for Black and White women

I’m going to Princeton next week to give a talk at the Office of Population Research. It’s a world-class population center, with some of the best trainers and trainees in the business, so I figured I’d polish up a little formal demography for them. (I figure if I run through this really fast they won’t have time to figure any mistakes I made.)

The talk is about Black and White marriage markets, which I’ve written about quite a bit, including when I posted the figure below, showing the extremely low number of local same-race, employed, single men per women Black women experience relative to White women — especially when they have less than a BA degree.

This figure was the basis for a video we made for my book, titled “Why are there so many single Black women?” For years I’ve been supporting the strong (“Wilsonian“) case that low marriage rates for Black women are driven by the shortage of “marriageable” men — living, employed, single, free men. I promised last year that Joanna Pepin and I were working on a paper about this, and we still are. So I’ll present some of this at Princeton.

Predictions off

Five years ago I wrote about the famous 2001 paper by Joshua Goldstein and Catherine Kenney, which made lifetime marriage predictions for cohorts through the Baby Boom, the youngest of whom were only 30 in the 1995 data the paper used. That’s gutsy, predicting lifetime marriage at age 30, so there’s no shame that they missed. They were closer for White women. They predicted that 88.6% of White women born 1960-1964 would eventually marry, and by the age 49-53 (in the 2013 American Community Survey) they were at 90.2%, with another 2.3% likely to marry by my estimates (see below). For Black women they missed by more. For the 1960-1964 cohort, they predicted only 63.8% would ever marry, but 71.3% were already married by 2013, and I’m projecting another 7.5% will marry. (I also wrote about a similar prediction, here.) If they actually get to 79%, that will be very different from the prediction.

Their amazing paper has been cited another 100 times since I wrote about it in 2010, but it doesn’t look like anyone has tried to test or extend their predictions.

Mass incarceration

Interestingly, Goldstein and Kenney undershot Black women’s marriage rates even though incarceration rates continued to rise after they wrote — a trend strongly implicated in the Black-White marriage disparity. This issue has increased salience today, with the release of a powerful new piece by Ta-Nehisi Coates in the Atlantic (my old job), which exposes the long reach of mass incarceration into Black families in ways that go way beyond the simple statistics about “available” men. The large ripple effects implied by his analysis — drawing from his own reporting and research by Deva Pager, Bruce Western, and Robert Sampson — suggest that any statistical model attempting to identify the impact of incarceration on family structure is likely to miss a lot of the action. That’s because people who’ve been out of prison for years are still affected by it, as are their relationships, their communities — and their children in the next generation.

Some new projections

I should note that some readers unfamiliar with demographic analysis may find parts of what follows morbidly depressing.

To set up the marriage market analysis I’m doing with Joanna — which isn’t ready to show here yet — I’m going to introduce some marriage projections at the talk. These use a different method than Goldstein and Kenney, because I have a different kind of data. This is a lifetable approach, in which I use first-marriage rates at every age to calculate how many women would get married at least once before they die if they lived 2010 over and over again from birth to death. I can do this because, unlike Goldstein and Kenney in 2001, I now have the American Community Survey (ACS), which asks a giant sample of people if they have married in the previous year, and how many times they’ve been married before, so I can calculate a first-marriage rate at every age. To this I add in death rates — making what we call a multiple-decrement life table — so that there are two ways out of the birth cohort: marriage or death. (Give me marriage or give me death.)

The way this works is you start with 100,00 people, and each year some of them die and some of them get married — according to the rates you have measured at one point in time. For example, in my tables, of 100,000 Black women at the start of year 0, only 98.7% make it to age 15, the first year they can be counted as married in the data. By the time you get down to age 30, there are only 67,922 left, as 2,236 have died and 29,843 have married for the first time. And so on down to the bottom. In the last row of the table, when they are all dead, you calculate how many got married before dying.*

The bottom line: 85.3% of White women, and 78.4% of Black women born and stuck in 2010 forever are projected to marry before they die — a surprisingly small gap. The first figure shows you that basic result:

NHBW life tables 2010.xlsx

Note that my projections of 85.3% of White women and 78.4% of Black women ever marrying are lower than, for example, the roughly 96% of White women and 91% of Black that were actually ever-married at age 85+ in 2010 (reported here), for several reasons. First, I count dead people against the ever-married number (additionally, married people live longer, not necessarily because they’re married). Second, today’s 90+ year-olds mostly got married 70 years ago, when times were different; my estimates are a projection of nowadays.

A very interesting age pattern emerges here, which is relevant to the incarceration and “available men” question. If you look back at the figure, notice that the big difference in marriage opens up early — peaking at 28 points by age 33, before narrowing to 7 points at the end.The big difference in marriage is that White women marry earlier. In fact, as the next figure shows, after age 33 Black women are more likely to marry than are White women. I don’t think I knew that. Here are the number marrying at each age:

NHBW life tables 2010.xlsx

Specifically, although White women are twice as likely to marry in their mid-twenties, of our fictional 100,000 women stuck in 2010, just 15.6% of White women, compared with 36.8% of Black women end up marrying after age 33.

The other way of looking at this — and an answer to a common question about marriage rates — is to see the chances of marrying after a given age if you haven’t married yet. This figure shows, for example, that a White women who lives to age 45 without marrying has a 26% chance of someday marrying, compared with a whopping 49% for Black women.

NHBW life tables 2010.xlsx

It is surprising that Black women, with lower cumulative odds of marrying at every age in the cohort, are so much more likely to marry conditional on getting to their 40s without marrying. Maybe you’ve got a better interpretation of this, but this is mine. Black women are not against marriage, and they are not ineligible for marriage in some way (even though most of these single women are already mothers**). Rather, they have not married earlier because they couldn’t find someone to marry. That’s because of all the Black men who are themselves dead, incarcerated or unemployed (or scarred by those experiences in their past) — or married to someone else. So within their respective marriage markets (which remain very segregated), the 45-year-old single White woman is much more likely to be someone that either doesn’t want to marry or can’t marry for some reason, while the 45-year-old single Black woman is more active and eligible in the marriage market. This fits with the errors in the earlier predictions, which failed to pick up on the upward shift in marriage age for Black women — marriage delayed rather than foregone.

What do you think of that interpretation? If you have a better idea I’ll mention you at Princeton next week.

Note: I found so many mistakes as I was doing this that it seems impossible there are any more. Nevertheless, caveat emptor: This analysis hasn’t been peer reviewed yet, so consider it only as reliable the latest economist’s NBER paper you read about on the front page of the every newspaper and website on earth. (And if you’re a journalist feel free to refer to this as a new working paper.)

* Technical notes: I used death rates from 2010 (found here), and marriage rates from the five-year ACS file for 2008-2012 (which has 2010 as its midpoint), from I adjusted the death rates because never-married people are more likely to die than average (I told you this was depressing). I had to use a 2007 estimate of mortality by age and marital status for that (found here), which is not that precise because it was in 10-year increments, which I didn’t bother to smooth because they didn’t have much effect anyway. The details of how to do a multiple-decrement lifetable are nicely described (with a lot of math) by Sam Preston here (though if you really want to replicate this, note one of his formulas is missing a negative sign, so plan to spend an extra few days on it). To help, I’m sharing my spreadsheet here, which has the formulas. (Note that survival in the life table doesn’t refer to being alive, it refers to being both alive and never-married.) The mortality and marriage rates are for non-Hispanic women; the never-married adjustment is for all women. For the marriage rates I used all Black and White women regardless of what other races they also specified (very few are multiple-race when you exclude Hispanics).

** In 2010, 63% of never-married Black women who lived in their households had at least own of their own children living with them.


Filed under Me @ work

Comment on Goffman’s survey, American Sociological Review rejection edition


Peer Review, by Gary Night.


  • I reviewed Alice Goffman’s book, On The Run.
  • I complained that her dissertation was not made public, despite being awarded the American Sociological Association’s dissertation prize. I proposed a rule change for the association, requiring that the winning dissertation be “publicly available through a suitable academic repository by the time of the ASA meeting at which the award is granted.” (The rule change is moving through the process.)
  • When her dissertation was released, I complained about the rationale for the delay.
  • My critique of the survey that was part of her research grew into a formal comment (PDF) submitted to American Sociological Review.

In this post I don’t have anything to add about Alice Goffman’s work. This is about what we can learn from this and other incidents to improve our social science and its contribution to the wider social discourse. As Goffman’s TED Talk passed 1 million views, we have had good conversations about replicability and transparency in research, and about ethics in ethnography. And of course about the impact of criminal justice system and over-policing on African Americans, the intended target of her work. This post is about how we deal with errors in our scholarly publishing.

My comment was rejected by the American Sociological Review.

You might not realize this, but unlike many scientific journals, except for “errata” notices, which are for typos and editing errors, ASR has no normal way of acknowledging or correcting errors in research. To my knowledge ASR has never retracted an article or published an editor’s note explaining how an article, or part of an article, is wrong. Instead, they publish Comments (and Replies). The Comments are submitted and reviewed anonymously by peer reviewers just like an article, and then if the Comment is accepted the original author responds (maybe followed by a rejoinder). It’s a cumbersome and often combative process, often mixing theoretical with methodological critiques. And it creates a very high hurdle to leap, and a long delay, before the journal can correct itself.

In this post I’ll briefly summarize my comment, then post the ASR editors’ decision letter and reviews.

Comment: Survey and ethnography

I wrote the comment about Goffman’s 2009 ASR article for accountability. The article turned out to be the first step toward a major book, so ASR played a gatekeeping role for a much wider reading audience, which is great. But then it should take responsibility to notify readers about errors in its pages.

My critique boiled down to these points:

  • The article describes the survey as including all households in the neighborhood, which is not the case, and used statistics from the survey to describe the neighborhood (its racial composition and rates of government assistance), which is not justified.
  • The survey includes some number (probably a lot) of men who did not live in the neighborhood, but who were described as “in residence” in the article, despite being “absent because they were in the military, at job training programs (like JobCorp), or away in jail, prison, drug rehab centers, or halfway houses.” There is no information about how or whether such men were contacted, or how the information about them was obtained (or how many in her sample were not actually “in residence”).
  • The survey results are incongruous with the description of the neighborhood in the text, and — when compared with data from other sources — describe an apparently anomalous social setting. She reported finding more than twice as many men (ages 18-30) per household as the Census Bureau reports from their American Community Survey of Black neighborhoods in Philadelphia (1.42 versus .60 per household). She reported that 39% of these men had warrants for violating probation or parole in the prior three years. Using some numbers from other sources on violation rates, that translates into between 65% and 79% of the young men in the neighborhood being on probation or parole — very high for a neighborhood described as “nice and quiet” and not “particularly dangerous or crime-ridden.”
  • None of this can be thoroughly evaluated because the reporting of the data and methodology for the survey were inadequate to replicate or even understand what was reported.

You can read my comment here in PDF. Since I aired it out on this blog before submitting it, making it about as anonymous as a lot of other peer-review submissions, I see no reason to shroud the process any further. The editors’ letter I received is signed by the current editors — Omar Lizardo, Rory McVeigh, and Sarah Mustillo — although I submitted the piece before they officially took over (the editors at the time of my submission were Larry W. Isaac and Holly J. McCammon). The reviewers are of course anonymous. My final comment is at the end.

ASR letter and reviews

Editors’ letter:


Dear Prof. Cohen:

The reviews are in on your manuscript, “Survey and ethnography: Comment on Goffman’s ‘On the Run’.” After careful reading and consideration, we have decided not to accept your manuscript for publication in American Sociological Review (ASR).  Our decision is based on the reviewers’ comments, our reading of the manuscript, an overall assessment of the significance of the contribution of the manuscript to sociological knowledge, and an estimate of the likelihood of a successful revision.

As you will see, there was a range of opinions among the reviewers of your submission.  Reviewer 1 feels strongly that the comment should not be published, reviewer 3 feels strongly that it should be published, and reviewer 2 falls in between.  That reviewer sees merit in the criticisms but also suggests that the author’s arguments seem overstated in places and stray at times from discussion that is directly relevant to a critique of the original article’s alleged shortcomings.

As editors of the journal, we feel it is essential that we focus on the comment’s critique of the original ASR article (which was published in 2009), rather than the recently published book or controversy and debate that is not directly related to the submitted comment.  We must consider not only the merits of the arguments and evidence in the submitted comment, but also whether the comment is important enough to occupy space that could otherwise be used for publishing new research.  With these factors in mind, we feel that the main result that would come from publishing the comment would be that valuable space in the journal would be devoted to making a point that Goffman has already acknowledged elsewhere (that she did not employ probability sampling).

As the author of the comment acknowledges, there is actually very little discussion of, or use of, the survey data in Goffman’s article.   We feel that the crux of the argument (about the survey) rests on a single sentence found on page 342 of the original article:  “The five blocks known as 6th street are 93 percent Black, according to a survey of residents that Chuck and I conducted in 2007.”  The comment author is interpreting that to mean that Goffman is claiming she conducted scientific probability sampling (with all households in the defined space as the sampling frame).  It is important to note here that Goffman does not actually make that claim in the article.  It is something that some readers might infer.  But we are quite sure that many other readers simply assumed that this is based on nonprobability sampling or convenience sampling.  Goffman speaks of it as a survey she conducted when she was an undergraduate student with one of the young men from the neighborhood.  Given that description of the survey, we expect many readers assumed it was a convenience sample rather than a well-designed probability sample.  Would it have been better if Goffman had made that more explicit in the original article?  Yes.

In hindsight, it seems safe to say that most scholars (probably including Goffman) would say that the brief mentions of the survey data should have been excluded from the article.  In part, this is because the reported survey findings play such a minor role in the contribution that the paper aims to make.

We truly appreciate the opportunity to review your manuscript, and hope that you will continue to think of ASR for your future research.


Omar Lizardo, Rory McVeigh, and Sarah Mustillo

Editors, American Sociological Review

Reviewer: 1

This paper seeks to provide a critique of the survey data employed in Goffman (2009).  Drawing on evidence from the American Community Survey, the author argues that data presented in Goffman (2009) about the community in which she conducted her ethnography is suspect.  The author draws attention to remarkably high numbers of men living in households (compared with estimates derived from ACS data) and what s/he calls an “extremely high number” of outstanding warrants reported by Goffman.  S/he raises the concern that Goffman (2009) did not provide readers with enough information about the survey and its methodology for them to independently evaluate its merits and thus, ultimately, calls into question the generalizability of Goffman’s survey results.

This paper joins a chorus of critiques of Goffman’s (2009) research and subsequent book.  This critique is novel in that the critique is focused on the survey aspect of the research rather than on Goffman’s persona or an expressed disbelief of or distaste for her research findings (although that could certainly be an implication of this critique).

I will not comment on the reliability, validity or generalizability of Goffman’s (2009) evidence, but I believe this paper is fundamentally flawed.  There are two key problems with this paper.  First the core argument of the paper (critique) is inadequately situated in relation to previous research and theory.  Second, the argument is insufficiently supported by empirical evidence.

The framing of the paper is not aligned with the core empirical aims of the paper.  I’m not exactly sure what to recommend here because it seems as if this is written for a more general audience and not a sociological one.  It strikes me as unusual, if not odd, to reference the popularity of a paper as a motivation for its critique.  Whether or not Goffman’s work is widely cited in sociological or other circles is irrelevant for this or any other critique of the work.  All social science research should be held to the same standards and each piece of scholarship should be evaluated on its own merits.

I would recommend that the author better align the framing of the paper with its empirical punchline.  In my reading the core criticism of this paper is that the Goffman (2009) has not provided sufficient information for someone to replicate or validate her results using existing survey data.  Although it may be less flashy, it seems more appropriate to frame the paper around how to evaluate social science research.  I’d advise the author to tone down the moralizing and discussion of ethics.  If one is to levy such a strong (and strongly worded) critique, one needs to root it firmly in established methods of social science.

That leads to the second, and perhaps even more fundamental, flaw.  If one is to levy such a strong (and strongly worded) critique, one needs to provide adequate empirical evidence to substantiate her/his claims.  Existing survey data from the ACS are not designed to address the kinds of questions Goffman engages in the paper and thus it is not appropriate for evaluating the reliability or validity of her survey research.  Numerous studies have established that large scale surveys like the ACS under-enumerate black men living in cities.  They fall into the “hard-to-reach” population that evade survey takers and census enumerators.  Survey researchers widely acknowledge this problem and Goffman’s research, rather than resolving the issue, raises important questions about the extent to which the criminal justice system may contribute to difficulties for conventional social science research data collection methods.  Perhaps the author can adopt a different, more scholarly, less authoritative, approach and turn the inconsistencies between her/his findings with the ACS and Goffman’s survey findings into a puzzle.  How can these two surveys generate such inconsistent findings?

Just like any survey, the ACS has many strengths.  But, the ACS is not well-suited to construct small area estimates of hard-to-reach populations.  The author’s attempt to do so is laudable but the simplicity of her/his analysis trivializes the difficultly in reaching some of the most disadvantaged segments of the population in conventional survey research.  It also trivializes one of the key insights of Goffman’s work and one that has been established previously and replicated by others: criminal justice contact fundamentally upends social relationships and living arrangements.

Furthermore, the ACS doesn’t ask any questions about criminal justice contact in a way that can help establish the validity of results for disadvantaged segments of the population who are most at-risk of criminal justice contact.  It is impossible to determine using the ACS how many men (or women) in the United States, Pennsylvania, or Philadelphia (or any neighborhood therein), have an outstanding warrant.  The ACS doesn’t ask about criminal justice contact, it doesn’t ask about outstanding warrants, and it isn’t designed to tap into the transient experiences of many people who have had criminal justice contact.  The author provides no data to evaluate the validity of Goffman’s claims about outstanding warrants.  Advancements in social science cannot be established from a “she said”, “he said” debate (e.g., FN 9-10).  That kind of argument risks a kind of intellectual policing that is antithetical to established standards of evaluating social science research.  That being said, someone should collect this evidence or at a minimum estimate, using indirect estimation methods, what fraction of different socio-demographic groups have outstanding warrants.

Although I believe that this paper is fundamentally flawed both in its framing and provision of evidence, I would like to encourage the author to replicate Goffman’s research.  That could involve an extended ethnography in a disadvantaged neighborhood in Philadelphia or another similar city.  That could also involve conducting a small area survey of a disadvantaged, predominantly black, neighborhood in a city with similar criminal justice policies and practices as Philadelphia in the period of Goffman’s study.  This kind of research is painstaking, time consuming, and sorely needed exactly because surveys like the ACS don’t – and can’t – adequately describe or explain social life among the most disadvantaged who are most likely to be missing from such surveys.

Reviewer: 2

I read this manuscript several times. It is more than a comment, it seems. It is 1) a critique of the description of survey methods in GASR and 2) a request for some action from ASR “to acknowledge errors when they occur.” The errors here have to do with Goffman’s description of survey methods in GASR, which the author describes in detail. This dual focus read as distracting at times. The manuscript would benefit from a more squarely focused critique of the description of survey methods in GASR.

Still, the author’s comment raises some valid concerns. The author’s primary concern is that the survey Goffman references in her 2009 ASR article is not described in enough detail to assess its accuracy or usefulness to a community of scholars. The author argues that some clarification is needed to properly understand the claims made in the book regarding the prevalence of men “on the run” and the degree to which the experience of the small group of men followed closely by Goffman is representative of most poor, Black men in segregated inner city communities. The author also cites a recent publication in which Goffman claims that the description provided in ASR is erroneous. If this is the case, it seems prudent for ASR to not only consider the author’s comments, but also to provide Goffman with an opportunity to correct the record.

I am not an expert in survey methods, but there are moments where the author’s interpretation of Goffman’s description seems overstated, which weakens the critique. For example, the author claims that Goffman is arguing that the entirety of the experience of the 6th Street crew is representative of the entire neighborhood, which is not necessarily what I gather from a close reading of GASR (although it may certainly be what has been taken up in popular discourse on the book). While there is overlap of the experience of being “on the run,” namely, your life is constrained in ways that it isn’t for those not on the run, it does appear that Goffman also uses the survey to describe a population that is distinct in important ways from the young men she followed on 6th street. The latter group has been “charged for more serious offenses like drugs and violent crimes,” she writes (this is the group that Sharkey argues might need to be “on the run”), while the larger group of men, whose information was gathered using survey data, were typically dealing with “more minor infractions”: “In the 6th Street neighborhood, a person was occasionally ‘on the run’ because he was a suspect in a shooting or robbery, but most people around 6th street had warrants out for far more minor infractions [emphasis mine].”

So, as I read it (I’ve also read the book), there are two groups: one “on the run” as a consequence of serious offenses and others “on the run” as a consequence of minor infractions. The consequence of being “on the run” is similar, even if the reason one is “on the run” varies.

The questions that remain are questions of prevalence and generalizability. The author asks: How many men in the neighborhood are “on the run” (for any reason)? How similar is this neighborhood to other neighborhoods? Answers to this question do rely on an accurate description of survey methods and data, as the author suggests.

This leads us to the most pressing and clearly argued question from the author: What is the survey population? Is it 1) “people around 6th Street” who also reside in the 6th Street neighborhood (of which, based on Goffman’s definition of in residence, are distributed across 217 distinct households in the neighborhood, however the neighborhood is defined e.g., 5 blocks or 6 blocks) or 2) the entirety of the neighborhood, which is made up of 217 households. It appears from the explanation from Goffman cited by the author that it is the former (“of the 217 households we interviewed,” which should probably read, of the 308 men we interviewed, all of whom reside in the neighborhood (based on Goffman’s definition of residence), 144 had a warrant…). Either way, the author makes a strong case for the need for clarification of this point.

The author goes on to explain the consequences of not accurately distinguishing among the two possibilities described above (or some other), but it seems like a good first step would be to request a clarification (the author could do this directly) and to allow more space than is allowed in a newspaper article to provide the type of explanation that could address the concerns of the author.

Is this the purpose of the comment? Or is the purpose of the comment merely to place a critique on record?  The primary objective is not entirely clear in the present manuscript.

The author’s comment is strong enough to encourage ASR to think through possibilities for correcting the record. As a critique of the survey methods, the comment would benefit from more focus. The comment could also do a better job of contextualizing or comparing/contrasting the use of survey methods in GASR with other ethnographic studies that incorporate survey methods (at the moment such references appear in footnotes).

Reviewer: 3

This comment exposes major errors in the survey methodology for Goffman’s article.  One major flaw is that the goffman article describes the survey as inclusive of all households in the neighborhood but later, in press interviews, discloses that it is not representative of all households in the neighborhood.  Another flaw that the author exposes is goffman’s data and methodological reporting not being up to par to sociological standards.  Finally, the author argues that the data from the survey does not match the ethnographic data.

Overall, I agree with the authors assertions that the survey component is flawed.  This is an important point because the article claims a large component of its substance from the survey instrument.  The survey helped goffman to bolster generalizability , and arguably, garner worthiness of publication in ASR.  If the massive errors in the survey had been exposed early on it is possible that ASR might have held back on publishing this article.

I am in agreement that ASR should correct the error highlighted on page 4 that the data set is not of the entire neighborhood but of random households/individuals given the survey in an informal way and that the sampling strategy should be described.  Goffman should aknowledge that this was a non-representative convenience sample, used for bolstering field observations.  It would follow then that the survey component of the ASR article would have to be rendered invalid and that only the field data in the article should be taken at face value.  Goffman should also be asked to provide a commentary on her survey methodology.

The author points out some compelling anomalies from the goffman survey and general social survey data and other representative data.  At best, goffman made serious mistakes with the survey and needs to be asked to show those mistakes and her survey methodology or she made up some of the data in the survey and appropriate action must be taken by ASR.  I agree with the authors final assessment, that the survey results be disregarded and the article be republished without mention of such results or with mention of the results albeit showing all of its errors and demonstrating the survey methodology.

My response

Regular readers can probably imagine my long, overblown, hyperventilating response to Reviewer 1, so I’ll just leave that to your imagination. On the bottom line, I disagree with the editors’ decision, but I can’t really blame them. Would it really be worth some number of pages in the journal, plus a reply and rejoinder, to hash this out? Within the constraints of the ASR format, maybe the pages aren’t worth it. And the result would not have been a definitive statement anyway, but rather just another debate among sociologists.

What else could they have done? Maybe it would have been better if the editors could simply append a note to the article advising readers that the survey is not accurately described, and cautioning against interpreting it as representative — with a link to the comment online somewhere explaining the problem. (Even so of course Goffman should have a chance to respond, and so on.)

It’s just wrong that now the editors acknowledge there is something wrong in their journal — although we seem to disagree about how serious the problem is — but no one is going to formally notify the future readers of the article. That seems like bad scholarly communication. I’ve said from the beginning that there’s no need for a high-volume conversation about this, or attack on anyone’s integrity or motives. There are important things in this research, and it’s also highly flawed. Acknowledge the errors — so they don’t compound — and move on.

This incident can help us learn lessons with implications up and down the publishing system. Here are a couple. At the level of social science research reporting: don’t publish survey results data without sufficient methodological documentation — let’s have the instrument and protocol, the code, and access to the data. At the system level of publishing, why do we still have journals with cost-defined page limits? Because for-profit publishing is more important than scholarly communication. The sooner we get out from under that 19th-century habit the better.


Filed under Me @ work