More fathers married when their first child is born? Probably not

A startling data brief from the National Center for Health Statistics reports that the percentage of fathers who weren’t married at the time of their first births fell from the 1980s to the 2000s. Here is the first “key finding”: “The percentage of fathers aged 15–44 whose first births were nonmarital was lower in the 2000s (36%) than in the previous 2 decades.”

That is shocking. How could we have a falling percentage of fathers not married at the time of their first births? The author, Gladys Martinez, writes:

Results from this study indicate that in the 2000s, the percentage of fathers with nonmarital first births declined. However, the percentage of fathers whose nonmarital first births occurred within a cohabiting union increased. This pattern differs from that for the mother. Data for women showed that the share of all births that occurred to unmarried women has doubled between 1988 and 2009–2013, and that the increase was driven by an increase in the share of births to cohabiting women.

Here is the main figure, showing the decline in nonmarital first births for fathers:

nchs-men-1But I think this is not correct (this concern was first raised to me by Pew researcher Gretchen Livingston). Here’s why. As the figure shows, the source for these three decades of data is the National Survey of Family Growth. The earliest this survey captured men’s births (awkward phrase, but you know what I mean) was in 2002. And the ages included in the survey were 15-44. But the figure has information about births in the years 1980-1989. By my math, the oldest a 15-44-year-old in 2002 could have been in 1989 is 31. So that 2002 survey is only returning data on the marital status of men ages 15-31 in the 1980s.

I always have to do one of these to make sure I’m not crazy when I’m trying to work something like this out. This is how old 15-44 year-olds in 2002 were in the 1980s, excluding those under 15 (click to enlarge):

age-in-80s

They’re all 15-31 (or younger) in the 1980s. In contrast, if they combine the 2006-2010 survey (collected over 5 years) with the 2011-2013 survey (collected over 3 years), they have men ages 15-42 in the 1990s and 15-44 in the 2000s. So, as the age of the men in the sample rose, the proportion married when they had their first birth rose, too. This is what we would expect: younger first-time parents are much less likely to be married.

Consider, then, the followup finding from the brief: for men of every age the proportion unmarried at the time of their first birth has increased:

nchs-men-2How can it be that the overall proportion unmarried is falling, while it’s rising for each age group? The answer in the data brief is that first-time unmarried fathers are getting older. But remember — the samples are getting older across these decades, because of the timing of the surveys: they age from 15-31 to 15-44. That explains the next figure perfectly. Look at that increase in the proportion of unmarried first-time fathers who are 25-44:

nchs-men-3In the 1980s, just 8% of first-time unmarried fathers were age 25-44, compared with a whopping 33% in the 2000s. But doesn’t it seem likely that you’ll have fewer men ages 25-44 in a group that only goes up to age 31, versus a group that goes all the way up to age 44?

This stuff gets confusing, but I’m pretty sure this is right. That is, wrong. I do not believe that there is a falling percentage of fathers having first births when they’re not married. What looked like a weird, complicated demographic problem — falling unmarried first-fatherhood along with rising unmarried first-motherhood — is probably an artifact of a weird, complicated problem in the analysis.

There is nothing in the data brief to suggest there was an adjustment for the changing age composition of the data for these decades, but maybe they did something I don’t understand. If not, I think NCHS should correct or retract this report.

1 Comment

Filed under Research reports

Proposed rule change for the American Sociological Association Dissertation Award

open science can

Dear Prof. Janoski and members of the ASA Dissertation Award Committee,

I am writing to suggest a rule change for the Dissertation Award. Jordan Robison at ASA told me:

Any rule change for an award committee is usually voted on by the same award committee, presented to Committee on Awards. If Committee on Awards approves any rule change it is then brought to Council, who makes the final vote.

So it is my hope to persuade you to take this rule change up the chain to the Committee on Awards (currently chaired by Jane Sell at Texas A&M), and from there to Council (where the current liaison to the Committee on Awards is Adia Harvey Wingfield at Georgia State, moving to Washington University in St. Louis).

Background

The issue has been raised with regard to 2011 winner Alice Goffman’s dissertation at Princeton. That dissertation is not available at the Princeton library. A query from me to the library produced this response from Martin A. Mbugua, Director of Media Relations & University Spokesperson:

Alice Goffman was granted an exemption from submitting her dissertation to the University Archives, so we do not have a copy of her dissertation in our collection. The Graduate School in 2012 instituted a dissertation embargo policy under which no dissertation would be exempted from the submission requirement. Thus, a dissertation may be embargoed for a period of time to allow for publication in other forms, but it must be submitted to the University’s Seeley G. Mudd Manuscript Library.

Rationale

Goffman’s case is extreme in that her dissertation apparently may never be available, but it remains the case that people at Princeton (and at other schools) may choose to embargo their dissertations, to keep them out of public circulation, while they publish them commercially. Perhaps other schools also allow a complete exemption. If that is their privilege, and they choose to exercise it, then I believe they should be ineligible for the ASA Dissertation Award. My logic is that, if ASA holds up a dissertation as the best of the year, the purpose of that honor is partly to inspire and inform other sociologists about what constitutes the best doctoral research. If the dissertation is not publicly available, that purpose is undermined. Further, I think it would be a welcome – albeit small – signal in support of emerging norms of open science for the association to affirm the principle that dissertation research should be publicly available.

Proposal

I propose the following rule be added:

In order to be considered for the ASA Dissertation Award, a nominee must commit to making his or her dissertation publicly available through a suitable academic repository by the time of the ASA meeting at which the award is granted.

Thank you for considering this request. I welcome your response, and would be happy to work with you on getting this rule – or something like it – passed by the ASA Council. Please let me know if there is anything else you need.

Sincerely,

Philip Cohen

19 Comments

Filed under Me @ work

Social class divides the futures of high school students

There is new research from the National Center for Education Statistics (NCES), written up by Susan Dynarsky at the New York Times Upshot. The striking finding is that poor children in the top quartile on high school math scores have a 41% chance of finishing a BA degree by their late twenties — the same chance as children from the second-lowest quartile in math scores who are high-socioeconomic status (SES). Poor children from the third-highest quartile in high school math have graduation about equal to the worst-scoring children form the richest group. Here’s the figure:

upshot-math-ba

The headline on the figure is misleading, actually, since SES is not measured by wealth, but by a combination of parental education, occupation, and income. (Low here means the bottom quartile of SES, Middle is the 25th to 75th percentile, and High is 75th and up.)

One possible mechanism for the disparity in college completion rates is education expectations. Dynarsky mentions expectations measured in the sophomore year of high school, which was 2002 for this cohort. What she doesn’t mention is how much those expectations changed by senior year. Going to the NCES source for that data (here) I found this chart, which I annotated in red:

Print

Between sophomore and senior year, the percentage expecting to finish a BA degree or more decreased and the percentage expecting to go to two-year college increased, across SES levels. But the change was much greater for lower SES students. So the gap in expecting to go to two-year college between high- and low-SES students grew from 6 to 17 percentage points; that is, from 9% versus 3% in the sophomore year to 22% versus 6% in the senior year. That’s a big crushing of expectations that happened in the formative years at the end of high school.

6 Comments

Filed under Research reports

Sociologists asking me questions

A few weeks ago I participating in an Ask Me Anything session on the Sociology Job Market Rumors board, a mostly anonymous board devoted to gossip and chatter about sociology and its subjects. It’s posted here (with some responses), but I’ve copied it below.

We thank renowned family and gender scholar, Dr. Philip Cohen, for participating in SJMR’s inaugural AMA. Below are Philip’s responses. You can learn more about his research by visiting his website and his blog.

1. What was your answer when, at the age of 9, people asked you: “what will you do when you grow up?”

At age 9 my plan was to be a writer. Then, in chronological order: artist, prophet, rock star, and eventually journalist.

2. If you could do your career over, what would you do differently? What would you *definitely not* do differently?

I would have learned more broadly earlier, starting in college: I should have taken more math and statistics, and natural sciences. A decision I would not change would be moving into demography. My original inclination was much more theoretical and historical (My MA thesis was on the women’s suffrage movement and intersectionality). But when I took my UMass MA to the University of Maryland the demographers there were very strong (and supportive): Harriet Presser, Suzanne Bianchi, Sonal Desai, and my advisor, Reeve Vanneman, and I got pulled in. That was great luck.

3. As one of the most prominent public sociologists, how do you balance your traditional scholarship with your public blog, media appearances/writings, etc. How do you decide which issue merits which approach? And how, if at all, do you think these roles can come into conflict?

There is not much method to my priorities, I’m afraid – I’m just not that well organized. But on the balance question, usually when people ask this they are concerned about academic productivity – real productivity – peer-reviewed research and grants. Sometimes when I’m chatting with someone who’s a regular research academic, we talk about the blog, and media stuff, and my textbook, and then they say, “So what are you working on these days?” I would – or at least could – have done more peer-reviewed research if I hadn’t written 760 blog posts and a textbook in the last 5 years. On the other hand, maybe I would have just been less productive altogether. Because all my research ideas now come from interaction around my public work, or from students I work with.

A lot of people worry that if they blog or get on Twitter or try to write op-eds they won’t be able to do research anymore. And for some people this could be a problem – if it doesn’t suit you, don’t do it. But if it does suit you it may make you more, not less, productive. You learn to write better, and faster, with more practice. And you might learn how to make your work more effective from the interactions you have.

On the other hand, the social media side is endlessly distracting, and this can be a serious problem. The only way I get anything else done is by making commitments and then forcing myself into fulfilling them. I’ve made some big commitments which help me focus by regularly imposing deadlines: writing my textbook, editing Contexts, and working with the Council on Contemporary Families – these all make deadlines for me. And working with students provides a steady stream of commitments (I have seven PhD student advisees) since we’re always co-authoring papers and meeting various deadlines.

Finally, as for topics, I’m very impressionable. I got heavily into the Regnerus affair because there was a need for organization and it was timely (my blog hosted the “200 researchers” letter that Gary Gates organized), and I was on the ASA Family Section council; I got into critiquing marriage promotion because I had the expertise and data necessary to debunk the nutty stuff those people were doing. And so on. In each of these cases I got positive reinforcement from a community of sociologists and others who appreciate my efforts.

On the conflict question, I don’t think I’ve had a problem with roles coming into conflict – though my critics might disagree. The fact is that the different kinds of work I do have overlapping audiences, so someone reviewing my article for a journal might know what I’ve written on the blog, and political critics have access to my scholarly work. That potential scrutiny is mostly good – my peer-reviewed work should be more carefully scrutinized if it relates to strongly-held value positions that I have made public.

4. How, if at all, should we study the role that culture may play in reproducing inequality? The sort of relevant cultural things (like in Lareau’s work, for instance) seem maddeningly difficult to measure and put into conventional regression models. Yet, simultaneously, there do seem to be strong cultural elements in play in addition to the well-established structural ones. Methodologically, epistemologically, politically, is it possible to move from either/or approaches to culture and structure to both/and approaches?

This is really important. Theoretically, this is all about capitalism, modernity, and social change – the core questions of sociology. What drives change? What defines our era and its identities? Etc. Politically, this is all about all the policy questions of inequality: Head Start versus welfare, K-12 versus community college versus Pell grants, the minimum wage and skyrocketing executive pay. And of course race, poverty, and crime. (The political discourse around this stuff is depressingly impoverished in either/or terms; “culture” usually means something is poor people’s fault, versus “structure” or “the economy,” which refer to the unquestioned behavior of elites, with their unassailable motivation for self-interest.)

To study it is of course very difficult, for the reasons you say. I am a big fan of Lareau, as well as some other book writers on the qualitative side in my area, like Kathryn Edin, Marianne Cooper, Shamus Khan, and Sarah Damaske (a non-exhaustive list off the top of my head), as well as historians. Not good for regression models, but great for getting a feel for inequality and generating hypotheses. In my stratification seminar I assigned some neighborhood work (Sharkey), as well as some work from audit studies of discrimination – showing the wide range of things you might lump under “culture.” Often, like with racism, “culture” lives in the residual of your regression model, or is proxied with unknown validity by variables such as education, race, nativity, family structure, and so on (though credit to Michael Gaddis and others who have tried to put this stuff right in the model).

I think it’s really important to shake up the either/or thing. For example, with regard to marriage, it doesn’t make sense to use “culture” to attribute blame to poor people for not marrying any more than it does to call failure to raise the minimum wage a cultural issue (which it is). Individuals act under the impositions of “culture,” which appears as structural to them as the economy – I’m thinking of norms that essentially force people into shotgun weddings or divorce. (When I’m in the mood, I actually love Bourdeiu’s structured structures and structuring structures.) Good luck with this in politics.

5. In the last few years, there has been growing numbers of economists studying traditionally sociological topics like the family and even gender. What do you think will be the impact of this influx on the way sociologists study these topics? Has it modified your own research in any way?

I don’t agree with the premise of the question. Gary Becker won a Nobel Prize for his work on subjects like that going back to the 1970s. When I was studying gender inequality in grad school in the 1990s, Claudia Goldin’s Understanding the Gender Gap (1990) and the work by Blau and Kahn from 1992-2000 was central. In fact, my own work on the gender gap in earnings is all based on assumptions from the human capital model. Economists of course cause all kinds of problems for sociologists, but that’s not a recent phenomenon. It takes real training for sociologists to learn how to understand what economists are doing – and why we might disagree with their approach – and I don’t think most of us get that training.

6. If you had free reign to recreate a grad program in sociology, what would the curriculum look like?

In my experience being in grad school in two departments and on the faculty in three, everyone has their own ideas of how the training should be – and we mostly want it to be like our training. I generally come down on the side of people who want the training to be more broad than narrow, more theoretical than not, and more demographic than not. The best seminars (and the ones that are hardest to teach) combine theory and methods around a substantive theme (one of the reasons demography is such great training is that their seminars do this). I also think grad students should learn how to write – including for different audiences – and (for those heading that direction) how to teach.

7. What’s the most important advice you give your grad students?

I have no idea. Read and write a lot. Go to talks. Support each other. Have a life. There’s no shame in leaving academia behind, but if you want to give it a try we’ll give it our best shot.

8. What is the current and future state of Maryland sociology given the deep budget cuts?

Deeply troubled. We have some great people and we’ll get through it. Pray for us.

9. Do you prefer Philip or Phil? I hear you refer to yourself as Philip, but others always seem to call you by Phil.

I prefer Philip – thanks for asking – but not enough to correct people. It’s just one of those names people shorten without asking. (In writing it has to do with not knowing how many L’s to use.)

10. What do you think of SJMR? Any theories as to why “right” wing posts and sentiments often dominate? What should it look like in the future?

I have no way of understanding what goes on because I have no idea who anyone is. People routinely lie about their status and affiliations, tell lies about other people, impersonate people, and spread misinformation. You can argue the benefits of anonymity, but it makes some awful stuff possible on here.

11. Almost all of your work focuses on the US (obvious exception: “Headed Toward Equality? Housework Change in Comparative Perspective”). Almost all of your blog focuses just on work done in the US (occasional exceptions: global comparisons). Michele Lamont said in her ASA presidential candidate personal statement:

“My intellectual agenda will be to promote a greater internationalization of American sociology, with a focus on cultural and social processes of inequality and stigmatization in the United States and abroad.”

Do you consider yourself a scholar of “inequality” or a scholar of “inequality in America”? What do you think the relationship should be between the study of inequality in the U.S. and the study of inequality globally? For instance, you often speak in your blog about inequality processes that are very specific to the US (say, mass incarceration or even the racial system more generally). While country-specific processes are interesting to study in the US, the only non-American work that seems to catch most people eye is more global, multi-country comparisons. What should be the role of (possibly) country-specific processes that take place in another country in the study of inequality? Beyond trite truisms like “I think it’s important to have a global perspective” that often get repeated (without that “global perspective” making much of an impact in citation networks), is there any way that American sociology can ever become less self-obsessed and provincial in its research interests?

It’s a really important question and a great subject, but I don’t know what to do about it. At Contexts, Syed Ali and I are really trying to recruit more writers from outside the U.S., with some success, but that’s just one editorship at one journal (where the editors have a lot of discretion).

American provincialism among sociologists is a big problem. We did a survey of people teaching family when I was preparing my textbook proposal, and very few teachers wanted global material. Which was fine with me, because most of the global comparisons you get in intro-level textbooks are obvious or cliché, like arranged marriages in India, polygamy in Africa, cohabitation in Sweden – things that make the superficial (though important) sociological point that it doesn’t have to be this way. I don’t know enough to write for real about societies other than this one, at least not without collaborators.

About 10 years ago some China scholars made a push, led by Wang Feng and Deborah Davis, to integrate studies of inequality in China and the U.S. They had some meetings and published an edited volume with Stanford called Creating Wealth and Poverty in Postsocialist China, that I was part of. They were tired of there being a China session at ASA, instead of having the papers on China spread around the substantive sessions that they were writing about. That was a great idea.

It’s difficult. There are institutional barriers, linguistic barriers. How much should a grad student risk studying in another country, learning a new language? What if the project doesn’t work out? There are some established comparative traditions, such as US/Scandinavia, or US/West Europe, so if you do one of those you can get funding, maybe a postdoc or a job. And there are demography opportunities in specific places. But what if you want to do less well-trodden cases? Picking a project is always the hard part. But when you’re choosing between Add Health and PSID then the harm from being wrong – and the cost of switching – is much less than if you’re choosing between Botswana and Cambodia. It’s just hard to promote that kind of risk-taking when funding is so precarious.

I have great respect for those sociologists who have developed projects in other countries, and trained students on them. (That includes my colleagues at Maryland doing the India Human Development Survey – these are often demographers.)

12. What’s the biggest mistake you see sociologists make? (You can interpret this in any way you wish.)

I don’t know whether it’s a mistake, but I don’t think a lot of academic sociologists take advantage of the privilege we have of studying whatever we want. My dissertation was on labor market inequality. Since then, I’ve written academic papers on children’s disability rates, divorce, gender and color preferences, Regnerus, race and genetics, and pornography – none of which I studied in grad school. And for my textbook I researched and wrote 15,000-word chapters on subjects I knew nothing about to start, such as family violence and sexuality. Wow! I love my job.

Thanks for inviting me to do this!

2 Comments

Filed under Me @ work

On the ropes (Goffman review)

westphilly

First a short overview, then my comments.

Alice Goffman’s book On the Run: Fugitive Life in an American City, is one of the major events in sociology of the last few years. How unusual is it for a book based on a sociology dissertation to get this treatment?

Cornel West endorsed it as “the best treatment I know of the wretched underside of neoliberal capitalist America.”  Writing in theNew York Times Book Review, Alex Kotlowitz said it was “a remarkable feat of reporting” with an “astonishing” level of detail and honesty.  The New Yorker’s Malcolm Gladwell called it “extraordinary,” and Christopher Jencks, in the New York Review of Books, predicted that it would “become an ethnographic classic.”  Tim Newburn, a highly regarded criminologist at the London School of Economics, hailed On the Run as “gloriously readable” and “sociology at its best.”

On the other hand, the book has lots of critics — here’s James Forman in the Atlantic — who think Goffman’s research subjects aren’t representative of the poor Black communities she wants to describe. And then there’s the exploitation/privilege/outsider argument, summarized by Claude Fischer (I added the links):

A typical line of criticism charges that outsiders cannot accurately describe their subjects of study. For example, one highly circulated review of Goffman’s book, alarmed at her white privilege, describes the study as “theft,” abetting “fantasies of black pathology,” and possibly causing harm by revealing to police the tricks of hiding. “Inner city Philadelphia isn’t Alice Goffman’s home,” another reviewer writes, “and it’s not her job to turn it into a jungle that needs interpreting.” A Buzzfeed writer simply tweeted, “Ban outsider ethnographies.”

Comments

On the morals and ethics, I’m not going to draw conclusions, except to say that I agree Goffman was wrong to help try to find and kill the guy who killed her friend, if what she says is true.

On the social science, I also have a limited perspective, because I don’t really read the book as social science; I think it’s much more a sociological memoir — and I don’t mean that as a criticism of ethnography, to which I would not apply that label in general.

In fact, the book is least persuasive when she tries to be most dispassionate. On the issue of representativeness, for example, Goffman clearly is wrong when she writes:

Initially I assumed that Chuck, Mike, and their friends represented an outlying group of delinquents: the bad apples of the neighborhood. After all, some of them occasionally sold marijuana and crack cocaine to local customers, and sometimes they even got into violent gun battles. I grew to understand that many young men from 6th Street were at least intermittently earning money by selling drugs, and the criminal justice entanglements of Chuck and his friends were on a par with what many other unemployed young men in the neighborhood were experiencing.

That’s a non sequitur, with the typical slippery use of “many” (which I have probably fallen into myself). The fact is her entire project was shaped by the guy she initially fell in with, and his friends, and they were by any measure an “outlying group.” If you don’t see people who have multiple running gun battles as unusual, your perspective may be a little skewed.

Patrick Sharkey’s review puts their outlier status in the context of declining violent crime — including in Philadelphia’s most violent neighborhoods:

The decision to engage in violence can be thought of as a rational response to the pressures and threats that young men perceive in this environment. But the decision to fire a gun in public space is one that is now universally rejected by every segment of American society, and it is a decision that comes with clear, long-term consequences that are understood by all of the young men in the neighborhood that Goffman studies. … As callous as it sounds given the hardships faced by this young man and the lack of choices available to him in this situation, my sense is that most residents of the block where he chose to fire his weapon would consider attempted murder to be an appropriate charge and would have been happy if he were located by the police and sentenced to prison.

In a number of passages it’s impossible to differentiate what Goffman knows versus what she was told. When the facts are wrong, that’s unfortunate. For example, she writes, “By the time Chuck entered his senior year of high school in 2002, young women outnumbered young men in his classes by more than 2:1.” Was that his perspective? People are reliably bad at estimating group sizes. The actual ratio of women to men among Black 17-year-olds living below the poverty line and enrolled in school in Philadelphia was skewed, but only 1.65-to-1.

Nevertheless, her description of the many ways the incarceration empire impinges on the daily lives of poor Black people in Philadelphia is sometimes insightful and useful. I mostly agree with Goffman’s political description and conclusions about the injustices here, and if the book does some harm it also will do some good. I would be happy to see the volume come down in this discussion and for us to treat this is a regular book — despite the inordinate attention it gets outside of academia. It’s deeply flawed but it’s worth reading. Its systematic evidence is weak but it’s thought-provoking and offers lots of food for thought in research and policy debates. It’s well-written and its topic is important. I have nothing against her and look forward to what she will do next.

That said…

My extremely shallow expertise in qualitative research is unfortunately not balanced by a vast knowledge of survey research methods. But that is my relative strength here, and on that I’d like to register an objection to Goffman’s study — both the book and her 2009 article in American Sociological Review (official link here, Google for a free copy).

There is an understandable problem with reproducibility even in the best ethnographies. As evidence for her conclusions, I simply discount her ostensibly meticulous counting of events, like this:

In that same eighteen-month period, I watched the police break down doors, search houses, and question, arrest, or chase people through houses fifty-two times. Nine times, police helicopters circled overhead and beamed searchlights onto local streets. I noted blocks taped off and traffic redirected as police searched for evidence— or, in police language, secured a crime scene— seventeen times. Fourteen times during my first eighteen months of near daily observation, I watched the police punch, choke, kick, stomp on, or beat young men with their nightsticks.

To me this just means “a lot.” First, there is no denominator. That is, no way to gauge how prevalent these events were compared with anything else. “A lot” is a fine metric for this kind of observation, because all the insights come from the details that follow, not from the recitation of frequencies. (There also is a frustrating lack of precision in these passages. Consider: “I watched the police break down doors, search houses, and question, arrest, or chase people through houses fifty-two times.” What exactly happened 52 times? What do “and” and “or” mean in that sentence?) However, if you think of it as sociological memoir, those numbers mean something because they tell you about her evolving perspective and experience. Wow, I think, if I witnessed 14 police beatings first-hand that would really affect me.

That part of the study is not reproducible. But if you do a survey while you’re doing an ethnography, you’re still doing a survey. The survey is not an ethnography. That means you should (my opinion, not a rule) make available your instrument, methods employed, and the data. Goffman has said she destroyed her field notes so they couldn’t be subpoenaed, but I don’t see why that should apply to her survey data, which could have been stripped of identifying information and processed like any other survey with sensitive material. Unlike the recitation of event counts, the survey is potentially reproducible. She reports (a few) percentages. Other researchers could conceivably conduct similar research in a different place or time and draw useful comparisons; someone might even attempt to reproduce her survey just to see if they get the same result.

Let me back up. I can, without violating the copyright rules regarding quotation length, reproduce everything she wrote about the survey portion of her study. (The much remarked upon 50-page “methodological note” at the end of the book never mentions the survey.) Here I will compare what she said in ASR versus the book, not as a gotcha exercise but because there is so little information in either that the variation between the two may be informative.

In ASR (passages excerpted across several pages of the article):

The five blocks known as 6th Street are 93 percent Black, according to a survey of residents that Chuck and I conducted in 2007. … Of the 217 households surveyed, roughly one fourth received housing vouchers. In all but two households, members reported receiving some type of government assistance in the past three years. … In the survey that Chuck and I conducted in 2007, of the 217 households that make up the 6th Street neighborhood, we found 308 men between the ages of 18 and 30 in residence. Of these men, 144 reported that they had a warrant issued for their arrest because of either delinquencies with court fines and fees or for failure to appear for a court date within the past three years. Also within the past three years, warrants had been issued to 119 men for technical violations of their probation or parole (e.g., drinking or breaking curfew).

The footnote to that last passage clarifies that “in residence” doesn’t really mean living there:

I counted men who lived in a house for three days a week or more (by their own estimates and in some cases, my knowledge) as members of the household. I included men who were absent because they were in the military, at job training programs (like JobCorp), or away in jail, prison, drug rehab centers, or halfway houses, if they expected to return to the house and had been living in the house before they went away.

From this we learn that the survey included 217 households — and also that those 217 households make up the entire 6th Street neighborhood (not its real name). That seems to imply a 100% response rate, because the 217 households surveyed corresponds to the 217 that “make up” the neighborhood. (That is an excellent response rate.) That is also important because of what comes next. If there were 217 households in the neighborhood, but she had only done interviews with, say, half of them, it would have been very surprising to find 308 men ages 18-30 living there. As it is, it’s extremely unlikely there were 308 men 18-30 living in all 217 households. The footnote says she included men who only lived there part time, or who were away for prison or other institutional spells. Frustratingly, it doesn’t say how many in the sample this applied to.

Here’s what tripped me up about that: 308 men 18-30 in 217 households is 1.4 per household — too many, I thought. So I looked at the 45 census tracts in West Philadelphia that are 75% Black or more (based on this map) in the 2005-2009 American Community Survey, and calculated the average number of men in that age range as .57 per household. (Actually I did the estimate for 18-29 because of how the data are published.) There is one tract with 1.04 men per household, but that’s an outlier. If she’s getting almost 2.5-times more than the average in her sample (1.4 versus .57), there must be a lot of missing men living in the neighborhoods not counted by the Census Bureau. Besides wondering if that’s accurate, you also ought to wonder who answered the questions about those men who weren’t actually residing in their residences.

The book’s version, unfortunately, doesn’t help clear that up:

In 2007 Chuck and I went door to door and conducted a household survey of the 6th Street neighborhood. We interviewed 308 men between the ages of eighteen and thirty. Of these young men, 144 reported that they had a warrant issued for their arrest because of either delinquencies with court fines and fees or failure to appear for a court date within the previous three years. For that same period, 119 men reported that they had been issued warrants for technical violations of their probation or parole (for example, drinking or breaking curfew).

This says they interviewed all 308 men, which seems like just an editing mistake. But then of the 144 who “reported that they had a warrant,” was that out of 308 or out of some smaller number of men whom they actually interviewed? Did someone else answer for the men who weren’t there?

Later in the book she describes the women she interviewed in the same survey:

In our 2007 household survey of the 6th Street neighborhood, 139 of 146 women reported that in the past three years, a partner, neighbor, or close male relative was either wanted by the police, serving a probation or parole sentence, going through a trial, living in a halfway house, or on house arrest. Of the women we interviewed, 67 percent said that during that same period, the police had pressured them to provide information about that person.

If she had limited the sample of women to the same age range as the men (which there’s no indication she did) then you would expect, based on the Black West Philadelphia census tracts, about 1.2 men for every woman, or about 250 women in the sample. However, if she included women of all ages, it should have been even more. Why are there only 146 women? Without any more information — and there is no more information in the book — it’s impossible to figure this out.

The last thing you might like to know if you wanted to pursue these very interesting and potentially important results, is what the survey instrument was like, and how it was administered. This was more than 450 people, a decent size survey, with good potential to yield useful results. We know they asked race, housing and other government assistance, warrant status (including the type of warrant) for the men, and criminal justice status for the partner/neighbor/relative (asked separately?) of the women. What strikes me as challenging here is asking these very sensitive questions and getting such a high response rate. Especially given all we learn from the book, knocking on doors and asking people if they have any warrants seems like it wouldn’t always be welcome. How long were the interviews? Did they make multiple visits when people weren’t home? So knowing how they did it would be very helpful for future work.

In the end, besides what we learn from On the Run itself, I hope we learn from the debate over it how we can better balance the need for protecting research subjects — while learning a lot from them — with the imperative to conduct research that is transparent, verifiable, and (as much as possible) reproducible.

ADDENDUM: The importance of the survey

The survey plays a very small part in the book and ASR article, only mentioned a few times, but it is important because it does offer a  hint of generalizability to her research. James Forman wrote in the Atlantic:

The fact that Goffman’s subjects have serious criminal histories impairs our ability to generalize from some of her findings. For example, her central characters are all wanted on warrants at one time or another, some of them repeatedly (Mike has 10 warrants altogether). To Goffman, this indicates that Philadelphia’s criminal-justice system issues too many warrants. But it may simply indicate that Mike and his friends are unusually criminally active.

Perhaps anticipating this challenge, Goffman extends her inquiry beyond the most criminally active members of the community. When she conducts a door-to-door survey of Sixth Street, she finds that about half the men there were wanted on warrants over a three-year period. This is astounding; no previous researcher has reported such a high concentration of fugitives living in one community. This raises questions that Goffman doesn’t answer with precision, but that I hope she and others will explore in the future: How many of these warrants were for failure to pay court costs—which should rarely if ever be imposed on poor people in the first place—versus something more serious, such as skipping a court date? Does fugitive status affect the lives of less criminally involved young men in the same ways it affects the lives of Mike and his friends? If it does, and if other communities harbor equally large proportions of fugitives, Goffman has discovered a profound social problem that deserves further research and a policy response.

The survey thus represents an important avenue for the book’s impact on future research.

ADDITIONAL ADDENDUM: Goffman’s response

In a statement on her website, Goffman responds to several of the recent criticisms. The first is that she participated in a conspiracy to commit murder. Her description there doesn’t change my impression of the scene she describes. But it is interesting for what it reveals about her sense of the men at the core of her story: Chuck and Mike. She writes:

In the months before he died, Chuck was actively working to preserve a precarious peace between his friends and a rival group living nearby. His sudden death was a devastating blow not only to his friends and family, but to the whole neighborhood. After Chuck was shot and killed, people in the neighborhood were putting a lot of pressure on Mike and on Chuck’s other friends to avenge his murder. It seemed that Chuck’s friends were expected to fulfill the neighborhood’s collective desire for retribution. Many of the residents were emphatic that justice should be served, and the man who killed Chuck must pay.

This surprises me, because I did not get the sense from the book that the gang rivalry Goffman described had motivated a collective conflict between entire neighborhoods. That contradicts the comment from Patrick Sharkey above, which speculates that most people in the neighborhood would have been glad to see the police arrest and incarcerate the people who conduct running gun battles in the street. Maybe Skarkey and I are victims of do-gooder liberal attitudes, and really there are neighborhoods in Philadelphia that see themselves at war with whole other neighborhoods and clamor for more shooting.

As an aside, I recently read Ghettoside: A True Story of Murder in America, in which journalist Jill Leovy argues that poor Black communities are at once over-policed — as innocent people are harassed, violated, prosecuted, and incarcerated for minor (or no) offenses — and under-policed such that murderers in Black communities are much less likely to be arrested and convicted than are those who kill Whites. This latter neglect by police contributes to the real problem of violence because it encourages informal (violent) means of addressing conflict in the community. By Leovy’s reasoning, and Sharkey might agree, majorities of people in violent Black neighborhoods would like fewer drug users incarcerated, but more of the people like Mike and whoever killed Chuck incarcerated.

Of course, I don’t know the situation in the “6th Street” neighborhood. But it’s one thing for an activist or community leader to say, “Our neighborhood stands united!”, and another for a sociologist to speak of “the neighborhood’s collective desire.” The former may be effective politics, but the latter is likely an overly-simplified research conclusion.

Which brings me finally back to the issue of Goffman’s survey. In her response, Goffman says Chuck was actively working on neighborhood peace in the months before his death, which occurred in the summer of 2007. Remarkably, that is the same time that she and Chuck conducted their survey of all 217 households in the neighborhood, with a reported 454 interviews. The timing of the survey is not completely clear, but she wrote that it was in 2007. and in an interview she said the survey was in the summer (“Yeah, so we did this survey, Chuck and I, one summer. We interviewed the households in this four-block radius…”). So it’s either a head-scratching story or an incredibly impressive image: Chuck and Goffman conducting a door-to-door household survey at the same time that he’s negotiating a delicate gang truce, interviewing every single household in the neighborhood in the weeks leading up to Chuck’s death, gathering the information that would allow Goffman to in fact accurately speak of their “collective desire.”

39 Comments

Filed under Research reports

New data on gender-segregated sociology

Four years ago I wrote about the gender composition of sociology and the internal segregation of the discipline. Not much has changed, at least on the old measures. Here’s an update including some new measures (with some passages copied from the old post).

People may (or may not) want to be sociologists, they may or may not be accepted to graduate schools, thrive there (with good mentoring or bad), freely choose specializations, complete PhDs, publish, get jobs, rise to positions of leadership, and so on.  As in workplaces, gender segregation in academic sociology represents the cumulative intentions and actions of people in different institutional settings and social locations. It’s also the outcome of gender politics and power struggles. So, very interesting!

A report from the research folks at the American Sociological Association (ASA) got me thinking about this in 2011. The conversation revived the other day when someone asked ASA Vice President Elect Barbara Risman (a friend and colleague of mine), “What do you make of the fact that increasingly the majority of ASA election candidates tend to be women?” As we’ll see, the premise may be wrong, but the gender dynamics of ASA are interesting anyway.

#1: ASA leadership

The last four people elected president of ASA have been women (Ruth Milkman, Paula England, Annette Lareau, and Cecilia Ridgeway), and the the next winner will be either Michele Lamont or Min Zhou, both women. That’s an unprecedented run for women, and the greatest stretch of gender domination since the early 1990s, when men won six times in row. Here is the trend, by decade, starting with the decades before a woman president, 1906 through the 1940s:

sociology segregation.xlsx

Clearly, women have surpassed parity at the top echelons of the association’s academic leadership. ASA elections are a complicated affair, with candidates nominated by a committee at something like two per position. For president, there are two candidates. In the last nine presidential elections, six have featured a man running against a woman, and the women won four of those contests. So women are more than half the candidates, and they’ve been more likely to win against men. That pattern is general across elected offices since 2007 (as far back as I looked): more than half the candidates are women, but even more women win (most elections have about 36 candidates for various positions):

sociology segregation.xlsx

The nominating committees pick (or convince) more women than men to run, and then the electorate favors the women candidates, for reasons we can’t tell from these data.

These elections are run in an association that became majority female in its membership only in 2005, reaching only 53% female in 2010. That trend is likely to continue as older members retire and the PhD pool continues to shift toward women.

#2: Phds

Since the mid-1990s, according to data from the National Science Foundation, women have outnumbered men as new sociology PhDs, and we are now approaching two-thirds female. (The data I used in the old post showed a drop in women after 2007, but with the update, which now comes from here, that’s gone.)

sociology segregation.xlsx

Producing mostly-female PhDs for a quarter of a century is getting to be long enough to start achieving a critical mass of women at the top of the discipline.

#3: Specialization

These numbers haven’t been updated by ASA since 2010. The pattern of section belonging at that time showed a marked level of gender segregation. On a scale of 1 to 100, I calculate the sections are segregated at a level of .25.

sociology segregation.xlsx

#4: Editors and editorial boards

Finally, prestigious academic journals have one or more editors, often some associate editors, and then an editorial board. In sociology, this is mostly the people who are called upon to review articles more often. Because journal publication is a key hurdle for jobs and promotions, these sociologists serve as gatekeepers for the discipline. In return they get some prestige, the occasional reception, and they might be on the way to being an editor themselves someday.

Journal leadership is dragging behind the trends in PhDs, ASA members, and ASA leadership. I selected the top 20 journals in the Sociology category from the Journal Citation Reports (excluding a few misplaced titles), plus Social Problems and Social Forces, because these are considered to be leading journals despite low impact factors. The editors of these journals are 41% female (or 40% if you use journals as the unit of analysis instead of editors). Here is the list in two parts — general journals and specialty journals — with each sorted by impact factor. For multiple editors I either list the gender if they’re all the same, or show the breakdown if they differ:

Book12

It looks like the gender gap is partly attributable to the difference between journals run by associations and those run as department fiefdoms or by for-profit publishers.

For editorial boards, I didn’t do a systematic review, but I looked at the two leading research journals — American Sociological Review and American Journal of Sociology, as well as two prestigious specialized journals — Sociological Methods and Research, and Gender and Society (which is run by its own association,Sociologists for Women in Society, whose membership includes both women and men). Here’s the update to my 2011 numbers:

sociology segregation.xlsx

I removed a couple board members I know to have died in the last year, so these lists might not be that up to date.

Note on the journals that SMR and AJS are fiefdoms with no accountability to anyone outside their cliques, so it’s not surprising they are decades behind. ASR and G&S, on the other hand, are run by associations with majority-female memberships and hierarchies, in the case of G&S with a feminist mission. (ASA demands reports on gender and race/ethnicity composition from its editors.) AJS has no excuse and should suffer opprobrium for this. SMR might argue they can’t recruit women for this job (but someone should ask them to at least make this case).

5 Comments

Filed under Uncategorized

Stop me before I fake again

In light of the news on social science fraud, I thought it was a good time to report on an experiment I did. I realize my results are startling, and I welcome the bright light of scrutiny that such findings might now attract.

The following information is fake.

An employee training program in a major city promises basic job skills and as well as job search assistance for people with a high school degree and no further education, ages 23-52 in 2012. Due to an unusual staffing practice, new applications were for a period in 2012 allocated at random to one of two caseworkers. One provided the basic services promised but nothing extra. The other embellished his services with extensive coaching on such “soft skills” as “mainstream” speech patterns, appropriate dress for the workplace, and a hard work ethic, among other elements. The program surveyed the participants in 2014 to see what their earnings were in the previous 12 months. The data provided to me does not include any information on response rates, or any information about those who did not respond. And it only includes participants who were employed at least part-time in 2014. Fortunately, the program also recorded which staff member each participant was assigned to.

Since this provides such an excellent opportunity for studying the effects of soft skills training, I think it’s worth publishing despite these obvious weaknesses. To help with the data collection and analysis, I got a grant from Big Neoliberal, a non-partisan foundation.

The data includes 1040 participants, 500 of whom had the bare-bones service and 540 of whom had the soft-skills add-on, which I refer to as the “treatment.” These are the descriptive statistics:

fake-descriptives

As you can see, the treatment group had higher earnings in 2014. The difference in logged annual earnings between the two groups is significant at p

fake-ols-results

As you can see in Model 1, the Black workers in 2014 earned significantly less than the White workers. This gap of .15 logged earnings points, or about 15%, is consistent with previous research on the race wage gap among high school graduates. Model 2 shows that the treatment training apparently was effective, raising earnings about 11%. However, The interactions in Model 3 confirm that the benefits of the treatment were concentrated among the Black workers. The non-Black workers did not receive a significant benefit, and the treatment effect among Black workers basically wiped out the race gap.

The effects are illustrated, with predicted probabilities, in this figure:

fake-marginsplot

Soft skills are awesome.

I have put the data file, in Stata format, here.

Discussion

What would you do if you saw this in a paper or at a conference? Would you suspect it was fake? Why or why not?

I confess I never seriously thought of faking a research study before. In my day coming up in sociology, people didn’t share code and datasets much (it was never compulsory). I always figured if someone was faking they were just changing the numbers on their tables to look better. I assumed this happens to some unknown, and unknowable, extent.

So when I heard about the Lacour & Green scandal, I thought whoever did it was tremendously clever. But when I looked into it more, I thought it was not such rocket science. So I gave it a try.

Details

I downloaded a sample of adults 25-54 from the 2014 ACS via IPUMS, with annual earnings, education, age, sex, race and Hispanic origin. I set the sample parameters to meet the conditions above, and then I applied the treatment, like this:

First, I randomly selected the treatment group:

gen temp = runiform()
gen treatment=0
replace treatment = 1 if temp >= .5
drop temp

Then I generated the basic effect, and the Black interaction effect:

gen effect = rnormal(.08,.05)
gen beffect = rnormal(.15,.05)

Starting with the logged wage variable, lnwage, I added the basic effect to all the treated subjects:

replace newlnwage = lnwage+effect if treatment==1

Then added the Black interaction effect to the treated Black subjects, and subtracted it from the non-treated ones.

replace newlnwage = newlnwage+beffect if (treatment==1 & black==1)
replace newlnwage = newlnwage-beffect if (treatment==0 & black==1)

This isn’t ideal, but when I just added the effect I didn’t have a significant Black deficit in the baseline model, so that seemed fishy.

That’s it. I spent about 20 minutes trying different parameters for the fake effects, trying to get them to seem reasonable. The whole thing took about an hour (not counting the write-up).

I put the complete fake files here: code, data.

Would I get caught for this? What are we going to do about this?

BUSTED UPDATE:

In the comments, ssgrad notices that if you exponentiate (unlog) the incomes, you get a funny list — some are binned at whole numbers, as you would expect from a survey of incomes, and some are random-looking and go out to multiple decimal places. For example, one person reports an even $25,000, and another supposedly reports $25251.37. This wouldn’t show up in the descriptive statistics, but is kind of obvious in a list. Here is a list of people with incomes between $20000 and $26000, broken down by race and treatment status. I rounded to whole numbers because even without the decimal points you can see that the only people who report normal incomes are non-Blacks in the non-treatment group. Busted!

fake-busted-tableSo, that only took a day — with a crowd-sourced team of thousands of social scientists poring over the replication file. Faith in the system restored?

11 Comments

Filed under In the news, Research reports