Tag Archives: social science

What’s with the historical trend juxtapositions?

When is it OK to juxtapose historical trends?

You have to watch out for this (via Boing Boing):

There is no reason to suspect that the rise in autism is linked to the rise in organic food sales.

But other times it seems reasonable to me, like this:

fert-wlfp-trend

There are lots of reasons the long-run decline in fertility is related to the rise in women’s employment rates. We know from lots of research that women with more children are less likely to have jobs; women with jobs are less likely to have children; and over time the proportion of women in the second group has grown relative to the first.

So it’s OK to use eyeballed historical trends when you have good research backing up the association. Your conclusions, then, don’t rely on the simple trend comparison. The trends are an illustration.

But trends need not have have a simple cause-and-effect relationship — or a unidirectional relationship — for it to be important to compare them. Sometimes the relationship is just descriptively important. So, looking at the graph above, it would be reasonable to say, “Women’s lives sure have changed. They have fewer children and more jobs, on average, than they used to.”

And then there is the negative case. The other day I complained when Kay Hymowitz implied that the rise in father-absent families caused an increase in crime among boys. And I offered this simple trend comparison to undermine that story:

It was not my intention to say there is no connection between father absence and boys’ criminal behavior. (I’ve sketched out some possible links in this old post; and made essentially the same comparison about single mothers and crime before.) But the lack of a strong correlation in the trends over time is a challenge. That’s what I’ve been arguing about cell phones and traffic accidents:

Of course driving and texting is dangerous. And of course single parents have a harder time (on average) supervising and disciplining their children than married parents. But if there is a big discordance between the trends — texting and driving, single parents and crime — then that’s a problem for the story that one trend is driven by the other. Causal relationships may be apparent in a lab, or at the margins, but to explain large-scale social change is more difficult — and that’s often what we’re trying to do when we draw from specific research to make political, policy, or theoretical arguments.

So, it’s OK to use discordant trends to take potshots at a proposed causal story, to express skepticism. The discordant trends are a hurdle for the theory to overcome.

If you have good research showing that single parenthood, and especially father absence, is harming boys more than girls, then it would be to OK to use trends as an illustration. It just can’t be your main evidence. So Kay Hymowitz could reasonably include a graph like this to accompany her extensive review of the research on family structure and trouble for boys:

hymowitz-response2

Yes, women’s advantage in high school and college completion has accompanied the trend toward father-absent living arrangements for young boys. That doesn’t fulfill her need to present more direct evidence, but it’s a piece of supporting evidence.

Conclusion: Juxtaposing historical trends is not how you prove a theory. It is a great tool for illustrating known associations, for describing social change, and for challenging theories or narratives.

6 Comments

Filed under Uncategorized

Regnerus delivers deceptive dialogue to delighted devotees

Not rehashing the whole thing, just a brief field report.

regnerus-10-4-2013

Grainy hidden-camera footage of shadowy propagandist.

I happened to hear Mark Regnerus deliver a lecture yesterday, titled, “Children and Sociology under Fire: Conducting Research on Same-Sex Parenting.”

The talk was sponsored by something called the Pontifical Studies John Paul II Institute for Studies on Marriage and Family at the Catholic University of America, but you might know it as the Mark Regnerus fan club. In the talk, he presented his paper from last year (read all about it under the Regnerus tag) to a fawning audience of, let’s just say, mostly White Catholics.

I went to see how he would present it (very boringly), what he might say about the controversy (nothing substantive), and whether there would be any pushback (if anyone objected, they did so silently).

I came away with three observations about his presentation (these were all apparent in the paper — as Andrew Perrin, Neal Caren and I wrote — but seeing him present it showed how they work on a receptive audience).

First, he crudely manipulated his audience’s ignorance of research methods. He first described in some detail the “standard set of controls” he used to test the relationship between having a father or mother who ever (reportedly) had a same-sex romantic relationship and his many negative outcome variables. And then he proceeded to present bivariate relationships as if they were the results of those tests. He didn’t say they were adjusted, but everyone thought the results he showed were controlling for everything.

For example, to gasps from the crowd, he revealed that 17 percent of “intact bio family” kids had ever received welfare growing up, compared with 70 percent for those whose mother (reportedly) ever had a same-sex romantic relationship. If you don’t realize that this is mostly just a comparison between stable married-couple families and single-mother families, that might seem like a shockingly large effect.

Second, Regnerus pitched his audience the simplistic version of “reduced kinship” theory from the paper, in which people caring for children naturally care less about them if they are not biologically related. This was quite self-evident to the audience, so he didn’t dwell on it. Then he arranged his unadjusted means more or less from best to worst, left to right, from “intact bio” on the left, to adopted, widowed, divorced, separated, single-mother-divorced, single-mother-never-married, and finally “mother lesbian relationship” on the right.

“As you move left to right,” he explained with regard to one variable, “you see the challenges facing these young adults increase.” With another variable he said, “If you graph this, you sort of see a big surge as you get to reduced kinship.”

What you actually get as you move along that row, of course, is not reduced kinship (after all, adopted were just next to “intact bio” on the left side), but rather increasingly female-headed. But the writing on the slide was small, the groups were poorly defined, and the “reduced kinship” narrative was pertinent.

Finally, Regnerus made a few explicit statements about how his study was not intended to show causality (and insisted that he intended it to have no impact on public policy, that he only wrote a brief against marriage equality because he was sure other brief-writers would misrepresent the true science regarding same-sex parents). But he tipped his hand grotesquely, making an argument I don’t remember tucked away in the original paper.

As you might remember, in the results there are more negative outcomes for the children of “mother lesbian relationship” than for the “father gay relationship” children. This is logical since you’re talking about female-headed versus male-headed families. In the paper Regnerus wrote:

While I have so far noted several distinctions between IBFs and GFs—respondents who said their father had a gay relationship—there are simply fewer statistically-significant distinctions to note between IBFs and GFs than between IBFs and LMs, which may or may not be due in part to the smaller sample of respondents with gay fathers in the NFSS, and the much smaller likelihood of having lived with their gay father while he was in a same-sex relationship (emphasis added).

I don’t think I noticed that at the time. Or maybe I discounted it because of the “may or may not be due in part to” language. But now he’s quite a bit more sure. He told the Catholic audience:

There are fewer differences when we look at adult children of fathers who had a gay relationship. I don’t encourage you to read too much into this, except to say that there are fewer cases of them. Those for whom this was true were less likely, of course, to have lived with their father [and his partner]. Plenty of people did not live with their dads, so it would be not surprising to see they have fewer distinctions. Plenty of them did not live with this person.

It came up again in the Q&A, when David Crawford asked:

Q: “… adverse outcome that we’re finding also more centered in the lesbian mothers rather than the gay father relations. I also found that to be interesting. Is that a question of the children tending to spend more time with the mothers and being exposed to whatever causes…”

A [interrupting]: “That would be my best guess.”

So, although he would never dream of making causal claims, his “best guess” for why children living with their mothers rather than their fathers had tougher childhoods is because they spent more time with a lesbian in the household (a very small amount of time, compared to almost none for the children of suspected-gay dads). As Crawford put it, “being exposed to whatever causes…” Or, although he would never dream of making causal claims, his “best guess” for the pattern he sees is that one group was more exposed to the treatment effect he was ostensibly not testing.

As for the whole subject of the hostile reaction from academia, that just boiled down to a few moments of saying that social scientists are all liberals, that the truth can’t come out because real scientists are afraid to touch controversial subjects, and that he’s suffered unbearably as a result of his devotion to the truth. The applause was strident.

Anyway, I also discovered that attempting to sketch him gets harder the more times you try.

regnerus-sketch

Leave a comment

Filed under In the news

Teachers might help students finish high school

In Guns, Germs and Steel: The Fates of Human Societies, Jared Diamond presents a story of world history in which the rise to power of some societies over others was driven by geographic and environmental dynamics.

Some of the stories involve simple chains of evidence like this: societies with more fertile territories settled down earlier and had greater population density, which led to more complex politics, including elites who controlled surpluses and then diverted resources to non-food producers such as artists, craftspeople and bureaucrats, resulting in the development of more advanced language and technology.

We rarely have such large-scale analyses of current social dynamics. One reason contemporary analyses is so much more complicated is because the variations involved are much more subtle and the time frames are much shorter. For example, trying to explain women’s market labor force participation rates that mostly vary between 70% and 85% across rich countries. Or looking at small, marginal effects that tell us something interesting without explaining what really drives the larger outcomes. For example, I found that 34% of mothers with girls named pink or purple as their favorite color, compared with 45% among those with boys — a subtle effect on a minor personality trait. Similarly, we learned recently that boys with sisters are more gender-traditional. But the effect is within a narrow range, and certainly smaller than other more fundamental determinants of gender attitudes.

As I listened to the audio version of Diamond’s book, I tried to imagine how he might describe some recent changes in the U.S., treating differences between states over recent decades as the outcome of such fundamental processes. Here’s one thing I think he might have said:

Those states that assigned greater numbers of people to the task of teaching children more rapidly produced populations with higher levels of education. In half a century, states tripled the number of teachers per student from about 5 per 100 to about 15 per hundred. As a result the percentage of young adults dropping out of high school fell by about half, from near 60% to about 30%.

Here is my figure, plotting teachers per student (age 6-14) against dropout rates (age 16-24) 10 years later:

teachers per kid and dropouts

Source: My analysis of Census data from IPUMS.org.

Is that a fair characterization of the history? Or does it overwrite fundamental variation or complexity and lead to the wrong story?

I want more simple stories, and I would also like them to be true.

4 Comments

Filed under In the news

Fundamentally opposed to science?

Conservative religious fundamentalists really don’t trust the scientific establishment.

In the discussion of academia’s liberalism, we should also consider the public’s mistrust of science, especially the conservative and fundamentalist public. Why would people who don’t trust science become scientists?

Last year Gordon Gauchat reported in American Sociological Review that Americans’ trust in the scientific community was holding steady except for political conservatives and those who attend church regularly, and that the trend was not explained by the lower education levels of conservatives or religious people (in fact, educated conservatives expressed the lowest levels of trust in science). His conclusion was that the trend showed the politicization of science, which is not the way modernity is supposed to go.

In response, Darren Sherkat blogged that Gauchat underestimated the importance of religion in explaining conservatives’ opposition to science because he only used the General Social Survey’s measure of the frequency of religious attendance instead of a measure of beliefs. And he provided a chart from the GSS showing that religious fundamentalists had lower trust in science whether they were Republicans or not. Sherkat wrote:

Any social scientist who studies politics, religion, and science should know that the reason why Republicans are at war against science is to court the vote of fundamentalist Christian simpletons who are opposed to science and reason. … What drives Republican opposition to science is that more Republicans are fundamentalists who believe that the Bible is the literal word of god.

You got your fundamentalism in my conservatism

As I look at it, conservatism and fundamentalism are both at fault. My take on the trends shows that, in addition to the growing divide between politically conservative fundamentalists and politically liberal non-fundamentalists, liberal fundamentalists have grown more trusting of science, while conservative non-fundamentalists have grown less trusting.

I used the GSS from 1974 through the latest 2012 survey. To highlight the polarization I show only those who are “extremely liberal,” “liberal,” “conservative,” or “extremely conservative,” leaving out those who are “slightly” liberal or conservative, or moderate. So this is not the whole population (I’ll return to that below).

The question was:

I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them? … Scientific community.

It’s as close as we get to a question about science itself. For fundamentalism, GSS asked whether the respondent’s religion was fundamentalist, moderate, or liberal. I dichotomized it to fundamentalists versus everyone else (including people with no religion).*

These are the people expressing a great deal of confidence in the scientific community:

confidence-in-scienceThese trends are heavily smoothed (down to four decades), because the numbers bounce around a lot from year to year, as the samples are only between 60 and 220 in each cell in the individual years. To do a simple test of the trends, I ran a regression using time and interactions between time and politics-fundamentlism dummy variables, with controls for age and sex (old people and men hate science more than regular people, net of religion and politics).

The regression confirms what the graph shows: significant declines in trust among conservatives whether fundamentalist or not, and an increase in trust among liberal fundamentalists. The trend for liberal non-fundamentalists was flat. (Details on request.)

I left out of that analysis the people who were slightly conservative, moderate, or slightly liberal. That’s a shrinking majority of the population, which breaks down like this from the 1970s to the last decade (click to enlarge):

confidence-in-science-popsSo the bad news for science is that the increasingly anti-science groups are increasing in the population: conservative fundamentalists and non-fundamentalists. The big green majority is not growing more or less anti-science (even when you break it down by fundamentalism), but it’s also shrinking. The liberal fundamentalists are getting more into science, but also vanishing.

Just wait till they find out (some) sociology is part of the “scientific community.”

Note: This is a blog-post, not peer-reviewed research. I might be wrong.

* Skerkat instead uses a question about how to interpret the Bible instead of the fundamentalism question (literal word of God, inspired word of God, book of fables). 95% of the people who described themselves as having a “fundamentalist” describe the Bible as either the literal or the inspired word of God.

 

 

37 Comments

Filed under Research reports

Responses on fatherhood: hormones, science and god

The fatherhood post yesterday has gotten (for this blog), a lot of readers and some interesting responses. As I wrote out some extended, disorganized comment responses, I realized I may as well elevate them to an independent post (still a disorganized rant though).

I like the discussion by the authors on the Scientific American blog suggested by szopeno. Like I said in the original post, it’s quite reasonable that caring behavior affects hormone levels, as we know things like stress and fear do as well, with all kinds of mental and physical effects. If you randomly subjected some people to competitive athletic coaching, and handed others an infant, I wouldn’t be surprised to see the competition people behaving more aggressively and the baby-holders being more nurturing on average three months later. That would be interesting.

What is the implication? Are we shocked that some aspects of fatherhood (or childcare or sex) provoke a “biological” response? If that shocks you, you might like to know that by simply showing people pictures of other people behaving in certain ways, their bodies are are more likely to undergo spontaneous physical transformations. Just from sitting there looking at pictures! Also, if you inject an athlete with testosterone he can ride his bike really fast.

Schwarzenegger

It does not follow from these findings of a hormonal response to life events that we should promote certain family arrangements as “natural,” which is where Wilcox and the religious-sociological-complex is taking this. If the goal is to change men’s testosterone levels, that might be done with medication. If the goal is to reduce aggressiveness, try teaching meditation in public schools. If we want people to be better parents, we can give them jobs, healthcare, housing and childcare support.

We have lots of ways of trying to promote happiness and pro-social behavior. However, like the crazy list of potential risks and side effects for men taking low-T medication, there are consequences to any such intervention.

Fortunately for individual freedom and human rights, some of us know that we can punish or prevent bad behavior — and reward or encourage good behavior — without attacking or rewarding whole status categories of people. Children with rich, married, college-educated parents are more likely to get into and finish college. So, we ought to fund a public school system, fund student loans for college — and also protect the children of the evil, sick or ineffective rich, married, college-educated parents from harm. But that doesn’t mean we should sterilize poor people.

So, is fatherhood good?

It’s not a question with one answer. One of the things Wilcox and the family “gold standard” promoters do is find ways that people in “traditional” families are doing better on average and use that to promote family conformity. But the averages conceal the sources of variation. Comparing the average father to the average non-father won’t tell you much about how fatherhood affects men because fatherhood occurs along with so many combinations of other transitions, experiences and resources. If you randomly assigned fatherhood to random men — at random moments in their lives — you could come up with an answer. Otherwise I’m not optimistic, and if it’s not answerable I doubt it’s a good question for social science.

Imagine three sets of outcomes: money, happiness and healthiness. Each is affected by social background and context. Then consider men entering fatherhood with different levels of each beforehand, and see how each outcome changes for all the different combinations (e.g., income changes for rich, happy, healthy; income changes for poor, happy, healthy; etc.). The possibilities multiply. If you’re Brad Wilcox you can work back from your goal — married nuclear families — and compare them to everyone else to cherry-pick any worse outcome at any time, and lo, discover that the Bible was right after all. If you really want to know it’s not so easy.

I haven’t yet read Doing the Best I Can: Fatherhood in the Inner City, the new book by Kathryn Edin and Timothy J. Nelson, but that seems promising for an in-depth look at fatherhood in the flow of men’s lives, with a lot of attention to the social context (education, employment, incarceration, complex families and relationships, etc.).

candle

Don’t take my Word for it

If you start from a God-given definition of what’s good, and science can’t change that, then science becomes just a convenient way of explaining what you already knew, which is not science — it’s what the Church calls “natural law.”

Wilcox denies that’s how it works, naturally. At a conference on the family and natural law, he was quoted as saying,

Our support for the renewal of marriage is not predicated on some … religious worldview. Rather, it’s based on a reasonable understanding of the human condition that is accessible to all men and women of good will. … Evidence suggests to us that intact, biological marriage is still the gold standard.

That depends on what you mean by “predicated.” Years before the “Regnerus affair,” during which Mark Regnerus joined Wilcox in a scheme to use science against marriage equality in the courts, Regnerus gave his view of the importance of (a certain kind of) marriage, and it did not originate from his scientific training:

The importance of Christian marriage as a symbol of God’s covenantal faithfulness to his people—and a witness to the future union of Christ and his bride—will only grow in significance as the wider Western culture diminishes both the meaning and actual practice of marriage. Marriage itself will become a witness to the gospel.

That divine law and natural law do not conflict is an article of faith (literally). I think Pope Leo XIII put it well when he wrote:

Now, reason itself clearly teaches that the truths of divine revelation and those of nature cannot really be opposed to one another, and that whatever is at variance with them must necessarily be false. Therefore, the divine teaching of the Church, so far from being an obstacle to the pursuit of learning and the progress of science, or in any way retarding the advance of civilization, in reality brings to them the sure guidance of shining light.

Thus, rather than see science as a candle in the dark, natural law says that science needs a candle in the dark, and God has one. Could any research penetrate that mindset? If your research contradicts the “truths of divine revelation” then your research is wrong. Try again! Science in this vein is just looking for ways to convince secular society that the Church is already right. It is a self-fulfilling prophecy (literally). From the same natural law conference news report:

While there’s limited data on the effects of same-sex marriage on children, Wilcox hypothesized that in a few years, research will show that children in lesbian or gay family situations will exhibit some of the same problems as children from father-less or cohabiting relationships.

That conference was in January 2011. At that point Wilcox already had the New Family Structure Study machinery in motion, which would end up confirming to the faithful what they already knew.

4 Comments

Filed under In the news

What is the logic of marriage denial?

And this should wrap up Homogamy Week here at Family Inequality…

I am confused by the logic in the arguments against extending marriage rights to gay and lesbian couples.

I earlier pointed to an essay I agree with, arguing that denying marriage rights on the basis of a child well-being argument is wrong-headed. And that’s even before we got the excellent review from the American Sociological Association reaffirming that homogamous-couples cause no demonstrable harm to children.

But now on the other side, Mark Regnerus and his colleagues have submitted a stunning brief for the Supreme Court’s upcoming marriage-rights cases. In it they argue that man-woman parents are best for children, but also that there are too many unanswered questions to draw any firm conclusions about child well-being in gay- and lesbian-parent families — so therefore the Defense of Marriage Act and California’s Proposition 8 should be upheld.

Back up

When Regnerus’s awful study came out — purporting to show (but not actually showing) that children of gay and lesbian parents were worse off then those in “intact biological families” — he disavowed any political intent or implications. When he interviewed himself, it went like this:

Q: Is there a political take-home message in the study?

A: No. As I stated in the article, “this study cannot answer political questions about same-sex relationships…”

Q: Come on. You can’t surmise what people will make of this study politically?

A: You know, I don’t think it easily lends itself to one particular answer to any of the politicized questions that are circulating about gay marriage, or parental rights, etc.

And when he wrote in Slate, he offered an ostensibly even-handed interpretation:

The political take-home message of the NFSS study is unclear, however. On the one hand, the instability detected in the NFSS could translate into a call for extending the relative security afforded by marriage to gay and lesbian couples. On the other hand, it may suggest that the household instability that the NFSS reveals is just too common among same-sex couples to take the social gamble of spending significant political and economic capital to esteem and support this new (but tiny) family form while Americans continue to flee the stable, two-parent biological married model, the far more common and accomplished workhorse of the American household, and still—according to the data, at least—the safest place for a kid.

In fact, Regnerus and his defenders were incensed that he was being treated as if his motives were political. And in his own defense he wrote of the original paper: “Some perceive it as a tool for this or that political project, a role it was never designed to fill. It cannot answer political or legal questions…”

That was then. So now to the amicus brief filed by Regnerus and several other social scientists. Their review of the evidence is irrelevant to their argument, because they conclude that we don’t know enough to draw any empirical conclusions. Still,

With so many significant outstanding questions about whether children develop as well in same-sex households as in opposite-sex households, it remains prudent for government to continue to recognize marriage as a union of a man and a woman, thereby promoting what is known to be an ideal environment for raising children. Marriage is the legal means by which children are stably united with their biological mothers and fathers and poised for optimal development. Opposite-sex parenting allows children to benefit from distinctive maternal and paternal contributions. Given these facts, safeguarding marriage is a liberty to be accorded to children at least as much as to their parents.

So man-woman parenting is good, OK. And we don’t know enough to say anything about gay and lesbian parenting causally. Therefore it is “prudent” to deny marriage rights to gay and lesbian couples, because “safeguarding marriage is a liberty to be accorded to children.”

That is, it was right to tax Edith Windsor $600,000 more because her spouse was female. For the children.

Edith Windsor

I really hope Regnerus gets a chance to testify as an expert on all this someday, because I’d love to hear more about this logic under cross examination.

22 Comments

Filed under Uncategorized

American Sociological Association releases SCOTUS brief

The American Sociological Association has released its long-awaited amicus brief on the same-sex (homogamy) marriage cases currently before the Supreme Court.

In the association’s press release, ASA President Cecilia Ridgeway is quoted as saying:

The results of our review are clear. There is no evidence that children with parents in stable same-sex or opposite-sex relationships differ in terms of well-being. Indeed, the greater stability offered by marriage for same-sex as well as opposite-sex parents may be an asset for child well-being. An issue at the heart of these cases is whether family composition, per se, affects the well-being of children and thus, provides a justification for limiting the right to marry. This core question is an empirical one and is the subject of a broad range of social science research. As a scientific body, ASA has a duty to provide the court with a systematic and balanced review of the evidence to assess what the consensus of scholarly research has shown.

The impetus for ASA action was the publication and aggressive dissemination of a paper by sociologist Mark Regnerus, with funding from right-wing foundations, that purported to show negative outcomes for children of gay and lesbian parents. The fallout included a letter denouncing the study signed by 200 researchers, an investigation into the review process by the journal, and a special issue dedicated to the controversy. (My review of the events through last August is, with links, is here; the special issue is here.)

Out of the discussion a number of sections within the association asked the ASA Council to commission a brief, a decision that was controversial among sociologistsSome people expressed reluctance to have the association offer a substantive conclusion about ongoing research, or to cast judgment on peer-reviewed work. There is a normal amount of anxiety over whether the association should take public positions perceived as political.

My opinion is that the brief, under the research direction of Wendy Manning, a top-notch researcher with a non-ideologue reputation, strikes the right balance and should allay most sociologists’ concerns. Of course it is argumentative in a way that the most peer-reviewed work is not – and doesn’t go out of its way to represent its opponents’ views. But the strong conclusion is well-justified: on the issue of same-sex parenting and children’s well-being, we have the closest thing to a (social) scientific consensus we could hope for:

When the social science evidence is exhaustively examined—which the ASA has done—the facts demonstrate that children fare just as well when raised by same-sex parents … Unsubstantiated fears regarding same-sex child rearing do not overcome these facts and do not justify upholding DOMA and Proposition 8.

And it correctly points out the biggest problems with the Regnerus study, its wrongheaded setup and comparisons, and its misleading interpretations.

In short, I highly recommend the brief as an excellent piece of scientific writing, and a model for argument from evidence. And I like to see statements that include the phrase, “As a scientific body, ASA has a duty to…”

12 Comments

Filed under Uncategorized

Take it from the Pope

pontificating

For the “World Day of Peace,” which is today, instead of congratulating the newly weds – who are upholding the transformed but still living (for better or worse, in sickness and in health) institution of marriage – Pope Benedict (Ratzinger) issued a statement that included this about homogamous marriage:

There is also a need to acknowledge and promote the natural structure of marriage as the union of a man and a woman in the face of attempts to make it juridically equivalent to radically different types of union; such attempts actually harm and help to destabilize marriage, obscuring its specific nature and its indispensable role in society.

These principles are not truths of faith, nor are they simply a corollary of the right to religious freedom. They are inscribed in human nature itself, accessible to reason and thus common to all humanity. The Church’s efforts to promote them are not therefore confessional in character, but addressed to all people, whatever their religious affiliation. Efforts of this kind are all the more necessary the more these principles are denied or misunderstood, since this constitutes an offence against the truth of the human person, with serious harm to justice and peace.

I’m not enough of a Pope-ologist to know how rare this is, but what struck me was his claim that his opinion is “accessible to reason and thus common to all humanity.”

There is a convention in the U.S. that we can criticize each other’s opinions, but it’s impolite to criticize each other’s beliefs (as long as those beliefs are religious, meaning not too recent in origin). So it’s fine for me to say that you are wrong about secular subjects, like physics and sports, but it’s impolite to say you are wrong if you believe that God speaks directly to you or that cavemen played with dinosaurs. Or, more directly relevant to the Pope, scientists can say that virgin conception is generally unlikely, but it would be impolite to say it never ever happened, not even once.

Anyway, that’s a long way of getting around to the point that I find the Pope’s statement galling. If he wants to express political opinions, fine. I have no objection to that as long as the giant, multibillion-dollar real estate and educational empire he runs isn’t tax exempt.

But if he’s going to make statements with that hat on — that is, subject to a declaration of infallibility* – he should lay off the social-science proclamations. If he wants to argue in the realm of reason, rather than faith, then we may weigh his record of expressed belief in fairy tales against his scientific credibility.

Believe it or not

Learning as I go here: turns out the Pope has a whole scientific academy called the Pontifical Academy of Sciences (where the “peer review” is not done by your peers, if you know what I mean). And naturally they’ve been all over this subject of reason and faith. I read a 2006 talk titled “Secularism, Faith and Freedom,” which was apparently presented to this audience:

secularismontrial

And I thought the American Sociological Association conference was a dynamic scene!

The paper says it’s necessary for religious people to argue their positions freely in a secular state’s public square. These positions include, “Faith is the root of freedom,” and “a proper secularism requires faith.” That is because liberal democracy otherwise is a moral vacuum of pragmatic consumerism with no higher purpose. So I gather that, just as any “gaps” in the fossil record summon Creation as an explanation, so does any lack of morality in the public sphere demand to be filled by faith — specifically, a “Creator who addresses us and engages us before ever we embark on social negotiation.” Absent that presence, “the liberal ideal becomes deeply anti-humanist.”

Although, after reading this whole paper and the Pope’s statement, I confess (my word choice) that I’m not sure “humanist” is really what they’re going for.

Can. 749 §1. By virtue of his office, the Supreme Pontiff possesses infallibility in teaching when as the supreme pastor and teacher of all the Christian faithful, who strengthens his brothers and sisters in the faith, he proclaims by definitive act that a doctrine of faith or morals is to be held.

33 Comments

Filed under In the news, Politics

Time travel: Regnerus study timeline suggests superhuman abilities

I will comment more soon on Mark Regnerus’s paper, “How different are the adult children of parents who have same-sex relationships? Findings from the New Family Structures Study.” (Short: I think it’s a bad-quality piece of research that should not have been published; and that Regnerus cynically manipulated promotion by the conservative press and anti-equality advocates eager to declare, “this new research tends to affirm that the ideal for a child is a married mom and dad.”)

But here’s a quick note:

In the controversy over the publication, several people have remarked on the extremely fast timeline by which the journal Social Science Research operated. According to the data listed at the top of the article, it was submitted, revised, and accepted within 6 weeks. That’s fast, but that doesn’t make it wrong, necessarily.

On the other hand, time travel is definitely cheating. Neal Caren pointed out to me this pattern: according to the study design document online at the University of Texas (here), and the article history dates listed by the journal Social Science Research (here), this is the timeline for the study:

  • Data collection start date: August 19, 2011
  • Email and postcard reminders sent to non-responders: December 15, 2011
  • Telephone reminder campaign: December 20, 2011-January 17, 2012
  • Paper received by Social Science Research: February 1, 2012
  • Data collection end date: February 21, 2012
  • Data file delivered to University of Texas “containing the collected data”: February 24, 2012
  • Revised paper received by Social Science Research: February 29, 2012
  • Paper accepted by Social Science Research: March 12, 2012
  • Paper published online: June 10, 2012

According to these documents, the paper was submitted for publication 20 days before the end of the data collection, and 23 days before the data were delivered to the University of Texas! That’s fast.

There must be some post-hoc excuse Regnerus or the journal could give to clear this up.

Commentary on the study includes:

Why Mark Regnerus’ study shouldn’t matter, even if it were the most scientifically robust study in the world,” by Ilana Yurkiewicz at Scientific American

Study questioning same-sex parenting draws fire, by Stephanie Pappas at Live Science

Do children of same-sex parents really fare worse? by Belinda Luscombe at Time Healthland

48 Comments

Filed under Uncategorized

One case of very similar publications, with some implications and suggestions

This post deals with problems in academic research publishing. It’s off the usual topic of the blog, although the publications in question do concern families and inequality. I decided to publish it here rather than try to place it somewhere else because I thought it might be controversial, and I want to take personal responsibility for it. I welcome discussion of these questions here in the comments, or in other forums where these issues are pertinent — you are welcome to repost this, with attribution.

The case here is a pair of articles by John R. Hipp, an associate professor of Criminology, Law and Society at U.C. Irvine. The two articles are:

Little of substance is learned from one that could not be learned from the other, they contain many nearly-identical passages, and they both claim to make the same major original contributions. This isn’t the most extreme case like this ever published, but it’s the most obvious one I’ve noticed that involves a major sociology journal. Without attributing any cause or motivation, we can call these two “very similar publications.”

The practice of publishing VSPs:

  • Wastes the time of editors, reviewers, and future researchers.
  • Takes up valuable space in journals, space for which other researchers are competing for their publications and to enhance their careers.
  • Misleads reviewers and administrators who are evaluating and comparing publication records.
  • Misleads the research community by creating a false impression of the weight of original research on the topic (e.g., “many studies show”).
It is also just one symptom of a pretty broken publication and promotion system in academia, which I will return to later. I don’t know anything about the history of these papers, or the author’s situation or motives, so I limit myself to discussing the content of the papers. After discussing the case, I have a few suggestions. I’m sorry this is so long.

The case

Here are the two abstracts. I don’t know an elegant way to show these side beside except a screen grab (click for higher resolution); I numbered the sentences in the abstracts.

I highlighted the major difference between the two articles: the Criminology article analyzes perceptions of crime in “microneighborhoods,” while the Social Problems piece analyzes reported violent crime rates in Census tracts (and adds a measure of short-term crime rate change). This is a substantive difference, and it involves using different versions of the American Housing Survey. But the abstracts are not written as a sequence of incremental discoveries; they claim to make a number of the same innovations and discoveries. Substantively, the difference could have been handled with one additional table, or even a hefty footnote. (The first paper reports that the violent crime rate and residents’ perceptions are correlated at about .70). In any event, the difference is not a significant part of the motivation for either article, as the abstracts make clear.

Here are the outlines of the articles, with Hipp’s headings. As you can see, the Social Problems article includes a measure of short-term crime rate change, and the Criminology article includes a section on sensitivity analyses. They are not identical (the first paper also includes a methodological appendix).

The second paper, from Social Problems, does acknowledge the existence of the first, but not in a way that would truly communicate the relationship between them. It notes (p. 411):

Recent scholarship has suggested that … residents’ perceptions of crime in the microneighborhood can differentially affect in-mobility and out-mobility for different racial/ethnic groups (Hipp 2010b). We extend this literature by using official violent crime rates within the broader neighborhood as measured by the census tract.

Later (p. 414), the Social Problems paper minimizes the findings of the Criminology paper:

One study provided suggestive evidence of disproportionate out-mobility using information on the perceptions of crime among residents living within a micro-neighborhood of the nearest 11 housing units (Hipp 2010b). Whereas white households perceiving more crime were more likely to move within four years, black and Latino households showed no such tendency (Hipp 2010b). Furthermore, whites living in microneighborhoods with a general perception of more crime were also more likely to leave the unit, whereas Latino and black households again showed no such tendency. This evidence of the importance of perceptions of crime within a small micro-environment is important, but it cannot assess whether such perceptions accurately capture the crime environment of the micro-area, nor whether the crime environment of the broader neighborhood is also important. The present study addresses these limitations.

The Criminology paper was published online in August 2010, and the Social Problems paper is dated August 2011, so I can’t tell if the reviewers for the Social Problems paper had access to a published copy of the first at the time of their review.

If the second paper constitutes a genuine additional contribution, it would be reasonable to publish it separately, making clear that it represents a methodological variation of findings already known. But instead the second paper announces the same contributions as the first. In fact, as the text from the sections titled “summary” in the introduction of both articles show, two out of three of the “important contributions to the literature” are identical:

The third contribution differs. However, by the publication of the second paper, the first two “important contributions” are no longer original (the generally understood meaning of “contribution”).

Later, in the conclusions, the assertions of originality (the “key/crucial implication” and “important takeaway”) are nearly identical. Here are some excepts from the conclusions:

The genuine relationship between the two papers is not revealed.

There is no substitute for reading the text with human eyes. However, there also are tools for displaying and analyzing similarity in documents. The U.S. Dept. of Health and Human Services Office of Research Integrity provides a reference to the Plagiarism Resource Center (once) at the University of Virginia, which distributes a program called WCopyfind, an open-source Windows-based document comparison tool.

After converting these two articles to text documents, and removing the tables, references, methodological appendix, and extraneous page numbers and other fragments, I subjected them to the WCopyfind comparison. These are the parameters I used, which were recommended for cases in which minor editing is presumed.

Shortest Phrase to Match: 6
Ignore Punctuation: No
Ignore Outer Punctuation: Yes
Ignore Letter Case: Yes
Skip Long Words: No
Most Imperfections to Allow: 0

The analysis found 2,235 words that were in duplicate 6-word strings, accounting for 20% of each article’s text. For example, here are two passages, with the strings that the algorithm flagged in red:

The present study provides an important corrective to the large volume of prior research finding a positive relationship between the size of racial/ethnic minority groups in a neighborhood and the rate of crime at one point in time and assuming that the causal direction runs from the presence of such minorities to higher rates of crime.

Prior research frequently has found a relationship between the presence of racial/ethnic minorities in a neighborhood and the rate of crime at one point in time. Although they sometimes posit different mechanisms, these studies almost always conclude that the causal direction runs from the presence of such minorities to higher rates of crime.

On the one hand, we can see that the passages are more similar than the red text implies, but on the other hand there are times when the algorithm appears to just catch phrases that occur in this line of research, such as “are more likely to move into.” If you set the shortest phrase to 5 words, the program flags 23% of the text; at 10 words it flags 12%.

The nice thing about the program is it creates a side-by-side comparison of the entire documents, with the common text strings linked, so you can click on the text in one article and see where it appears in the other, in context. I have put this side-by-side file here for your perusal. The full articles (behind paywalls) are linked above.

(Odd aside: the Criminology paper is in the first person singular, while the Social Problems paper is written in the first person plural, although both have only one author.)

What is this?

This is not a duplicate paper, although it includes a large amount of copied text, which would normally be called “self-plagiarism,” defined by Miguel Roig like this:

Whereas plagiarism involves the presentation of others’ ideas, text, data, images, etc., as the products of our own creation, self-plagiarism, occurs when we decide to reuse in whole or in part our own previously disseminated ideas, text, data, etc without any indication of their prior dissemination. Perhaps the most commonly-known form of self-plagiarism is duplicate publication, but other forms exist and include redundant publication, augmented publication, also known as meat extender, and segmented publication, also known as salami, piecemeal, or fragmented publication. The key feature in all forms of self-plagiarism is the presence of significant overlap between publications and, most importantly, the absence of a clear indication as to the relationship between the various duplicates or related papers.

Thinly-sliced “salami” articles do not necessarily include any duplication, but rather just tiny increments of scientific progress. There is some of that here, as well as elements of a “meat extender” case, in which small amounts of additional data or analysis are added — without a transparent disclaimer and rationale. However, regardless of the differences in analysis, the texts — especially the framing and concluding bread around the salami — are too similar to be justified.

That last distinction in Roig’s definition, the “key feature,” is what’s important here. Using a little boilerplate theoretical language in related work, or a very similar equation or description of variables when using the same datasets, doesn’t undermine the value of the work, as long as the original source is attributed. This distinction appears in a Nature news story:

…although the repetition of the methods section of a paper is not necessarily considered inappropriate by the scientific community, “we would expect that results, discussion and the abstract present novel results”, says Harold Garner, a bioinformatician at Virginia Polytechnic Institute and State University in Blacksburg.

There are in fact some “novel results” in the second paper, but the novelty is not as important as the basic findings, which are described nearly identically — as original in both cases.

In the normal course of pursuing a research agenda across multiple article-length publications, some repetition is justifiable and even helpful. But without a “a clear indication as to the relationship between the various duplicates or related papers,” in Roig’s words, the sandwich is not kosher.

Consider an alternative example, in an economics paper, “Booms, Busts, and Divorce,” by Judith Hellerstein and Melinda Morrill, which I wrote about recently. Their main contribution is the finding that divorce was pro-cyclical from 1976 to 2009 – that is, there was more divorce when the economy grew. The “main analysis” is this:

we combine data on state-by-year unemployment rates with state-by-year vital statistics data across the United States over the period 1976 to 2009. We assess the impact of local macroeconomic conditions on state-level divorce rates, controlling for year fixed effects, state-specific time trends, and state-specific time-invariant determinants of divorce rates.

But they then add this:

Our basic finding of pro-cyclical divorce is robust to alternative empirical specifications and is found when we allow the effect of unemployment rates on divorce to vary by the fraction of the population that is Catholic … the census region, and by time period. We also show that this finding is robust to extending our unemployment series back to 1970 … Finally, we replace the unemployment rate at the state-by-year level as our measure of macroeconomic conditions with two alternative measures: state-by-year per capita gross domestic product (GDP) and state-by-year per capita income.

Each of those different steps is a way of extending and corroborating the “main analysis,” and each required adding data from a different source and conducting a new statistical analysis. But they did not make the main finding a new discovery warranting an additional publication with the same motivation and conclusions as the first.

Look at me

To head off an inevitable question, you are welcome to look at my own publication record. There are two specific cases in which my co-author Matt Huffman and I wrote pairs of articles addressing similar questions. They were: whether variation in gender segregation across labor markets affected patterns of gender wage devaluation (here and here); and whether the presence of female managers is associated with lower levels of gender inequality (here and here).

In each case the later paper acknowledged the earlier (or, they acknowledged each other, in the first pair), and explained the differences in approach — they involved different kinds of data and statistical methods in each case as well. The results were consistent in each pair, and thus the conclusions were strengthened, the pattern corroborated. Maybe we could have combined each pair into one very dense article, but that might very well have been rejected as too long or complicated for the journals we used, and they weren’t ready at the same time. (For what it’s worth, I also checked the WCopyfind comparison between the latter pair of our articles, and found 37 words in matched strings of 6 words or more.)

What to do

There is considerable research on the problem of duplicate publication in the medical literature, as Nature reports about 0.4% of articles in MedLine are probable duplicates. I don’t know how widespread the various kinds of duplication are in the social sciences. I can’t say this is a rampant problem, or that this case is an extreme one — I don’t know.

However, based on what I have read and shown above, I’m satisfied this case is a problem the likes of which we should try to avoid.

First, however, we should consider this kind of problem in the context of the “cycle of publication overproduction,” in which the Academy finds itself. That’s the phrase from a report by Diane Harley and Sophia Krzys Acord, “Peer review in academic promotion and publishing: its meaning, locus, and future,” published by Center for Studies in Higher Education at Berkeley. They write:

…the problems we face in scholarly communication are not about publishing, per se, or the process of peer review in that system. Instead, the problems lie with the current advancement system in a multitude of higher education sectors globally that increasingly demand unrealistic publication requirements of their members in lieu of conducting thorough and context-appropriate institutional peer review, at the center of which should be a close reading and evaluation of a scholar’s overall contributions to a field in the context of each institution’s primary mission.

And one of their first recommendations is: “Encourage scholars to publish peer-reviewed work less frequently and more meaningfully.”

While we’re working on that, I have more immediate suggestions for four stages of the publication process.
  • Journal peer review. A large survey of peer reviewers found that, in the social sciences, about 88% believe ensuring acknowledgement of previous work should be one of the purposes of peer review, but only 59% believe the system is current able to do so. The editors and associations in this case – Criminology (the American Society of Criminology) and Social Problems (the Society for the Study of Social Problems) — or others have an incentive to prevent this waste of their resources. Editors and reviewers must be told when highly similar work exists, obviously, and the current ethics statement for American Sociological Review states that “Significant findings or contributions that have already appeared (or will appear) elsewhere must be clearly identified.” But in one concrete step to strengthen that, we could require that referenced work that has not been published — in press, under review, forthcoming, etc. — be accompanied by access to the papers, so the editors can look for redundancy. As it is, these papers are not available to reviewers or editors for verification.
  • Promotion committees and administrators. Counting up articles, and weighting them according to journal prestige or impact factor, is widely practiced in promotion decisions. Really reading the work is much better, but it requires more time, and more specialized skill. The Harley and Acord report has some great systemic recommendations. But in the meantime one thing I might start doing when reviewing cases is to ask myself, “Could any of these articles have been shaved into multiple salami slices?” If they could, but weren’t, let’s give credit for that decision (even if the work wasn’t in a top journal).
  • Ethics guidelines. The American Sociological Association’s Code of Ethics prohibits publishing “data or findings that they have previously published elsewhere.” However, the plagiarism provision only mentions “another person’s” work. The “self-plagiarism” (or duplication) aspect of this should be beefed up.
  • Shorter articles. Thin slices of salami are not so distasteful when eaten without a full load of bread and condiments. Sociology journals tend to have long articles compared with some other social sciences (psychology, economics, public health). If one of the two articles in this case could have been published as a brief report, with a few references and a clear emphasis on what was new, that might be reasonable. In many cases, the copying is in the theory, literature review, and methods. I know I just said we publish too much as it is, but sometimes shorter articles would help with this problem.
Addendum: Why?

Why would I write this and make it public? I certainly have nothing personal against John Hipp, whom I hardly know. But part of the privilege of having tenure protection is that the fear of offending people shouldn’t prevent me from speaking up on issues I think are important, and I am worked up about this issue. Academia has the unrivaled privilege of policing itself, and we have built a system that runs on trust, believe it or not.

I have done a few promotion reviews since I’ve been tenured, and reviewed many journal articles (more than 70 in the past 7 years). I have evaluated hundreds of job candidates and thousands of graduate student applications. Maybe it’s too much to hope that academia will abandon its addiction to bean-counting in the evaluation of productivity and merit any time soon. But we can increase our sensitivity to the flaws of that approach, and promote the honesty and transparency that are prerequisites for its functioning.

These are some other interesting sites and pages on these issues:

28 Comments

Filed under Me @ work, Research reports