Paul Amato on reviewing Regnerus

I recently discussed Paul Amato’s role in the Regnerus Affair. I offered my opinion that, if Amato was a reviewer of the Regnerus article, he should not have been, mostly because he had served as a paid consultant on the study. (My long version of the Affair is here, a critique of the paper I co-authored is here.)

I regret that before writing that post I didn’t directly ask Amato if he wanted to discuss his role and whether he served as a reviewer. After the post appeared he sent me this statement, which I agreed to post. I added some more comments of my own below. (He also reminded me that he had voted for the Family Section Council resolution asking ASA to respond to the Regnerus study, which they did.)

Amato is President Elect of the National Council on Family Relations and a Distinguished Professor at Penn State University.

Thoughts on the Mark Regnerus (2012) study, by Paul Amato

One year has passed since Mark Regnerus (2012) published a highly controversial article on the children of parents who have same-sex relationships. Given that time tends to improve one’s perspective, this seems like a good time to reflect on the study and its aftermath.

My involvement

I worked for two days at the University of Texas as a consultant on the New Family Structures Study (NFSS). As I recall, seven consultants were at the meeting, along with a statistician from the survey research organization that later collected the data. I consulted primarily on sampling and measurement issues, and I was paid for two days of my time, plus travel expenses for myself (and my wife, who accompanied me). I charged for two days at my usual fee, which is $150 per hour. So I earned about $2,400. I received no further compensation after that.

About six months later, the editor of Social Science Research (SSR) asked me to review a manuscript written by Mark Regnerus. I informed the editor that I had worked as a paid consultant on the survey on which the manuscript was based. The editor said that he would like to have my views on the paper anyway, so I shared my views as honestly as I could.

This situation comes up now and then in my experience. When reviewing manuscripts for journals, I occasionally discover that I know the author and have some sort of relationship with the author or the study. In one case, for example, the author was a friend and colleague of mine, and I had read an earlier version of the paper and provided comments to the author. In this and every other case in which I have brought information like this to the editor’s attention, the editor has asked me to do the review anyway. Journal editors often have a difficult time getting reviews, and I assume they treat these reviews as one more data point. So the editor of SSR was doing what other editors do, as far as I know.

Was this particular case a conflict of interest for me? The American Sociological Association (ASA) defines a conflict of interest in the following manner:

Conflicts of interest arise when sociologists’ personal or financial interests prevent them from performing their professional work in an unbiased manner.

With respect to the Regnerus manuscript, I had no personal or financial interest in whether the paper was published. So by this definition, there was not a conflict of interest. Of course, sometimes there is the appearance of a conflict of interest. In these cases, the ASA code states:

Sociologists disclose relevant sources of financial support and relevant personal or professional relationships that may have the appearance of or potential for a conflict of interest…

As noted earlier, I disclosed to the editor that I had worked as a paid consultant on the NFSS. I also disclosed my role as a paid consultant in the commentary that I wrote for the Regnerus article, which appeared in SSR. I never attempted to hide the fact that I was part of the team that consulted on survey design.

In retrospect, I understand that providing a review was not a good idea, because one should avoid even the hint of impropriety in matters like this. At the time, however, I simply felt that I was helping the editor and being a good colleague.

Contrary to the views of some (but not all) of my colleagues, I thought the Regnerus manuscript was worth publishing. My key recommendation, however, was that the editor should publish the paper with commentaries from authors who hold a variety of perspectives, including gay and lesbian scholars who had published in this area. I believed that the Regnerus paper, accompanied by a diverse set of commentaries, could represent a useful contribution to the literature on LGBT families. Unfortunately, the editor was unable to recruit any gay or lesbian scholars to contribute commentaries, so my idea for an exchange of views fell flat. (The subsequent issue of SSR devoted to the controversy came closer to what I had envisioned.)

Almost everyone got it wrong

When the study was published, criticism from the political left was swift and harsh. Unfortunately, some commentary devolved into ad hominem attacks, accusations of fraud, and name-calling. Rather than intellectually engage the findings, the goal of some critics was to thoroughly discredit the study—and the author. While they were at it, many critics also attacked the editor, the reviewers, the consultants, those who wrote commentaries—even the survey research firm that collected the data! Anyone with any form of contact with the study became an enemy of the people.

This is unfortunate, because the political left could have benefitted from a strategic appropriation of the findings. The study involved a national sample of young adults with an LGBT parent. As the study noted, few of these young adults spent long periods of time in households with two parents of the same sex. Instead, most were born into heterosexual families that later broke up, presumably when one parent came out as gay or lesbian. Many of these youth went on to experience a variety of other family structures before reaching adulthood. One out of seven spent time in foster care. Previous research shows that instability in the family of origin increases the risk of a variety of long-term social and psychological problems for offspring. Consistent with this research, young adults in the study had modestly elevated problem profiles. It is reasonable to conclude that the elevated number of problems observed in these young adults was due to family instability rather than the sexual orientation of parents. For this reason, most observers have noted correctly that this study contributes nothing to our understanding of how children fare when raised by same-sex parents in stable households

Rather than dismiss these finding as being irrelevant, however, it’s useful to dig more deeply into the results. Why did these marriages end in divorce? More importantly, why did gays and lesbians wind up in heterosexual marriages in the first place? The explanation probably would go something like this: Like heterosexuals, many gays and lesbians wish to have families and raise children. But a generation ago, intolerance was the rule and discrimination against gays and lesbians was endemic. For many, forming heterosexual unions appeared to be the only way to achieve the dream of family and children. But these unions tended to be unstable, with problematic consequences for adults and children. Presumably, as our society becomes more accepting of LGBT families, the unfortunate circumstances of children and parents described in the Regnerus study will become less common. The freedom to marry, in particular, should increase stability in the lives of children with gay and lesbian parents.

In short, findings from the Regnerus study can be interpreted as strong evidence in support of same-sex marriage. The American Psychological Association and ASA research briefs emphasized the fact that almost all prior studies found no differences between children with heterosexual parents and children with gay or lesbian parents. The “no difference” perspective suggests that children will not be harmed by same-sex marriage. The lesson from the Regnerus study, however, is that children thrive on family stability, including children with gay and lesbian parents. We know that marriage tends to stabilize relationships, yet same-sex marriage is not allowed in most states. Given that children benefit from the stability provided by marriage, it is unfair and unkind to deny children the right to live with married parents. In contrast to the “no difference” perspective, a “family stability” perspective implies that we need to change our laws NOW to protect and benefit children.

If the political left missed an opportunity by failing to understand the full implications of the Regnerus study, the political right made even more serious blunders. Many conservative observers have cited the Regnerus study as if it provided evidence that being raised by gay or lesbian parents is harmful to children. This claim is disingenuous, because the study found no such thing. A noteworthy example came from Regnerus himself, who signed an amicus brief to the Supreme Court citing his study as evidence against same-sex marriage. This is curious because on page 766 in his 2012 article, Regnerus stated that his study was not intended to either affirm or undermine the legal right to same-sex marriage. And on page 768 of his response to the commentaries in the same issue, he stated that his data should not be used to press any political program. Given these cautious early statements, it is exasperating to see Regnerus later cite his own study as evidence against same-sex marriage.

Concluding thoughts

Many observers have argued that the Regnerus study should never have been published. It is important, however, to focus on what the study actually showed, and not on what people claim that it showed or wanted it to show. The study showed that family instability is not good for children, and many children with gay and lesbian parents, a generation ago, experienced a lot of family instability. It is not difficult to see how the personal problems of these families were affected by the restrictive social milieus in which they lived.

Since the Regnerus study was published, studies by Potter (2012) and Allen, Pakuluk, and Price (2013) have shown associations between having same-sex parents and child problems. Like the Regnerus paper, both of these articles survived the peer review process and, in fact, were published in top-tier social science journals. Rather than try to discredit these studies (and any future studies that may show similar results), it is better to examine the findings carefully and figure out what is going on. In fact, both studies are entirely consistent with the family stability perspective described earlier.

In conclusion, the political left discredited the Regnerus study without fully considering its findings, and the political right used the study disingenuously to further their political goals. Few people have focused thoughtfully on what the data actually show and what we can learn from the study. The controversy over the Regnerus study provides a sobering illustration of what can go wrong when ideology distorts social research.

References

Allen, Douglas W., Catherine Pakaluk, and Joseph Price. (2013). “Nontraditional Families and Childhood Progress through School: A Comment on Rosenfeld.” Demography 50:955-961.

Amato, Paul R. (2012). “The Well-Being of Children with Gay and Lesbian Parents.” Social Science Research 44:771-774.

American Sociological Association. (undated). American Sociological Association Code of Ethics (http://www.asanet.org/images/asa/docs/pdf/CodeofEthics.pdf)

Potter, Daniel. (2012). “Same-Sex Parent Families and Children’s Academic Achievement.” Journal of Marriage and Family 74:556-571.

Regnerus, Mark (2012). “How different are the adult children of parents who have same-sex relationships? Findings from the New Family Structures Study.” Social Science Research 41:752-770.

Regnerus, Mark (2012). “Response to Paul Amato, David Eggebeen, and Cynthia Osborne. Social Science Research 41:786-787.

wilcox-echo-chamber

Philip’s followup comments

I don’t think it was a gross ethical violation for Amato to review the paper, in which material or ideological gain lay behind his decision to do the review and led him to recommend publishing the paper against his better professional judgment. Rather, he thought it was a reasonable paper and offered that opinion when asked — which is unsurprising given his involvement in the project. So my real disagreement with Amato is over the value of the paper. I think it’s a worthless paper, done wrong, and only advanced because of the author’s ideological attachment to its results; that it accidentally helps reveal something true about family instability does not make it worth publishing.

Being a consultant on a small project like the NFSS is not like being a consultant on a giant project like the National Longitudinal Study of Adolescent Health. Consulting on Add Health, which has yielded thousands of publications and involved dozens of experts at many agencies, does not disqualify a person from reviewing a study using that data. In contrast, the NFSS has so far yielded one paper, solo-authored by the PI, who personally invited the several consultants. With only a few people involved, being one of them matters more.

Anyway, beyond the appearance of conflict, the problem with Amato serving as a reviewer is that it did not provide an outside perspective to the editor, James Wright. How could Amato’s review help Wright make the decision? Getting the input of a consultant on the project might help an editor shape a revision or build a special issue, but given Amato’s involvement his endorsement should not have counted as part of the peer review process. When Amato revealed his role, Wright should have declined his review.

Taking for granted the unethical behavior of Regnerus, and Brad Wilcox, on whose behalf Regnerus acted, the real failure here is by Wright. Instead of seriously reviewing the paper, he essentially whispered into an echo chamber of backers and consultants, “We should publish this, right?”

I believe the paper should be retracted because the conclusions are demonstrably wrong, because the author lied in the paper about the involvement of the institute that funded it, and because the peer review process was compromised by conflicts of interest. As long as this remains uncorrected, and James Wright remains editor, the integrity of the journal is indelibly tarnished.

While Wright is editor, I will no longer review for or submit to Social Science Research. I hope others will join me in that decision.

Comments that rehash well-known opinions or make person attacks will be shortened or deleted.

20 Comments

Filed under Me @ work

20 responses to “Paul Amato on reviewing Regnerus

  1. [comment edited]
    [...] I was following along mostly okay, until I got to this,
    “Since the Regnerus study was published, studies by Potter (2012) and Allen, Pakuluk, and Price (2013) have shown associations between having same-sex parents and child problems.
    [...]
    First in your Original Commentary you said, that the Sarantakos (out of Australia) study supports the Marks and Regnerus papers. In fact Dr. Sarantakos was very diligent about documenting the extreme bullying and social stigmatization that the children with same sex parents experienced.
    [...]
    Additionally Dr. Sarantakos notes that the evaluations of the students was made by teachers who are subject to their own prejudices. In fact he said within this study, that he wanted to do a follow up study on the level of disapproval of homosexuality by teachers.
    [...]
    Today he references Potter, saying that the Potter research supports the Regnerus research, so let me quote from Potter,
    “The results indicated that children in same-sex parent families scored
    lower than their peers in married, 2-biological parent households, but the difference was nonsignificant net of family transitions.”
    [...]
    Since the US Census data does not give you the historical data on these families Dr. Rosenfeld made a very wise choice. He put a five year cut off on these families. In other words he only looked at in his study Mom/Mom/Child or Dad/Dad/Child families who had been together 5 years or more. Under 5 years he would not know if the child came into the family through foster care or divorce etc. By looking at families who were in this structure for 5 years or more he eliminated transitory relationships, and other circumstances that he would have no way of knowing based on the Census data. Dr. Rosenfeld’s researched showed no difference.

    Along come the 3 Economists and they re-evaluated the same data and removed the five year limit. By doing so they showed the kids with same sex parents (or to be more clear we don’t even know that both are the legal parents to the child) do worse. Rosenfeld made the right decision as he wanted to isolate out same sex parenting, and thus he had to start at 5 years because in 5 years or greater he could compare apples to apples and oranges to oranges.
    [...]
    I am highly offended that Dr. Amato chooses to misrepresents the other research when sticking up for the Regnerus paper today. It give the appearence of, “See I’m right about this (I did a good peer review), other research says the same thing.” When the other research dos NOT show the same thing. It seems like a CYA statement to me.

  2. etseq

    [comment edited]
    He certainly has the Christian Smith right wing talking points down – “left wing attack on science” etc. Noticed he took several paragraphs bemoaning all the “left wing assaults” but only took one paragaph to meekly condemn the right and Regnerus himself. Also, he sounds just like that unctious Will Saletan from Slate who always trys to play the Platonic Moderate condeming both left and right and who also have Regnerus front page in a supposedly liberal rag to defend himself with what most people considered very weak criticism.
    [...]
    And I am sorry but citing a study by Douglas Allen who is known anti-gay catholic nut in Canada ( and a freaking economist for god’s sake) to support the claim that there are “…associations between having same-sex parents and child problems” is just indefensible. Also, Allen did not do any original research, he simply tried to pull the same crap Regnerus did by “re-analyzing” Rosenfeld’s data from two years prior. Rosenfeld replied to Allen shortly thereafter pointing this out.
    [...]
    Potter was published in a low ranking journal that is known to be evangelical friendly (they even published that loon from Pat Robertson’s university Mark Youhouse claiming that ex-gay therapy really works)!!

    This guy just sounds angry at the blowback and the fact that he has waited over a year to reveal this makes me think he is just trying to cover his ass (although not in a a very effective way).
    [...]
    Thanks Philip for posting this – many of us in the activist community appreciate and respect how you have handles this matter. I know some people can get a little intense in the comments but they are understandly upset at the fallout over this issue. It is definitely a black eye for Sociology and the peer review process in general but you and others have done your best to address it head on. I only wish I could say the same thing for Wright, Wilcox, Amato and a few others who shall remain nameless.

    One other thing…Marks, along with Rekers, has actually been scolded by judges in published opinions when they have been called to testify as expert witnesses against gay parenting, most recently in Florida. The judge was merciless in her criticisms of the religious biases in their work. What editor would seriously publish these people after such an incident. Shouldn’t that pretty much preclude them from publishing on matters that they clearly have religious conflicts of interest? Marks has been a well known Mormon anti-gay culture warrior for ages – how could James Wright not be aware of this???
    [...]

  3. Really? He thinks women leave straight marriages and enter lesbian marriages because they have finally dared to come out as lesbians? I highly doubt in. Far more often, in my experience, women leave straight marriages for the same reasons anyone does – they’re miserable, or they’re abused – and enter lesbian marriages when they find that middle-aged men are only interested in women ten years younger than they are, and middle-aged women are easier to attract. Like so much else, it’s a patriarchy issue.

    • etseq

      I hope this was an attempt at sarcasm or are we resurrecting the old myth that lesbians are just man-hating women???

      • Scott Rose

        I note that in his follow-up NFSS piece in November, 2012, Regnerus in answering to critics who imagined that his respondents were born to “mixed orientation” couples wrote “Whether these were in fact mixed-orientation marriages or relationships is of course impossible to discern with confidence.” Yet, above, Dr. Amato states his opinion that everybody should “dig deeper” into the findings . .. by discerning with confidence that these were in fact mixed-orientation marriages or relationships. Regnerus defends himself by saying that one can not possibly dig deeper into the findings in that particular way, while Amato says that we must dig deeper into the findings in that particular way. They can not both be right about whether the findings allow one to “discern with confidence” that these were in fact mixed-orientation relationships.

  4. Phillip, we disagree on the source of blame, and I think once you are editor a major journal, which I hope you will be very soon, you will think differently about having reviews in hand with consistent evaluations of a manuscript—even if a couple are tainted (one was not, and by someone just as distinguished as Paul Amato–who is also not a shil for the Christian Right). What has gotten us into this mess is the attitude of unhelpfulness expressed in your statement that you will not review for SSR. So, I guess the editor will just have to send stuff on family inequality issues to people like Bradley Wilcox or other right wing scholars? Where are you going to find those thousands of qualified reviewers if all of the left of center people now boycotting because they didn’t like an editorial decision? I hope you will reconsider. And, that you will rethink your perspective on Wright. He is retiring soon and SSR will need a good strong editor, like Jim Wright has been for a long time, and I’d love to see you in that slot.

  5. Scott Rose

    [edited]
    [...]
    The NFSS should no more be used to attempt to support gay rights than to oppose them, because from the data, it is impossible to understand whether any of the respondents were substantially raised by one and much less by two gay parents.
    [...]
    If the assumptions Dr. Amato is making about why a lesbian woman might have married a heterosexual man 40 years ago are just grafted onto the NFSS data as an afterthought, in an attempt to defend the indefensible, we might do well to think of certain assumptions not yet made in connection with the data. For example, 40 years ago in many jurisdictions, the mere fact of a parent’s being known as homosexual could be enough for that parent to be ordered never to see their own child unsupervised. The NFSS survey doesn’t make it possible to know whether the respondents suffered that fate, but empirically we know that it did happen. Obviously, for a court to demonize one’s parent on the basis of their sexual orientation and to forbid contact on that basis alone would be extremely difficult for a child.
    [...]

  6. @sherkat: Whether or not to accept a manuscript for publication is one of the last decisions an editor makes, so it seems strange to consider that in isolation. Most critically, the editorial office selects the pool of reviewers. Relying heavily on author recommendations, like SSR does, seems ripe for abuse. Additionally, for an article that seeks to overturn a paradigm such as the “no difference hypothesis”, you would think an editor would make sure that one, or more, of the reviewers works in that paradigm. I’ve seen no discussion where any of the reviewers selected for the article were described as having a) been selected independently of Regnerus’s recommendations, or b) experience studying children in same-sex families.

  7. Claude Summers

    The whole purpose of peer review is to weed out poorly conceived studies that do not contribute to the advancement of science and the pursuit of truth. Obviously, not all peer-reviewed articles are necessarily “true” or make a genuine contribution to science and scholars have strong disagreements about methodology and conclusions, and from those disagreements science is often advanced; but the expectation is that studies published in scholarly journals after a rigorous peer-review are good faith efforts to reach the truth and are not mere propaganda or efforts designed to mislead.

    Reviewers have an obligation not only to disclose conflicts of interest but also to make certain that the studies they recommend have academic integrity and actually do what they say they are doing. Amato’s failure to do this is what, it seems to me, makes him complicit in the Regnerus hoax.

    Federal courts have very specific criteria for certifying “expert” witnesses and allowing the citation of scholarly papers. One of those criteria is that witnesses have relevant credentials and have published in peer-reviewed scholarly journals. That is why, for example, the opinions of the Prop 8 star witness David Blankenhorn were disallowed in the Prop 8 case. He has no Ph.D. in a relevant field and his only publication in a peer-reviewed journal had nothing to do with marriage.

    Similarly, the citations of scholarly evidence in the official briefs of the plaintiffs and defendants in federal cases also must meet a similar standard. Hence, it was crucial to the attorneys attempting to preserve DOMA and Prop 8 that they have some respectable “evidence” to counteract the overwhelming scholarly consensus that same-sex parents are as capable as opposite-sex parents. They got what they were looking for, thanks to the $700,000 spent by the Witherspoon Institute to manufacture Regnerus’s junk science and the complicity of Social Science Research and its reviewers.

  8. Pingback: Here’s what caught my eye today…8/2/13 | Florida Higher Education Spectator

  9. Ria VR

    Philip, I keep thinking as I go through the complex and rigorous IRB process at my current institution that if reviewers had to disclose potential conflicts of interest (i.e. any payments, including reimbursement, for work, which is what I’m required to do now) in the same way that PIs do, we might be having a different conversation here. It would add an immense amount of stress and cost us resources, but a colleague at the IRB office and I have been talking a lot about how the IRB is set up at the data collection and analysis points but not necessarily at the publication level. I wonder if you and/or others have thoughts on this?

  10. Pingback: Regnerus Affair timeline, with maze | Family Inequality

  11. etseq

    I know this is an old thread but I just ran across the most bizarre article that Regnerus published a few weeks ago that I must have missed. Mark is VERY worried about the disastrous consequences of gay marriage which he discusses in great detail. Using an odd mixture of sexist evolutionary psychology and homophobic of catholic natural law, he concludes that heterosexual marriage will become even more unstable due to increased infidelity by otherwise monogamous husbands and wives. This is based on the following FACTS:

    1. Lesbian relationships are inherently unstable because women don’t really want as much sex as men so when you put two women together, you get lesbian death bed and so lesbians have to find new partners frequently

    2. Gay male relationships are also inherently unstable but for the opposite reason – men cannot control their sexual desires so without a woman moderating, gay men either cheat or have open relationships.

    3. Allowing gays and lesbian relationships to be officially sanctioned and regarded as equal to heterosexuals will somehow convince heterosexuals that they don’t have to be monogamous to be married. Straight men will envy gay men and try to demand more partners from women and women who would normally constrain their male partners will suddenly give in because they are more likely to support same sex marriage in opinion polls.

    4. Adultery laws will have to be abolished because apparently adultery is defined as PIV (sorry have to abbreviate to avoid filters) heterosexual intercourse and we all know gays don’t have the correct body parts for this.

    Also, did you know that the increase in heterosexual anal sex was caused by gay men setting an example for their straight counterparts? Well, know you do!

    Finally, he even quotes Randy Shilts from The Band Played On who died in the freaking 1980s complaining about gay male sex habits that led to AIDS.

    You have to read this article to believe it – he apparently isn’t hiding his true opinions anymore. He tries to appear academic and cites lots of studies but in very misleading ways. What struck me the most wasn’t so much the homophobia but the sexism. He really believes in innate gender roles, that men only want sex and women have to basically to hold out until marriage otherwise men have no incentive to form relationships.

    http://www.thepublicdiscourse.com/2013/06/10325/

  12. Pingback: The Many Faces Of Brad Wilcox: Instignator, Funder, Paid Consultant, And Peer Reviewer Of The Same Study | Alas, a Blog

  13. Pingback: Year-end report | Family Inequality

  14. Pingback: The blogger will be heard, Michigan trial edition | Family Inequality

  15. Pingback: Dead Baby | Irritated Queer

  16. Pingback: It’s modernity, stupid (Book review of The Sacred Project of American Sociology, by Christian Smith) | Family Inequality

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s