Tag Archives: american sociological association

Let’s use award incentives to promote open scholarship (at ASA this year!)

At the American Sociological Association of America meetings in Montreal next month, I will begin a one-year term as chair of the Family Section. I’m honored to have been elected to this position, and will do my best to make a positive contribution in that role. Besides doing the job in the normal ways — organizing our sessions at the conference next year, coordinating committees, and so on — I will bring a proposal to the section’s council to open our graduate student paper award. Here’s what I mean.

Steps toward solutions

Sociology has an inertia problem with regard to open scholarship. Lots of us understand that it would be better if our work was shared faster and more freely. That would be better for the generation and dissemination of new knowledge, it would promote collaboration, reduce costs to the public, and increase our capacity for engagement with each other and the public. Unfortunately, the individual steps toward that goal are unclear or daunting. Many of us need promotion and tenure, which requires prestige, which is still driven by publication in the paywalled journals that work against our open goals: they slow down dissemination, restrict access to our work, and bilk our institutions with exorbitant subscription fees.

To help overcome this inertia, a group of us have created SocArXiv, a non-profit, open access, open source archive of social science research that allows free, immediate publication of papers at any stage of the publication process. When and if the papers are published in a peer-reviewed journal, the preprint version can link to the journal version, providing a free copy of the paywalled paper. (Here’s an example of a new paper published in American Sociological Review, with a free copy on SocArXiv, which includes a link to the ASR version). In the meantime, the paper is available to our peers and the public. It provides a time-stamped record of the development of our original ideas, and is discoverable through Google Scholar and other search tools. People can still get their jobs and promotions, but the quality, efficiency, and reach of our research is improved. And part of what we are rewarding is open scholarship itself.

flipaward

Using awards

SocArXiv, of which I’m director, is trying to get the word out and encourage the use of our system, and open scholarship in general. One of our new ideas is opening paper awards. This may help people get in the habit of openness — and start to see its benefits — and also work against the negative impression that many people have of open access as a cesspool of low quality work. We hope this intervention will be especially effective coming early in the career of up-and-coming scholars.

Using its grant money and support from academic libraries, SocArXiv is offering sections of the ASA — like the Family Section — $400 to transport their paper award winner to the conference next year, if they using the archive as the submission platform for their awards. I’m bringing this proposal to the Family Section (and one just like it to the Population Section, of which I’m Secretary Treasurer).

We hope the open paper award will become a common best practice in our association — still providing the prestige and reward functions of the award, but also promoting best practices with regard to open scholarship, increasing our visibility, building the scholarly communication infrastructure of the future, and generating buzz for our conference and our research.

There are possible objections to this idea. Here are a few, with my responses:

  • Sharing unpublished work will lead to someone stealing their ideas. You protect yourself by posting it publicly.
  • We shouldn’t promote the dissemination of research that hasn’t been peer reviewed yet. We do this all the time at conferences, and SocArXiv allows posting updated versions that replace the original when it is revised.
  • This would impose a burden on people submitting papers. Being considered for an award is a privilege, not a right; it’s OK to require a short, free submission process.
  • Sharing a paper publicly will compromise its publishability later. All ASA journals, and all journals worthy of our support, allow posting preprints prior to publication. Here’s a list of 25 top journals and their policies.

Details

In the case of the Family Section, it looks like no change in the bylaws is needed, because they don’t specify the submission process for the graduate student paper award. They state:

Best Graduate Student Paper Award. The committee will be chaired by the Section Chair. Two additional members of the Section will be appointed by the Section Chair. The committee will select a best paper from among nominations submitted. Papers, dealing with a family-related topic, may be either published or unpublished and must have been writted by a graduate student (or group of graudate students) while still enrolled in a graduate program. The award, in the form of a Plaque and citation, shall be presented at a Section Reception (or, in the event no reception is held, at a Business Meeting of the Section).

Instead, I think we can just revise the call for award nominations, like this:

The Family Section Outstanding Graduate Student Paper Award

​Deadline: 3/13/2018

Graduate students are invited to submit an article-length paper on the family. The paper should represent a finished product rather than a proposal for future work. The submission can be based on a course paper, a recently published journal article, a manuscript under review at a journal, or a conference presentation. Co-authored papers are acceptable if all authors are students, although the prize will be shared. The paper must have been written when the author was enrolled in a graduate program. The paper may not exceed 30 pages or 11,000 words. Submissions are made by posting the paper on SocArXiv and sending a link to the paper to the committee chair, Philip N. Cohen, at pnc@umd.edu. To submit your paper, go to SocArXiv.org, and click “Add a preprint.” If you don’t yet have an account, you will fill out a short form — it’s free, non-profit, and won’t spam you! For assistance, contact socarxiv@gmail.com or consult the FAQ page. Please indicate whether you would like your paper to be included in a public list of submissions (this will not affect your chances of winning). The winner will receive a plaque and travel reimbursement up to $400 to attend the 2018 Family Section reception at the ASA meetings.

The Family Section Council will consider this proposal next month in Montreal. Please let us know what you think!

1 Comment

Filed under Me @ work

Paul Amato on reviewing Regnerus

I recently discussed Paul Amato’s role in the Regnerus Affair. I offered my opinion that, if Amato was a reviewer of the Regnerus article, he should not have been, mostly because he had served as a paid consultant on the study. (My long version of the Affair is here, a critique of the paper I co-authored is here.)

I regret that before writing that post I didn’t directly ask Amato if he wanted to discuss his role and whether he served as a reviewer. After the post appeared he sent me this statement, which I agreed to post. I added some more comments of my own below. (He also reminded me that he had voted for the Family Section Council resolution asking ASA to respond to the Regnerus study, which they did.)

Amato is President Elect of the National Council on Family Relations and a Distinguished Professor at Penn State University.

Thoughts on the Mark Regnerus (2012) study, by Paul Amato

One year has passed since Mark Regnerus (2012) published a highly controversial article on the children of parents who have same-sex relationships. Given that time tends to improve one’s perspective, this seems like a good time to reflect on the study and its aftermath.

My involvement

I worked for two days at the University of Texas as a consultant on the New Family Structures Study (NFSS). As I recall, seven consultants were at the meeting, along with a statistician from the survey research organization that later collected the data. I consulted primarily on sampling and measurement issues, and I was paid for two days of my time, plus travel expenses for myself (and my wife, who accompanied me). I charged for two days at my usual fee, which is $150 per hour. So I earned about $2,400. I received no further compensation after that.

About six months later, the editor of Social Science Research (SSR) asked me to review a manuscript written by Mark Regnerus. I informed the editor that I had worked as a paid consultant on the survey on which the manuscript was based. The editor said that he would like to have my views on the paper anyway, so I shared my views as honestly as I could.

This situation comes up now and then in my experience. When reviewing manuscripts for journals, I occasionally discover that I know the author and have some sort of relationship with the author or the study. In one case, for example, the author was a friend and colleague of mine, and I had read an earlier version of the paper and provided comments to the author. In this and every other case in which I have brought information like this to the editor’s attention, the editor has asked me to do the review anyway. Journal editors often have a difficult time getting reviews, and I assume they treat these reviews as one more data point. So the editor of SSR was doing what other editors do, as far as I know.

Was this particular case a conflict of interest for me? The American Sociological Association (ASA) defines a conflict of interest in the following manner:

Conflicts of interest arise when sociologists’ personal or financial interests prevent them from performing their professional work in an unbiased manner.

With respect to the Regnerus manuscript, I had no personal or financial interest in whether the paper was published. So by this definition, there was not a conflict of interest. Of course, sometimes there is the appearance of a conflict of interest. In these cases, the ASA code states:

Sociologists disclose relevant sources of financial support and relevant personal or professional relationships that may have the appearance of or potential for a conflict of interest…

As noted earlier, I disclosed to the editor that I had worked as a paid consultant on the NFSS. I also disclosed my role as a paid consultant in the commentary that I wrote for the Regnerus article, which appeared in SSR. I never attempted to hide the fact that I was part of the team that consulted on survey design.

In retrospect, I understand that providing a review was not a good idea, because one should avoid even the hint of impropriety in matters like this. At the time, however, I simply felt that I was helping the editor and being a good colleague.

Contrary to the views of some (but not all) of my colleagues, I thought the Regnerus manuscript was worth publishing. My key recommendation, however, was that the editor should publish the paper with commentaries from authors who hold a variety of perspectives, including gay and lesbian scholars who had published in this area. I believed that the Regnerus paper, accompanied by a diverse set of commentaries, could represent a useful contribution to the literature on LGBT families. Unfortunately, the editor was unable to recruit any gay or lesbian scholars to contribute commentaries, so my idea for an exchange of views fell flat. (The subsequent issue of SSR devoted to the controversy came closer to what I had envisioned.)

Almost everyone got it wrong

When the study was published, criticism from the political left was swift and harsh. Unfortunately, some commentary devolved into ad hominem attacks, accusations of fraud, and name-calling. Rather than intellectually engage the findings, the goal of some critics was to thoroughly discredit the study—and the author. While they were at it, many critics also attacked the editor, the reviewers, the consultants, those who wrote commentaries—even the survey research firm that collected the data! Anyone with any form of contact with the study became an enemy of the people.

This is unfortunate, because the political left could have benefitted from a strategic appropriation of the findings. The study involved a national sample of young adults with an LGBT parent. As the study noted, few of these young adults spent long periods of time in households with two parents of the same sex. Instead, most were born into heterosexual families that later broke up, presumably when one parent came out as gay or lesbian. Many of these youth went on to experience a variety of other family structures before reaching adulthood. One out of seven spent time in foster care. Previous research shows that instability in the family of origin increases the risk of a variety of long-term social and psychological problems for offspring. Consistent with this research, young adults in the study had modestly elevated problem profiles. It is reasonable to conclude that the elevated number of problems observed in these young adults was due to family instability rather than the sexual orientation of parents. For this reason, most observers have noted correctly that this study contributes nothing to our understanding of how children fare when raised by same-sex parents in stable households

Rather than dismiss these finding as being irrelevant, however, it’s useful to dig more deeply into the results. Why did these marriages end in divorce? More importantly, why did gays and lesbians wind up in heterosexual marriages in the first place? The explanation probably would go something like this: Like heterosexuals, many gays and lesbians wish to have families and raise children. But a generation ago, intolerance was the rule and discrimination against gays and lesbians was endemic. For many, forming heterosexual unions appeared to be the only way to achieve the dream of family and children. But these unions tended to be unstable, with problematic consequences for adults and children. Presumably, as our society becomes more accepting of LGBT families, the unfortunate circumstances of children and parents described in the Regnerus study will become less common. The freedom to marry, in particular, should increase stability in the lives of children with gay and lesbian parents.

In short, findings from the Regnerus study can be interpreted as strong evidence in support of same-sex marriage. The American Psychological Association and ASA research briefs emphasized the fact that almost all prior studies found no differences between children with heterosexual parents and children with gay or lesbian parents. The “no difference” perspective suggests that children will not be harmed by same-sex marriage. The lesson from the Regnerus study, however, is that children thrive on family stability, including children with gay and lesbian parents. We know that marriage tends to stabilize relationships, yet same-sex marriage is not allowed in most states. Given that children benefit from the stability provided by marriage, it is unfair and unkind to deny children the right to live with married parents. In contrast to the “no difference” perspective, a “family stability” perspective implies that we need to change our laws NOW to protect and benefit children.

If the political left missed an opportunity by failing to understand the full implications of the Regnerus study, the political right made even more serious blunders. Many conservative observers have cited the Regnerus study as if it provided evidence that being raised by gay or lesbian parents is harmful to children. This claim is disingenuous, because the study found no such thing. A noteworthy example came from Regnerus himself, who signed an amicus brief to the Supreme Court citing his study as evidence against same-sex marriage. This is curious because on page 766 in his 2012 article, Regnerus stated that his study was not intended to either affirm or undermine the legal right to same-sex marriage. And on page 768 of his response to the commentaries in the same issue, he stated that his data should not be used to press any political program. Given these cautious early statements, it is exasperating to see Regnerus later cite his own study as evidence against same-sex marriage.

Concluding thoughts

Many observers have argued that the Regnerus study should never have been published. It is important, however, to focus on what the study actually showed, and not on what people claim that it showed or wanted it to show. The study showed that family instability is not good for children, and many children with gay and lesbian parents, a generation ago, experienced a lot of family instability. It is not difficult to see how the personal problems of these families were affected by the restrictive social milieus in which they lived.

Since the Regnerus study was published, studies by Potter (2012) and Allen, Pakuluk, and Price (2013) have shown associations between having same-sex parents and child problems. Like the Regnerus paper, both of these articles survived the peer review process and, in fact, were published in top-tier social science journals. Rather than try to discredit these studies (and any future studies that may show similar results), it is better to examine the findings carefully and figure out what is going on. In fact, both studies are entirely consistent with the family stability perspective described earlier.

In conclusion, the political left discredited the Regnerus study without fully considering its findings, and the political right used the study disingenuously to further their political goals. Few people have focused thoughtfully on what the data actually show and what we can learn from the study. The controversy over the Regnerus study provides a sobering illustration of what can go wrong when ideology distorts social research.

References

Allen, Douglas W., Catherine Pakaluk, and Joseph Price. (2013). “Nontraditional Families and Childhood Progress through School: A Comment on Rosenfeld.” Demography 50:955-961.

Amato, Paul R. (2012). “The Well-Being of Children with Gay and Lesbian Parents.” Social Science Research 44:771-774.

American Sociological Association. (undated). American Sociological Association Code of Ethics (http://www.asanet.org/images/asa/docs/pdf/CodeofEthics.pdf)

Potter, Daniel. (2012). “Same-Sex Parent Families and Children’s Academic Achievement.” Journal of Marriage and Family 74:556-571.

Regnerus, Mark (2012). “How different are the adult children of parents who have same-sex relationships? Findings from the New Family Structures Study.” Social Science Research 41:752-770.

Regnerus, Mark (2012). “Response to Paul Amato, David Eggebeen, and Cynthia Osborne. Social Science Research 41:786-787.

wilcox-echo-chamber

Philip’s followup comments

I don’t think it was a gross ethical violation for Amato to review the paper, in which material or ideological gain lay behind his decision to do the review and led him to recommend publishing the paper against his better professional judgment. Rather, he thought it was a reasonable paper and offered that opinion when asked — which is unsurprising given his involvement in the project. So my real disagreement with Amato is over the value of the paper. I think it’s a worthless paper, done wrong, and only advanced because of the author’s ideological attachment to its results; that it accidentally helps reveal something true about family instability does not make it worth publishing.

Being a consultant on a small project like the NFSS is not like being a consultant on a giant project like the National Longitudinal Study of Adolescent Health. Consulting on Add Health, which has yielded thousands of publications and involved dozens of experts at many agencies, does not disqualify a person from reviewing a study using that data. In contrast, the NFSS has so far yielded one paper, solo-authored by the PI, who personally invited the several consultants. With only a few people involved, being one of them matters more.

Anyway, beyond the appearance of conflict, the problem with Amato serving as a reviewer is that it did not provide an outside perspective to the editor, James Wright. How could Amato’s review help Wright make the decision? Getting the input of a consultant on the project might help an editor shape a revision or build a special issue, but given Amato’s involvement his endorsement should not have counted as part of the peer review process. When Amato revealed his role, Wright should have declined his review.

Taking for granted the unethical behavior of Regnerus, and Brad Wilcox, on whose behalf Regnerus acted, the real failure here is by Wright. Instead of seriously reviewing the paper, he essentially whispered into an echo chamber of backers and consultants, “We should publish this, right?”

I believe the paper should be retracted because the conclusions are demonstrably wrong, because the author lied in the paper about the involvement of the institute that funded it, and because the peer review process was compromised by conflicts of interest. As long as this remains uncorrected, and James Wright remains editor, the integrity of the journal is indelibly tarnished.

While Wright is editor, I will no longer review for or submit to Social Science Research. I hope others will join me in that decision.

Comments that rehash well-known opinions or make person attacks will be shortened or deleted.

21 Comments

Filed under Me @ work

Paul Amato, Regnerus postscript

Scott Rose has a new post excoriating Paul Amato as a “villain” in the Regnerus Affair (here is the background).

Amato was arguably the most prominent sociologist involved in scandal: He is the President Elect of the National Council on Family Relations, the leading professional association for family scholars; he has been chair of the Family Section of the American Sociological Association; and he has a named chair at Penn State University, where he is a Distinguished Professor.

We know from his own commentary on the study in Social Science Research that Amato was a paid consultant to Regnerus (confirmed by UT-released documents). In that commentary, he cautioned that Regnerus’s results shouldn’t be used to undermine gay and lesbian civil rights, but he also lent legitimacy to the study and did not criticize its obvious flaws. I have seen the emails between Amato and Mark Regnerus that Scott Rose writes about here, which were obtained through public records document requests. I’m not revealing anything Rose hasn’t.

There are three new things in the emails:

  1. Amato apparently asked for, and received, money from Regnerus to bring his wife with him to Austin in addition to his own travel expenses and fees (“…about that second ticket…”, “…consider it done.”)
  2. Regnerus says he suggested Amato to Social Science Research editor James Wright as a reviewer for the study, and flatteringly urged Amato to accept the request if asked. (“I’d hope that if you’re asked to review it, you would consider doing so. I think you’re one of the fairest, level-headed scholars out there in this domain.”)
  3. Amato told Regnerus he defended Regnerus’s credibility after the study was published, and asked for access to the data for additional research.

However, Rose goes further and asserts that Amato also was a peer reviewer of the study for the journal Social Science Research, which he has said before. I believe I have seen the evidence Rose has for that and, although suggestive, it is not conclusive: Amato’s name is on a list that seems to be of reviewers for the Regnerus paper and the paper that accompanied it in the journal, which Darren Sherkat used for his investigation on behalf of the journal).

IInsist

It is already well established that Regnerus acted shamelessly and unethically. We now know that extended to not only suggesting his own consultant Amato as a reviewer to SSR, but also telling Amato he had done that and urging him to agree to do the review. I’m not sure what rule that breaks, but it’s wrong in my book. The fact that he had previously agreed to give Amato tickets for his wife’s travel just makes it a little worse, because it seems like calling in a favor. I assume Witherspoon, which bankrolled the study, didn’t care about that kind of slush, although the University of Texas Population Research Center might not appreciate that use of money they were managing. (At my state university, even money I have discretion over can’t be spent on things like family travel. If I want to pay for my family to travel, I can take the money as summer salary, pay taxes on it, and then spend it on whatever I want.)

Anyway, Regnerus isn’t an issue anymore. But what about Amato? Based on the new emails and the rest of the background, I have drawn these conclusions about so far:

  1. If Amato wants to take conservative foundation money from Regnerus for his family’s travel, that’s fine with me (I recommend he check with his tax adviser, however).
  2. If Amato wants to defend Regnerus’s credibility and use the data for additional research, that’s up to him.
  3. But if Amato was a reviewer for the paper in SSR, I believe he should not have been, even if he disclosed this to James Wright. Unless the anti-Regnerus activists are successful in getting SSR‘s files through public records requests we may never know for sure, unless Amato publicly discloses it. I hope he will.

I have no beef with Paul Amato, and no personal relationship with him. I served as a member of the Family Section council for some of the time he was the chair. Under his leadership, the Family Section accepted my proposal to request an amicus brief from the American Sociological Association on the Regnerus study claims. I don’t remember if he voted or abstained on the decision to make the request, but he didn’t block it, anyway.

Comment briefly, please.

20 Comments

Filed under Uncategorized

The Regnerus study goes to court, trailing briefs

The Regnerus study is going to court.

I wrote about the study previously here and here, and 200 researchers signed a letter about it. The claim of the study is that gay fathers and lesbian mothers are bad for children, and the basic critique is that the study doesn’t address that question (and, what it does address, it does poorly).

The paper was rushed into print with fanfare and press releases, just in time for it to be referenced in the case of Golinski v. United States Office of Personnel Management, in which the federal Defense of Marriage Act is being challenged, currently before the Ninth Circuit Court of Appeals. In fact, as Neal Caren pointed out to me, the brief was filed the day after the study was published online, June 11. (On a related DOMA case, here’s my take.)

Karin Golinski (right) with her spouse, Amy Cunninghis.

That reference came in a brief submitted by the American College of Pediatricians (not to be confused with the American Academy of Pediatrics). Taking Regnerus at his word that the study actually measured outcomes associated with “gay fathers” and “lesbian mothers,” they wrote:

A brand new study in the peer-reviewed journal Social Science Research uses a large random national sample to assess these outcomes. The study is based on interviews with 3,000 respondents, 175 of whom were raised by two women and 73 by two men. It looked at “social behaviors, health behaviors, and relationships” comparing child outcomes (as reported by the adult children rather than by those who raised them) among various groups including married biological parents (labeled as IBF for “intact biological family”) and children raised by same-sex couples (labeled LM for lesbian mothers and GF for gay fathers). On the forty outcomes measured, there were significant differences between those in the IBF and LM groups on twenty of those measures (the smaller sample size for fathers did not allow for as many findings of significance). Some of the statistically significant differences where children raised by two women fared worse than children raised by married biological parents included: cohabitation (9% of the IBF and 24% of the LM group), receiving welfare while growing up (17% of the IBF and 69% of the LM group), currently receiving public assistance (10% of the IBF and 38% of the LM group), current employment (49% of the IBF and 26% of the LM group), current unemployment (8% of the IBF and 28% of the LM group), having an affair while married or cohabiting (13% of the IBF and 40% of the LM group), having been touched sexually by a parent or other adult (2% of the IBF and 23% of the LM group), and ever having been forced to have sex against their will (8% of the IBF and 31% of the LM group). In addition, the children raised by two women were significantly less likely to identify as heterosexual (90% of the IBF and 61% of the LM group). Other measures where the children of same-sex couples had significantly greater experience than the children of married biological parents include marijuana use, smoking, being arrested, and numbers of sex partners.

In response, a brief by the American Psychological Association and others offered this correction:

Amicus American College of Pediatricians – not to be confused with amicus herein, the American Academy of Pediatrics – seriously mischaracterizes a recent study (“the Regnerus study”) as having compared children of married heterosexual parents with those “raised by same-sex couples.” Amicus Brief at 6. The Regnerus study placed participants (individuals between the age of 18 and 39) into one of eight categories, six of which were defined by the family structure in which they grew up — e.g., married biological parents, divorced parent, divorced but remarried parent, etc. There was no category for “same-sex couple.” Instead, the final two categories included all participants, regardless of family structure, who believed that at some time between birth and their 18th birthday their mother or their father “ever ha[d] a romantic relationship with someone of the same sex.” Hence the data does not show whether the perceived romantic relationship ever in fact occurred; nor whether the parent self-identified as gay or lesbian; nor whether the same sex relationship was continuous, episodic, or one-time only; nor whether the individual in these categories was actually raised by a homosexual parent (children of gay fathers are often raised by their heterosexual mothers following divorce), much less a parent in a long-term relationship with a same-sex partner. Indeed, most of the participants in these groups spent very little, if any, time being raised by a “same-sex couple.” Hence the Regnerus study sheds no light on the parenting of stable, committed same-sex couples – as Regnerus himself acknowledges – and thus it is gravely misleading to say, as the American College of Pediatricians does (p. 6), that the study involved 175 participants who “were raised by two women and 73 by two men.”

What is an association to do?

Should the American Sociological Association get involved?

Last year there was a spirited debate on the blogs and around the ASA about the ASA’s brief intervening in the Wal-Mart class-action discrimination case. See it at Orgtheory here, here, and here; and at Scatterplot here.

The association can take a minimal approach and simply point out that the Regnerus study doesn’t support the claims it’s carrying here, or it can take a maximal approach and evaluate the research on homogamous-couple parenting, as the APA and other organizations have done. Or it can do nothing.

It seems to be too late to submit something to the Ninth Circuit on the Golinksy case, but if it goes to the U.S. Supreme Court, or when one of the other marriage rights cases rises to this level, the opportunity — or obligation — will arise again.

13 Comments

Filed under Me @ work, Politics