Tag Archives: academia

Bogus versus extremely low-quality, Sullins edition

https://flic.kr/p/e6sfpP

Photo by CTBTO from Flickr Creative Commons (modified)

Calling a study “peer-reviewed” gives it at least some legitimacy. And if a finding is confirmed by “many peer-reviewed studies,” that’s even better. So the proliferation of bogus journals publishing hundreds of thousands of “peer-reviewed” articles of extremely low quality is bad news both for the progress of science and for public discourse that relies on academic research.

Two weeks ago I briefly reviewed some articles published by D. Paul Sullins, the anti-gay professor at Catholic University, on the hazards of being raised by gay and lesbian parents. I called the journals, published by Science Domain International (SDI), “bogus,” but said you could make an argument for extremely low quality instead.

After that Sullins sent me an email with some boilerplate from the publisher in defense of the journals, and he accused me of having a conflict of interest because his conclusions contradict one of my published articles. After correctly pointing out that a sting operation by Science failed to entrap an SDI journal with a bogus paper about cancer research, he said:

SDI is a new and emerging publisher. … While I would not say SDI is yet in the top tier, and I don’t like their journal names much either [which mimic real journal titles], for the reasons listed above I submit that this publisher is far from ‘bogus.’

How far from bogus?

Since that post, the reviews on the third of Sullins’ papers have been published by Science Domain and its journal, the (non-) British Journal of Education, Society & Behavioural Science. So we have some more information on which to judge.

The paper, “Emotional Problems among Children with Same-sex Parents: Difference by Definition,” was reviewed by three anonymous reviewers (from the USA, Brazil, Nigeria) and one identified as Paulo Verlaine Borges e Azevêdo, from Brazil. I summarize them here.

Anonymous USA

This reviewer only suggested minor revisions (nothing in the “compulsory revision” section). These were the suggestions: Avoid the first person, clarify the race of study participants, discuss the results in more detail, don’t use the word “trivial,” add citations to several statements, grammar check.

Anonymous, Brazil

This review demanded compulsory revisions: Clarify the level of statistical significance used, explain acronyms, clarify use of “biological parents” when discussing same-sex parents. And some minor revisions: one typo, one font-size change, standardize number of decimal places.

Anonymous, Nigeria

This reviewer included compulsory revisions: mention instrument used in the abstract, clarify measures used in previous studies on children’s well-being, test all four hypotheses proposed (not just three), clarify use of instrument used, shorten the discussion. Minor revisions: check for typos.

Paulo Verlaine Borges e Azevêdo, Brazil

This reviewer requested reorganizing the text, like this:

Would be better to redistribute the lengths of results (lessened), discussion (up) and conclusion (down) sections. In many moments, in the Result section the author deal with I believe would be better located in the Discussion (e. eg., between lines 345 and 355). I suggest that the subsections of Results would be reviewed by author and parts that discuss the results be transferred to the Discussion section … Strengths and Limitations would be better located in the discussion section too.

A few additional minor text modifications were included in the marked up manuscript.

Round two

Upon revision, Sullins was subjected to a punishing second round of reviews.

This included an interesting if ultimately fruitless attempt by Anonymous Brazil to object to this somewhat nutty sentence by Sullins: “biological parentage uniquely and powerfully distinguishes child outcomes between children with opposite-sex parents and those with same-sex parents.” What he meant was, when he controlled for the biological relationship between children and their parents — since hetero parents are more likely to have any biological parentage (and they’re the only ones with two bio parents) — it statistically reduced the gap in children’s mental health between married hetero versus same-sex parents. Although the exchange was meaningless in the decision whether to publish, and Sullins didn’t change it, and the reviewer dropped the objection, and the editors just said “publish it,” you would have to say this was a moment of actual review.

OK then

That’s it. None of this touched on the obvious fatal flaws in the study — that Sullins combines children in all same-sex families into one category while breaking those currently with different-sex parents into different groups (step-parents, cohabitors, single parents, etc.) — and that he has no data on how long the children currently with same-sex couples have lived with them, or how they came to live with them. So it leaves us right where we started on the question of same-sex parenting “effects” on children.

Of course, lots of individual reviews are screwed up. So, is this journal bogus or merely extremely low quality? Do we have a way of identifying these so-bad-they’re-basically-bogus journals that is meaningful to the various audiences they are reaching?

This matters is because journalists, judges, researchers, and the concerned public would like some way to evaluate the veracity of scientific claims that bear on current social controversies, such as marriage equality and the rights of gay and lesbian parents.

2 Comments

Filed under Research reports

Children in same-sex parent families, dead horse edition

Not that child well-being in different kinds of families isn’t a legitimate research topic, but this idea of proving same-sex parents are bad to whip up the right-wing religious base and influence court cases is really a shark jumping over a dead horse.

Without getting into all the possible detail and angles, here are some comments on the new research published by D. Paul Sullins, which claims to show negative outcomes for children with same-sex parents. Fortunately, I believe the legal efficacy of this kind of well-being witch-hunt research evaporated with Anthony Kennedy’s Windsor decision. Nevertheless, the gay-parents-are-bad-for-kids research community is still attempting to cause harm, and they still have big backers, so it’s important to respond to their work.

Research integrity

Below I will comment a little on the merits of the new studies, but first a look at the publication process and venues. As in the case of the Regnerus affair, in which Brad Wilcox, Mark Regnerus, and their backers conspired to manufacture mainstream legitimacy, Sullins is attempting to create the image of legitimate research, which can then be cited by advocates to the public and in court cases.

Although he has in the past published in legitimate journals (CV here), Sullins’ work now appears to have veered into the netherworld of scam open access journals (which, of course, does not include all open-access journals). Maybe this is just the decline of his career, but it seems they think a new round of desperate “peer-reviewed” publishing will somehow help with the impending legal door-slam against marriage inequality, so they’re rushing into these journals.

Sullins has three new articles about the mental health of children with same-sex parents. The first, I think, is “Bias in Recruited Sample Research on Children with Same-Sex Parents Using the Strength and Difficulties Questionnaire (SDQ).” This was published in the Journal of Scientific Research and Reports. The point of it is that same-sex parents who are asked to report about their children’s well-being exaggerate how well they’re doing.

The second paper is “Child Attention-Deficit Hyperactivity Disorder (ADHD) in Same-Sex Parent Families in the United States: Prevalence and Comorbidities.” It was published on January 21 in the British Journal of Medicine & Medical Research. It claims that children living with same-sex parents, surveyed in the National Health Interview Survey, are more likely to have ADHD than “natural” children of married couples.

The third — the one I call third because it doesn’t seem to have actually been published yet — is, “Emotional Problems among Children with Same-sex Parents: Difference by Definition,” in the British Journal of Education, Society & Behavioural Science. It’s point is the same as the second, with slightly different variables. (The author’s preprint is here.) This is the one Mark Regnerus referred to in a post calling attention to Sullins’ work. (The legitimacy strategy is apparent in Regnerus naming the fancy-sounding journal in the opening sentence of his post.)

What makes these scam journals? The first clue is that two of them have “British” in the name, despite not being British in any way (not that there’s anything wrong with that). They are all published by Science Domain, which is listed on “Beall’s List” of “potential, possible, or probable predatory scholarly open-access publishers.” They are not published by academic societies, they are not indexed by major academic journal databases, they publish thousands of papers with little or no peer review (at the expense of the authors), and they recruit authors, editors, and reviewers through worldwide spam campaigns that sweep up shady pseudo-scholars.

For the first two, which have been published, Science Domain documents the review process. The first paper, “Bias in Recruited Sample…,” first had to overcome Reviewer 1, Friday Okwaraji, a medical lecturer at the University of Nigeria, who recommended correcting a single typo. Reviewer 2, identified as “anonymous/Brazil,” apparently read the paper, suggesting several style changes and moving some sentences, and expressing misgivings about the whole point. After revisions, the editor considered the two reviews carefully, and then wrote to the managing editor, “Please accept the paper, it is okay.” It was submitted November 18, 2014 and accepted December 17, 2014.

The second paper, “Child ADHD…,” also shows its peer review process. Reviewer 1 was Renata Marques de Oliveira at the University of São Paulo, Brazil. In 2012 she was listed as a masters student in psychiatric nursing, and is now an RN. This is the entirety of her review of Sullins’ paper:

sullinsreview1

OK, then.

The second review is by Rejani Thudalikunnil Gopalan, described as a faculty member at Universiti Malaysia Sabah, or maybe Gujarat Forensic Sciences University, Gujarat, India. She was recently spotted drumming up submissions for a special issue of the scammy American Journal of Applied Psychology (“What? We didn’t say it was the same journal as the Journal of Applied Psychology, published by the American Psychological Association!”). The journal AJAP is published by the Science Publishing Group (see Beall’s List), but I couldn’t investigate further because their website happens to be down.

Unlike Oliveira, Gopalan seems to have read the paper, and offered a few superficial questions and suggestions – not quite the very worst review from a legitimate journal that I have ever read. After a cursory reply, the editor responded (in full): “The authors have addressed all reviewers’ concerns in a satisfactory way. This is an outstanding paper worthy of publication in BJMMR.” It was accepted two weeks after submission.

I don’t want to imply that three journals are illegitimate just because they are run for profit by low-status academics from developing countries. But looking at the evidence so far I think it’s fair to call these journals bogus. However, I wouldn’t argue too much if you wanted instead to say they are merely of the very lowest quality.

Why does a guy at a real university, with tenure, publish three articles in two months at a paper mill like Science Domain? I fear our dear Dr. Sullins has fallen out of love with the scientific establishment. Anyways.

Content

You might say we should just ignore these papers because of their provenance, but they’re out there. Plus, I want people to take my totally unreviewed blog posts seriously, so I should take these at least a little seriously. Fortunately, I can write them off based on simple, complete objections.

Combining the 1997-2013 National Health Interview Surveys, about 200,000 children, Sullins gets 512 children who are living with a same-sex couple (about 16% married, he says). In both the second and third papers, he compares these children to those living with married, biological or adoptive parents who are of different sexes. The basic problem here is obvious, and was apparent in the infamous Regnerus paper as well: same-sex couples, regardless of their history — married, divorced, never-married, just-married, married before the kid was born, just got together yesterday when the kid was 15, and so on — are all combined in one undifferentiated category. This just can’t show you the “effect” of same-sex parenting. (When Regnerus says this research supports the ” basic narrative … that children who grow up with a married mother and father fare best at face value,” he’s slipping in “grow up with,” though he knows the study doesn’t have the information necessary to make that claim.)

However, if Sullins did the data manipulations right — which I cannot judge because I don’t know the data, little detail is provided, and the reviewers have no expertise with it either — there is a simple descriptive finding here that is interesting, if unsurprising: children living with same-sex parents over the period 1997-2013, the vast majority of whom are not married, and presumably did not conceive or adopt the child in their relationship, have more emotional problems and ADHD than children living with their married, biological parents. We have to be smart enough to consider that — if it’s true — without falling into accepting the claim that such problems are the result of same-sex parenting, because that has not been established. Of course, this supports an argument for marriage equality, but it’s also just an empirical pattern worth understanding. If Sullins, Regnerus, and their ilk weren’t so hellbent on opposing homosexuality they could actually provide useful information that might be part of a knowledge base we use to improve children’s lives.

Sullins’ judgment is no doubt clouded by his overarching religious objection to homosexuality, which, he believes, like abortion and contraception,

contravene the natural operation of the body in order to conform human sexuality to the ideals of modernity… By severing the link between sex and children, both [abortion and homosexuality] increase privatization, diminish the social intentionality and form of the sexual union, and undermine the unitive good and the transcendent goal of marriage.

So for him it’s already settled — long before he extruded these papers (and Regnerus has expressed similar views). Apparently they think they just need a few bogus publications to bring the public along.

29 Comments

Filed under Research reports

Exceptions overwhelm the rules in economics

I wrote a short essay for the New York Times Room for Debate feature. The question was, “Have we given economists too much authority?” Here’s my answer, as edited by the Times. You can read the other essays and comments here.

rfdecon-sfSpan

Exceptions Overwhelm Economic Rules

There is a lot to be said for the common critique of economists: They see society as the product of freely acting, rationally calculating individuals for whom monetary reward is the primary source of motivation. Free markets, to them, are the pure expression of social function and economic growth through their realization is the only outcome that matters.

But people do not simply act rationally to maximize their economic rewards, because they can have incomplete or inaccurate information, ideological biases, conflicting desires or collective interests. Exploitation, dishonesty, violence, ignorance and demagoguery set vast areas of social life apart outside the model. The multiplying exceptions overwhelm the rule bringing the model’s utility into question.

Group behavior and social structure are central to understanding society. Collective identity yields networks of solidarity that drive social interaction in ways individual self-interest alone cannot determine. Economic growth is one of many legitimate goals.

In reality, many economists don’t hew so firmly to these mainstream dogmas. But economists’ influence is largely proportional to the degree with which their analysis comports with the interests of those who make the most influential decisions. The free market orientation, individualist logic and materialist values of some economists serve well the captains of industry (or, nowadays, of finance), who in turn reward their compliant consultants with privileged perches around the seats of power.

Jeb Bush reflected this alliance in his speech to the Detroit Economic Club on Wednesday, when he asked, “If a law or a rule doesn’t contribute to growth, why do it?” Going out on a limb, other justifications for government action might include reducing inequality, improving social cohesion, reducing conflict, enhancing health or protecting the environment.

If their influence is dependent on their contribution to already-powerful agendas, maybe economists don’t have as much real influence as it seems. On the other hand, people with training in the other social sciences have more impact than we often think, partly because they work not as “sociologists,” say, but under job titles such as analyst, demographer, statistician, consultant, teacher, organizer or survey director.

Of course, the common belief that economists have outsized influence is not wholly false, and they have worked hard to build it, but the uncritical acceptance of that image is part of what makes it a reality.

5 Comments

Filed under Me @ work

On Contexts: Sociology unfound

Over on the Contexts blog, I wrote a follow-up to Justin Wolfers’ piece about economists dominating the news: here it is.

1 Comment

Filed under In the news

Campus sexual assault op-ed

chron14

I have an op-ed in the Chronicle of Higher Education titled, “College Sex-Assault Trials Belong in Court, Not on Campus,” and there’s already a lively discussion in the comments. I would welcome your thoughts, on this site or that one.

3 Comments

Filed under Me @ work

9 uses of “the ways in which” that should be replaced by “how”

Searching through sociology for the ways in which is literally like shooting ducks in a barrel (easy).

Shooting Ducks in a Barrel

For this post I made sure to include some giants in the field, and major journals, to underscore the ways in which this problem is not limited to the over-wrought fringe.

The “how” rule is not universally applicable. In some cases “the ways” would be a better replacement. But in these 9 examples “how” is enough.

Reproducing Stories: Strategic Narratives of Teen Pregnancy and Motherhood

Within this narrative, there is no space for negotiating or even acknowledging the ways in which poverty, racism, and sexism affect the lives of young mothers.

Social Network Analysis: An Introduction

… her research showed the importance of ties across kin groups and households and the ways in which the strength of membership within families varied…

The New Institutionalism in Organizational Analysis

Whereas economists and political scientists offer functional explanations of the ways in which institutions represent efficient solutions to problems of governance, sociologists reject functional explanations and focus instead on the ways in which institutions complicate and constitute the paths by which solutions are sought.

The Transformation of Intimacy: Sexuality, Love, and Eroticism in Modern Societies

She found major differences between the ways in which the boys discussed sex (they did not often speak of love) in the course of her lengthy interviews with them and the responses of the girls.

Practicing Intersectionality in Sociological Research: A Critical Analysis of Inclusions, Interactions, and Institutions in the Study of Inequalities

A good example of this can be found in Hays’s (2003) discussion of the ways in which college students are prone to the drug use and sexual activity that are so strongly condemned among poor teens.

The Division of Labor in Society (Introduction)

The new introduction to this edition takes a different tack, focusing on the ways in which this work is of present-day sociological interest.

Video Game Culture, Contentious Masculinities, and Reproducing Racialized Social Class Divisions in Middle School*

Recent feminist theorizing on relations between gender and technology emphasizes the ways in which the two mutually shape each other.

Pattern Variables Revisited: A Response to Robert Dubin

The Editor’s invitation to comment on his paper has given me the opportunity to work out an overdue clarification of the ways in which Model II builds on and goes beyond, rather than replaces, Model I.

Gastronationalism: Food Traditions and Authenticity Politics in the European Union

I use Brubaker’s (1996:10) broad definition of nationalism — a set of idioms, practices, and possibilities available in cultural and political life, delimited by social or physical boundaries — to consider the ways in which a nation’s people are defined, or self-define, as a distinct group.

*This was one of eight pieces in the Summer 2014 issue of Signs that came up in my search. For a previous criticism of the writing in Signs, see this post.

9 Comments

Filed under Research reports

How (and how much) academics talk about inequality, in one chart

Reader advisory: When I say “in one chart,” I never really mean it.

Updated with new chart at the end.

Because someone asked, here is the article count from Web of Science (an academic journal database with emphasis on science), showing the frequency of articles (of all types) according to the inequality-related phrases in their titles. This is obviously not an exhaustive list of work on these subjects, but I did want to show all combinations of race, class, and gender (click to enlarge).

strat terms.xlsx

  • “Social inequality” now completely dominates, but it once was second to “social stratification.”
  • The most common of the three-word combinations is “race, class, and gender.”
  • “Gender, race, and class” has almost always been second.
  • “Gender, class, and race” made a run in the late 1990s, but has since faded.

I’ve written a little more about language and intersectional concerns here.

Update:

Don Tomaskovic-Devey sent along this figure, which shows newspaper articles using inequality related terms. The dotted line shows articles with rich, wealthy, top 1%, top one %, while the solid line shows income inequality. He suggests the dotted line may reflect an Occupy Wall Street effect, while the solid line shows the Thomas Piketty framing process:

lexisineq

 

3 Comments

Filed under Research reports