Rural COVID-19 paper peer reviewed. OK?

Twelve days ago I posted my paper on the COVID-19 epidemic in rural US counties. I put it on the blog, and on the SocArXiv paper server. At this writing the blog post has been shared on Facebook 69 times, the paper has been downloaded 149 times, and tweeted about by a handful of people. No one has told me it’s wrong yet, but not one has formally endorsed it yet, either.

Until now, that is. The paper, which I then submitted to the European Journal of Environment and Public Health, has now been peer reviewed and accepted. I’ve updated the SocArXiv version to the journal page proofs. Satisfied?

It’s a good question. We’ll come back to it.

Preprints

The other day (I think, not good at counting days anymore) a group of scholars published — or should I say posted — a paper titled, “Preprinting a pandemic: the role of preprints in the COVID-19 pandemic,” which reported that there have already been 16,000 scientific articles published about COVID-19, of which 6,000 were posted on preprint servers. That is, they weren’t peer-reviewed before being shared with the research community and the public. Some of these preprints are great and important, some are wrong and terrible, some are pretty rough, and some just aren’t important. This figure from the paper shows the preprint explosion:

All this rapid scientific response to a worldwide crisis is extremely heartening. You can see the little sliver that SocArXiv (which I direct) represents in all that — about 100 papers so far (this link takes you to a search for the covid-19 tag), on subjects ranging from political attitudes to mortality rates to traffic patterns, from many countries around the world. I’m thrilled to be contributing to that, and really enjoy my shifts on the moderation desk these days.

On the other hand some bad papers have gotten out there. Most notoriously, an erroneous paper comparing COVID-19 to HIV stoked conspiracy theories that the virus was deliberately created by evil scientists. It was quickly “withdrawn,” meaning no longer endorsed by the authors, but it remains available to read. More subtly, a study (by more famous researchers) done in Santa Clara County, California, claimed to find a very high rate of infection in the general population, implying COVID-19 has a very low death rate (good news!), but it was riddled with design and execution errors (oh well), and accusations of bias and corruption. And some others.

Less remarked upon has been the widespread reporting by major news organizations on preprints that aren’t as controversial but have become part of the knowledge base of the crisis. For example, the New York Times ran a report on this preprint on page 1, under the headline, “Lockdown Delays Cost at Least 36,000 Lives, Data Show” (which looks reasonable in my opinion, although the interpretation is debatable), and the Washington Post led with, “U.S. Deaths Soared in Early Weeks of Pandemic, Far Exceeding Number Attributed to Covid-19,” based on this preprint. These media organizations offer a kind of endorsement, too. How could you not find this credible?

Peer review

To help sort out the veracity or truthiness of rapid publications, the administrators of the bioRxiv and medRxiv preprint servers (who are working together) have added this disclaimer in red to the top of their pages:

Caution: Preprints are preliminary reports of work that have not been certified by peer review. They should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

That’s reasonable. You don’t want people jumping the gun on clinical decisions, or news reports. Unless they should, of course. And, on the other hand, lots of peer reviewed research is wrong, too. I’m not compiling examples of this, but you can always consult the Retraction Watch database, which, for example, lists 130 papers published in Elsevier journals in 2019 that have been retracted for reasons ranging from plagiarism to “fake peer review” to forged authorship to simple errors. The database lists a few peer-reviewed COVID-19 papers that have already been retracted as well.

This comparison suggests that the standard of truthiness cannot be down to the simple dichotomy of peer reviewed or not. We need signals, but they don’t have to be that crude. In real life, we use a variety of signals for credibility that help determine how much to trust a piece of research. These include:

• The reputation of the authors (their degrees, awards, twitter following, media presence)
• The institutions that employ them (everyone loves to refer to these when they are fancy universities reporting results they favor, e.g., “the Columbia study showed…”)
• Who published it (a journal, an association, a book publisher), which implies a whole secondary layer of endorsements (e.g., the editor of the journal, the assumed expertise of the reviewers, the prestige or impact factor of the journal as a whole, etc.)
• Perceived conflicts of interest among the authors or publishers
• The transparency of the research (e.g., are the data and materials available for inspection and replication)
• Informal endorsements, from, e.g., people we respect on social media, or people using the Plaudit button (which is great and you should definitely use if you’re a researcher)
• And finally, of course, our own assessment of the quality of the work, if it’s something we believe ourselves qualified to assess

As with the debate over the SAT/GRE for admissions, the quiet indicators sometimes do a lot of the work. Call something a “Harvard study” or a “New York Times report,” and people don’t often pry into the details of the peer review process.

Analogy: People who want to eat only kosher food need something to go on in daily life, and so they have erected a set of institutional devices that deliver such a seal (in fact, there are competing seal brands, but they all offer the same service: a yes/no endorsement by an organization one decides to trust). The seals cost money, which is added to the cost of the food; if people like it, they’re willing to pay. But, as God would presumably tell you, the seal should not always substitute for your own good judgment because even rabbis or honest food producers can make mistakes. And in the absence of a good kosher inspection to rely on altogether, you still have to eat — you just have to reason things through to the best of your ability. (In a pinch, maybe follow the guy with the big hat and see what he eats.) Finally, crucially for the analogy, anyone who tells you to ignore the evidence before you and always trust the authority that’s selling the dichotomous indicator is probably serving their own interests as least as much as they’re serving yours.

In the case of peer review, giant corporations, major institutions, and millions of careers depend on people believing that peer review is what you need to decide what to trust. And they also happen to be selling peer review services.

My COVID-19 paper

So should you trust my paper? Looking back at our list, you can see that I have degrees and some minor awards, some previous publications, some twitter followers, and some journalists who trust me. I work at a public research university that has its own reputation to protect. I have no apparent way of profiting from you believing one thing or another about COVID-19 in rural areas (I declared no conflicts of interest on the SocArXiv submission form). I made my data and code available (even if no one checks it, the fact that it’s there should increase your confidence). And of course you can read it.

And then I submitted it to the European Journal of Environment and Public Health, which, after peer review, endorsed its quality and agreed to publish it. The journal is published by Veritas Publications in the UK with the support of Tsinghua University in China. It’s an open access journal that has been publishing for only three years. It’s not indexed by Web of Science or listed in the Directory of Open Access Journals. It is, in short, a low-status journal. On the plus side, it has an editorial board of real researchers, albeit mostly at lower status institutions. It publishes real papers, and (at least for now) it doesn’t charge authors any  publication fee, it does a little peer review, and it is fast. My paper was accepted in four days with essentially no revisions, after one reviewer read it (based on the summary, I believe they did read it). It’s open access, and I kept my copyright. I chose it partly because one of the papers I found on Google Scholar during my literature search was published there and it seemed OK.

So, now it’s peer reviewed.

Here’s a lesson: when you set a dichotomous standard like peer-reviewed yes/no and tell the public to trust it, you create the incentive for people to do the least they can to just barely get over that bar. This is why we have a giant industry of tens of thousands of academic journals producing products all branded as peer reviewed. Half a century ago, some academics declared themselves the gatekeepers of quality, and called their system peer review. To protect the authority of their expertise (and probably because they believed they knew best), they insisted it was the standard that mattered. But they couldn’t prevent other people from doing it, too. And so we have a constant struggle over what gets to be counted, and an effort to disqualify some journals with labels like “predatory,” even though it’s the billion-dollar corporations at the top of this system that are preying on us the most (along with lots of smaller scam purveyors).

In the case of my paper, I wouldn’t tell you to trust it much more because it’s in EJEPH, although I don’t think the journal is a scam. It’s just one indicator. But I can say it’s peer reviewed now and you can’t stop me.

Aside on service and reciprocity: Immediately after I submitted my paper, the EJEPH editors sent me a paper to review, which I respect. I declined because it wasn’t qualified, and then they sent me another. This assignment I accepted. The paper was definitely outside my areas of expertise, but it was a small study quite transparently done, in Nigeria. I was able to verify important details — like the relevance of the question asked (from cited literature), the nature of the study site (from Google maps and directories), the standards of measurement used (from other studies), the type of the instruments used (widely available), and the statistical analysis. I suggested some improvements to the contextualization of the write-up and recommended publication. I see no reason why this paper shouldn’t be published with the peer review seal of approval. If it turns out to be important, great. If not, fine. Like my paper, honestly. I have to say, it was a refreshing peer review experience on both ends.

Sci-Hub users cost ASA journals thousands of downloads, and that’s OK

UPDATED to include Sci-Hub data from six months: September 2015–February 2016, and correcting a coding error that inflated download counts.

---

Well, they might not have lost the downloads, but they didn’t get them.

Sci-Hub is a pirate operation that uses stolen university login credentials to harvest, store, and distribute for free virtually every academic article published anywhere. It is a simple, if criminal, solution to a very big problem: the lack of access to published research for people who can’t pay for it. When someone goes to the Sci-Hub site and requests an article, by simply pasting in the DOI or URL, the system either serves them the paper, or goes and steals it for them and then keeps a copy for the next user. For us university people who are used to dealing with the maze of logins and forwarding and proxies that come between us and the information we seek, it’s unbelievably fast and almost never fails.

Their most recent claim is an archive of 76 million papers and 400,000 users per day.

Today is 5 September and Sci-Hub is 8 years old. In eight years, the website grew from zero to 76,000,000 research articles available for free reading, and from 2,000 users per day to 400,000.

— Sci Hub (@Sci_Hub) September 5, 2019

Currently available at sci-hub.se or –.tw, it sometimes moves, but this site always lists where you can find it now. Naturally, both civil and criminal authorities are trying to shut it down, preferably by catching its mastermind, Alexandra Elbakyan, the elusive student programmer from Kazakhstan.

That picture is from the excellent (free streaming) documentary Paywall: The Business of Scholarship. Chris Bourg, the Director of Libraries at MIT (and a sociologist), also interviewed in the movie, said of Sci-Hub:

Those of us who work in scholarly communications, writ large, really have to look at Sci-Hub as sort of a poke in the side that says, “Do better.” We need to look to Sci-Hub to say, “What is it that we could be doing differently about the infrastructure that we developed to distribute journal articles, to distribute scholarship?” … I think we need to look at what’s happening with Sci-Hub, how it evolved, who’s using it, who’s accessing it, and let it be a lesson to us for what we should be doing differently.

Sociology’s stolen papers

Science magazine writer John Bohannon reached Elbakyan in 2016, and she turned over to him a 6-month cache of Sci-Hub server logs for a piece titled, “Who’s downloading pirated papers? Everyone.” He analyzed 28 million downloads, and Science made the data available for analysis, here. Eight million of those hits were from India and China, and the busiest location was Tehran.

The data archive includes only the time and date, the DOI number of each paper downloaded, and the location of the user. I’m not expert in DOI analysis, but Bohannon included a guide that shows the prefix 10.1177 is associated with Sage Publications, which publishes the American Sociological Association’s journals. Looking at the entire six-month series, September 2015 — February 2016, I found 171,000 Sage items, downloaded 377,000 times. Of those (if I got the DOIs right), 805 titles downloaded 1628 times came from the ASA research journals (my Stata code is here).

---

ASA / Sage downloads from Sci-Hub, Sept 2015 – Feb 2016
	Articles	Downloads
American Sociological Review	239	693
Teaching Sociology	221	269
Journal of Health and Social Behavior	94	188
Social Psychology Quarterly	77	152
Sociology of Education	73	157
Sociological Methodology	57	76
Sociological Theory	44	93
Total	805	1628

---

On an annualized basis, that would be 750,000 Sage downloads, and 3,200 from ASA journals specifically. For comparison, the most popular article in ASR in 2017 was downloaded about 10,000 times from the Sage site, so it’s a small share of the legitimate traffic. So over the life of Sci-Hub it cost (and saved) ASA thousands of downloads, probably a few tens of thousands. [Note in the first version of this post, I had a coding error that multiplied the counts, and this read “hundreds of thousands”. I regret the error.]

The most-downloaded ASR paper for the entire period was:

Mears, Ashley. 2015. “Working for Free in the VIP: Relational Work and the Production of Consent.” American Sociological Review 80 (6): 1099–1122. (downloaded 33 times)

The most-downloaded from a different journal was:

Kanazawa, Satoshi. 2010. “Why Liberals and Atheists Are More Intelligent:” Social Psychology Quarterly, February. (29 times)

I looked at a couple of them in more detail, and found, for example, that Paula England’s 2015 ASA Presidential Address was downloaded by users in Seoul (South Korea), Durban (South Africa), New Delhi, London, Chicago, Washington, and Virgie (Kentucky).

Interestingly, at least one of the popular papers, Lizardo et al.’s introduction to their editorial tenure at ASR, is already ungated on the Sage site, so you don’t need to use Sci-Hub to get it. This suggests, as Bohannon also noted, that some Sci-Hub users are just using the site because it’s convenient, not because they don’t have access to the papers.

Do you Sci-Hub?

I use Sci-Hub a lot, often for things that I also have subscription access to. (I do not, however, contribute anything to the system; I free-ride off their criminality.) Why? I’m not in the paywall game business, I’m in the information business. I am always behind on my work, and adding a few seconds or minutes of hunting for the legitimate way to get each of the many articles I look at every day is not worth it. (And when I find my university doesn’t subscribe? Interlibrary loan is wonderful, but I don’t want to spend more time with it than necessary.) Does my choice cost the American Sociological Association a few cents, by reducing legitimate downloads, which somehow factors into the profits that get kicked back to the association from Sage? I don’t know.

Of course, one of the dumb things about the paywall system is that it’s expensive and time-consuming to manage who has access to what information — it’s not a small task to keep information from reaching millions of determined readers from all around the world. (I assume one of the reasons my university recently introduced two-factor authentication — requiring me to click a pop-up on my phone every time I log in to university resources [even when I’m in my office] — is because of Sci-Hub. Ironic!)

Chris Bourg is right: “let it be a lesson to us for what we should be doing differently.” Elbakyan may have committed the most efficient product theft in history, in terms of list price of stolen goods per unit of effort or expense on her part. Her archive has been copied and distributed to different sites around the world (it fits in a large suitcase). And it was made possible by the irrational, corrupt nature of the scholarly communication infrastructure. Her success is the system’s failure.

For more information, read my report, “Scholarly Communication in Sociology.“

 

ASA’s letter against the public interest and our values

Update 1: I submitted a resolution to the ASA Committee on Publications, for consideration at our January meeting. You can read and comment on it here.

Update 2: The Committee on Publications on January 23 voted to approve the following statement: “The ASA Committee on Publications expresses our opposition to the decision by the ASA to sign the December 18, 2019 letter.”

The American Sociological Association has signed a letter that profoundly betrays the public interest and goes against the values that many of us in the scholarly community embrace.

The letter to President Trump, signed by dozes of academic societies, voices opposition to a rumored federal policy change that would require federally funded research be made freely available upon publication, rather than according to the currently mandated 12-month embargo — which ASA similarly, bitterly, opposed in 2012. ASA has not said who made the decision to sign this letter. All I know is that, as a member of the Committee on Publications, I wasn’t consulted or notified. I don’t know what the ASA rules are for issuing such statements in our name, but this one is disgraceful.

The argument is that ASA would not be able to make money selling research generated by federal funding if it were required to be distributed for free. And because ASA would suffer, science and the public interest would suffer. Like when Trump says getting Ukraine to help him win re-election is by definition in the American interest — what helps ASA is what’s good for science.

The letter says:

Currently, free distribution of research findings is subject to a 12-month embargo, enabling American publishers to recover the investment made in curating and assuring the quality of scientific research content. … The current 12-month embargo period provides science and engineering society publishers the financial stability that enables us to support peer review that ensures the quality and integrity of the research enterprise.

That is funny, because in 2012 ASA director Sally Hillsman (since retired) said the 12-month embargo policy “could threaten the ability of scholarly societies, including the ASA, to continue publishing journals” and was “likely to seriously erode and eventually jeopardize our financial ability to perform the critical, value added peer review and editorial functions of scientific publishing.”

The current letter, at least with regard to ASA, tell this whopper: “we support open access and have a strong history of advancing open access through a broad array of operational models.” They literally oppose open access, including in this letter, and including the current, weak, open access policy.

The ASA-signed letter is very similar to one sent about the same time by a different (but overlapping) large group of publishers, including Elsevier, and the U.S. Chamber of Commerce, claiming the rumored policy would hurt ‘merica. But there are subtle differences. The ASA letter refers to “the current proven and successful model for reporting, curating and archiving scientific results and advancing the U.S. research enterprise,” which should not be tampered with. The other letter warns of the danger of “step[ing] into the private marketplace” in which they sell research. Knowledge philosopher Peter Suber offered an excellent critique of the market claims here in this Twitter thread:

1/ Many of the new publisher lobbying statements use the standard rhetoric that government #openaccess policies interfere with the market. Many go a step further and assert that the market is working well.

For example:

— Peter Suber (@petersuber) December 19, 2019

ASA and the other money-making societies really want you to believe there is no way to do curation and peer review without them. If we jeopardize their business model, ASA says, the services they provide would not happen. In fact, the current subscription models and paywalls stand in the way of developing the cheaper, more efficient models we could build right now to replace them. All we need to do is take the money we currently devote to journal subscriptions and publisher profits, and redirect it to the tasks of curation and peer review without profits and paywalls — and free distribution (which is a lot cheaper to administer than paywalled distribution).

The sooner we start working on that the better. In this effort — and in the absence of leadership by scholarly societies — the university libraries are our strongest allies. This is explained by UNC Librarian Elaine Westbrooks in this Twitter thread:

People in the US are getting all worked up over the threat that articles produced as a result of federally funded research could be open.

— Elaine L Westbrooks (@UNC_Librarian) December 19, 2019

Compare this forwarding thinking librarian’s statement with Elsevier. In proudly sharing the publishers’ statement, Elsevier vice president Ann Gabriel said, “Imagine a world without scientific, medical societies and publishers who support scholarship, discovery and infrastructures for peer review, data archiving and networks.” Notice two things in this statement. First, she does not mention libraries, which are the academy-owned institutions that do literally all this as well. And second, see how she bundles publishers and societies. This is the sad reality. If instead of “societies and publishers” we had “societies and libraries” maybe we’d be getting somewhere. Instead, our societies, including the American Sociological Association, are effectively captured by publishers, and represent their interests instead of the public interest, and the values of our community.

I remain very pessimistic about ASA, which is run by a professional group with allegiance to the paywall industry, along with mostly transient, naive, and/or ineffectual academics (of which I am certainly one). But I’m torn, because I want to see a model of scholarly societies that works, which is why I agreed to serve of the ASA Committee on Publications — which mostly does busy work for the association while providing the cover of legitimacy for the professional staff.

Letter of opposition

So I posted a letter expressing opposition to the ASA letter. If you are a sociologist, I hope you will consider sharing and signing it. We got 100 signatures on the first day, but it will probably take more for ASA to care. To share the letter, you can use this link: https://forms.gle/ecvYk3hUmEh2jrETA.

It reads:

In light of a rumored new White House Open Access Policy, the American Sociological Association (ASA), and other scholarly societies, signed a letter to President Trump in support of continued embargoes for federally-funded research.

We are sociologists who join with libraries and other advocates in the research community in support of federal policy to make the results of taxpayer-funded research immediately available to the public for free. We endorse a policy that would eliminate the current 12-month waiting period for open access to the outputs of taxpayer-funded scientific research. Ensuring full open access to publicly-funded research contributes to the public good by improving scientific productivity and equalizing access — including international access — to valuable knowledge that the public has already paid for. The U.S. should join the many other countries that already have strong open access policies.

We oppose the decision by ASA to sign this letter, which goes against our values as members of the research community, and urge the association to rescind its endorsement, to join the growing consensus in favor of open access to to scholarship, including our own.

The Coming Divorce Decline, Socius edition

“The Coming Divorce Decline, ” which I first posted a year ago, has now been published by the journal Socius.  Three thousand people have downloaded it from SocArXiv, I presented it at the Population Association, and it’s been widely reported (media reports), but now it’s also “peer reviewed.” Since Socius is open access, I posted their PDF on SocArXiv, and now that version appears first at the same DOI or web address (paper), while the former editions are also available.

Improvement: Last time I posted about it here I had a crude measure of divorce risk with one point each for various risk factors. For the new version I fixed it up, using a divorce prediction model for people married less than 10 years in 2017 to generate a set of divorce probabilities that I apply to the newly-wed women from 2008 to 2017:

…the coefficients from this model are applied to newly married women from 2008 to 2017 to generate a predicted divorce probability based on 2017 effects. The analysis asks what proportion of the newly married women would divorce in each of their first 10 years of marriage if 2017 divorce propensities prevailed and their characteristics did not change.

The result looks like this, showing the annual probability falling from almost 2.7% to less than 2.4%:

The fact that this predicted probability is falling is the (now improved) basis for my prediction that divorce rates will continue to decline in the coming years: the people marrying now have fewer risk factors. (The data and code for all this is up, too).

---

Prediction aside: The short description of study preregistration is “specifying your plan in advance, before you gather data.” You do this with a time-stamped report so readers know you’re not rejiggering the results after you collect data to make it look like you were right all along. This doesn’t always make sense with secondary data because the data is already collected before we get there. However, in this case I was making predictions about future data not yet released. So the first version of this paper, posted last September and preserved with a time stamp on SocArXiv, is like a preregistration of the later versions, effectively predicting I would find a decline in subsequent years if I used the same models — which I did. People who use data that is released on a regular schedule, like ACS, CPS, or GSS, might consider doing this in the future.

---

Rejection addendum: Sociological Science rejected this — as they do, in about 30 days, with very brief reviews — and based on their misunderstandings I made some clarifications and explained the limitations before sending it to Socius. Since the paper was publicly available the whole time this didn’t slow down the progress of science, and then I improved it, so I’m happy about it.

Just in case you’re worried that this rejections means the paper might be wrong, I’m sharing their reviews here, as summarized by the editor. If you read the current version you’ll see how I clarified these points.

* While the analyses are generally sensible, both Consulting Editors point out the paper’s modest contribution to the literature relative to Kennedy and Ruggles (2014) and Hemez (2017). The paper cites both of these papers but does not make clear how the paper adds to our understanding derived from those papers. If the relatively modest extension in the time frame in this paper is sociologically consequential, the paper does not make the case clearly.

* There is more novelty in the paper’s estimates of the annual divorce probability for newly-married women (shown in Table 3 and Figure 3), based on estimating a divorce model for the most recent survey year, and then applying the coefficients from that model to prior years. This procedure was somewhat difficult for the readers to follow, but issues were raised, most notably the question of the sensitivity of the results to the adjustments made. As one CE noted, “Excluding those in the first year of marriage is problematic as newlyweds have a high rate of divorce. Also, why just married in the last 10 years? Consider whether married for the first time vs remarried matters. Also, investigate the merits of an age restriction given the aging of the population Kennedy and Ruggles point to.”

Proposal for ASA to adopt TOP Guidelines and Open Science Badges

My term on the American Sociological Association Committee on Publications begins in January, so I drafted the first proposal from my platform.

This is for ASA to adopt the Transparency and Openness Promotion Guidelines from the Center for Open Science, and to start using their Open Science Badges, which recognize authors who provide open data, open materials, or use preregistration for their studies.

I put the proposal up in Google Docs, where you can read and comment on it if you like: here.

Amend your ASA/Sage author agreement!

This is a followup to a previous post, and contains some duplication.

I have spoken well of the policy that permits authors to post preprint versions of their papers before submitting them to journals of the American Sociological Association. That means you can get your work out more broadly while it’s going through the review process. The rule says:

ASA authors may post working versions of their papers on their personal web sites and non-peer-reviewed repositories. Such postings are not considered by ASA as previous publication.

The policy goes on to ask that authors modify their posted papers to acknowledge publication if they are subsequently published. That’s all reasonable. This is why SocArXiv and other services offer authors the opportunity to link their papers to the DOI (record locator) for the published version, should it become available. This allows citation aggregators such as Google Scholar to link the records.

The problem

Unfortunately, the good part of this policy is undermined by the ASA / Sage author agreement that authors sign when their paper is accepted. It transfers the copyright of the paper to ASA, and sets conditions under which authors can distribute the paper in the future. The key passage here is this:

1. Subject to the conditions in this paragraph, without further permission each Contributor may …

• At any time, circulate or post on any repository or website, the version of the Contribution that Contributors submitted to the Journal (i.e. the version before peer-review) or an abstract of the Contribution.
• No sooner than 12 months after initial publication, post on any non-commercial repository or website the version of the Contribution that was accepted for publication.

This is not good. It means that if you post a paper publicly, e.g., on SocArXiv, and then submit it to ASA, you can’t update it to the revised version as your paper moves through the process. Only 12 months after ASA publishes it can you update the preprint version to match the version that the journal approved.

This policy, if followed, would produce multiple bad outcomes.

One scenario is that people post papers publicly, and submit them to ASA journals for review. Over the course of the next year or so, the paper is substantially revised and eventually published, but the preprint version is not updated until a full year after that, often two years after the initial submission. That means readers don’t get to see the improved version, and authors have to live with people reading and sharing their unimproved work. This discourages people from sharing their papers in the first place.

In the other scenario, people update their preprints as the paper goes through the revision process, so they and their readers get the benefit of access to the latest work. However, when the paper is accepted authors are expected to remove from public view that revised paper, and only share the pre-review version. If this were feasible, it would be terrible for science and the public interest, as well as the author’s career interests. Of course, this isn’t really feasible — you can’t unring the bell of internet distribution (SocArXiv and other preprint services do not allow removing papers, which would corrupt the scholarly record.) This would also discourage people from sharing their papers in the first place.

The individual solution

Fortunately, you are a volitional agent in a free market information ecosystem, and you don’t have to just sign whatever PDF some corporate conglomerate puts in front of you. My suggestion is that you amend the agreement before you sign it. After receiving your acceptance, when the managing editor sends you the author agreement for your signature, politely notify the editor that you will be happy to sign the agreement with a minor amendment. Then strike through the offending text and add the amendment. I recommend the following text:

• No sooner than 12 months after initial publication, post on any non-commercial repository or website the version of the Contribution that was accepted for publication.
• At any time, post to SocArXiv (a non-commercial, open-access repository) the version of the Contribution that was accepted for publication, with a DOI link and bibliographic reference to the published Contribution.

Then sign the agreement and return it. Here’s a visual depiction of the amendment:

Don’t panic! Yes, this publication may be the last thing standing between you and tenure or a better job. But the journal will not cancel your publication when you do this. The very worst thing that will happen is they will say “No!” Then you can roll over and accept the original agreement. (After the dust settles, I’d love it if you let me know this happened.) People amend these agreements all the time. Give it a try!

Here’s the relevant passage in “Alice’s Restaurant” (@ 14:32)

And the only reason I’m singing you this song now is cause you may know somebody in a similar situation, or you may be in a similar situation,

And if you’re in a situation like that there’s only one thing you can do and that’s walk into The shrink wherever you are, just walk in say “Shrink, You can get anything you want, at Alice’s restaurant.” And walk out.

You know, if one person, just one person does it they may think he’s really sick and they won’t take him. And if two people, two people do it, in harmony – they may think they’re both faggots and they won’t take either of them. And three people do it, three, can you imagine, three people walking in singing a bar of Alice’s Restaurant and walking out. They may think it’s an organization. And can you, can you imagine fifty people a day, I said fifty people a day walking in singin a bar of Alice’s Restaurant and walking out.

And friends they may think it’s a movement And that’s what it is, the Alice’s Restaurant Anti-Massacree Movement, and all you got to do to join is sing it the next time it comes around on the guitar. With feeling.

Fix the policy

So, what possible reason can there be for this policy? It is clearly intended to punish the public in order to buttress the revenue stream of Sage, which returns some of its profits to ASA, at the expense of our libraries, which pay for subscriptions to ASA journals.

I assume this policy is never enforced, as I’ve never heard of it, but I don’t know that for a fact. It’s also possible that whoever wrote the Publications policy I linked above didn’t realize that it contradicted the Sage author agreement, which basically no one reads. I also assume that such a policy does not in fact have any effect on Sage’s profits, or the profits that it kick backs to ASA. So it’s probably useless, but if it has any effects at all they’re bad, by discouraging people from distributing their work. ASA should change this author agreement.

I just got elected to the ASA Publications Committee, so I will add making this change to my platform, which I outlined here. I’m not optimistic about making policy changes at ASA in the current environment, but I am sure that the more people who join in the individual efforts, the greater our chances will be.

Let’s improve the ASA/Sage journal author agreement

I have spoken well of the policy that permits authors to post preprint versions of their papers before submitting them to journals of the American Sociological Association. That means you can get your work out more broadly while it’s going through the review process. The rule says:

ASA authors may post working versions of their papers on their personal web sites and non-peer-reviewed repositories. Such postings are not considered by ASA as previous publication.

The policy goes on to ask that authors modify their posted papers to acknowledge publication if they are subsequently published. That’s all reasonable. This is why SocArXiv and other services offer authors the opportunity to link their papers to the DOI (record locator) for the published version, should it become available. This allows citation aggregators such as Google Scholar link the records.

Unfortunately, the good part of this policy is undermined by the ASA / Sage author agreement that authors sign when their paper is accepted. It transfers the copyright of the paper to ASA, and sets conditions under which authors can distribute the paper in the future. The key passage here is this:

1. Subject to the conditions in this paragraph, without further permission each Contributor may …

• At any time, circulate or post on any repository or website, the version of the Contribution that Contributors submitted to the Journal (i.e. the version before peer-review) or an abstract of the Contribution.
• No sooner than 12 months after initial publication, post on any non-commercial repository or website the version of the Contribution that was accepted for publication.

This is not good. It means that if you post a paper publicly, e.g., on SocArXiv, and then submit it to ASA, you can’t update it to the revised version as your paper moves through the process. Only 12 months after ASA publishes it can you update the preprint version to match the version that the journal approved.

This policy, if followed, would produce multiple bad outcomes.

One scenario is that people post papers publicly, and submit them to ASA journals for review. Over the course of the next year or so, the paper is substantially revised and eventually published, but the preprint version is not updated until a full year after that, often two years after the initial submission. That means readers don’t get to see the improved version, and authors have to live with people reading and sharing their unimproved work. This discourages people from sharing their papers in the first place.

In the other scenario, people update their preprints as the paper goes through the revision process, so they and their readers get the benefit of access to the latest work. However, when the paper is accepted authors are expected to remove from public view that revised paper, and only share the pre-review version. If this were feasible, it would be terrible for science and the public interest, as well as the author’s career interests. Of course, this isn’t really feasible — you can’t unring the bell of internet distribution (SocArXiv and other preprint services do not allow removing papers, which would corrupt the scholarly record.) This would also discourage people from sharing their papers in the first place.

So, what possible reason can there be for this policy? It is clearly intended to punish the public in order to buttress the revenue stream of Sage, which returns some of its profits to ASA, at the expense of our libraries, which pay for subscriptions to ASA journals.

I assume this policy is never enforced, as I’ve never heard of it, but I don’t know that for a fact. It’s also possible that whoever wrote the Publications policy I linked above didn’t realize that it contradicted the Sage author agreement, which basically no one reads. I also assume that such a policy does not in fact have any effect on Sage’s profits, or the profits that it kick backs to ASA. So it’s probably useless, but if it has any effects at all they’re bad, by discouraging people from distributing their work. ASA should change this author agreement.

I will be on the ballot for the ASA Publications Committee this spring. If elected, I will add making this change to my platform, which I outlined here. If I’m not elected, I’ll try to do this anyway.

Review essay: Public engagement and the influence imperative

I have written a review essay at the invitation of Contemporary Sociology. Here’s a preprint version on SocArXiv: https://osf.io/preprints/socarxiv/v27xk/.

This is the abstract. Feedback welcome!

Public engagement and the influence imperative

Abstract: A review essay discussing three advice books for social scientists. Sociologists, in responding to the imperative to make their work more influential, must go beyond doing “public sociology” to embrace doing sociology “in public” (Healy 2017). Rather than using public engagement primarily for publicity – to make our research matter – we should use engagement to help us do research that matters in the first place. Next, I caution that the drive to be professionally rewarded for public intellectualism is fraught with conflicts that may be irreconcilable. To be a public intellectual today requires being both public in one’s intellectual life and intellectual in one’s public life, and for academics in the era of the “market university” (Berman 2011), trying to get paid for that leads to a neoliberal trap. Finally, I argue for a move beyond personal strategies toward the development of the open scholarship as an institutional response that ultimately may be responsible for sociology’s survival.

Here is the SocArXiv citation:

Cohen, Philip N., 2018. “Public Engagement and the Influence Imperative”. SocArXiv. April 7. doi:10.17605/OSF.IO/V27XK.

How I engaged my way to excellent research success and you can too

Kid photo CC from MB Photography; collage by pnc.

Too often sociologists think of social media, or online communications generally, primarily as a way of broadcasting their ideas and building their audience, instead of as a way of deepening their engagement with different people and perspectives. You see this when academics start a twitter account right when their book is coming out. Nothing wrong with that, but it’s very limited. A crucial part of being a public scholar, public intellectual, or a public sociologist, etc., is reading, listening, and learning through engagement, and digital communication can enhance the metabolism of that process. Especially important is the chance to learn from people you don’t normally interact with. For all the complaints about social media bubbles, some true, social media also offers huge efficiencies for meeting and learning from new people.

As I’m writing an essay about this, I thought of my work on divorce as an example. So here’s that thread, condensed.

A divorce story

In 2008 I was teaching an undergraduate Family Sociology course at the University of North Carolina, and included a section on divorce based on other people’s research. I was also developing a proposal for my own textbook, which at the time framed family structures and events, including divorce, as consequences and causes of inequality. I was reading research about divorce along with many other family issues that were outside of my formal training and experience (the closest I had come to a family demography or family sociology course was a seminar on Gender, Work & Family in grad school).

Then in 2009, I wrote a post on my pretty new blog criticizing something bad the Brad Wilcox had written about divorce. I was trying to be newsy and current, and he was claiming that the recession was lowering divorce rates because hard times pulled people together. We didn’t yet know what would happen in the recession. (In the comments, Louise Roth suggested it would take time for divorces “caused” by the recession to show up, which turned out to be true.)

I kept on that path for a while, criticizing Wilcox again for similar work in 2011. By then — prompted by the combination of my reading, the blog debates, and the news coverage around families and the recession — I was working on a paper on divorce using the American Community Survey. I presented it at a demography meeting in the summer of 2011, then revised and presented it at the Population Association of America the following spring. I blogged about this a couple more times as I worked on it, using data on state variation, and Google searches, each time getting feedback from readers.

A version of the paper was rejected by Demography in the summer of 2011 (which generated useful reviews). Although now discredited as not peer-review-publishable (which no one knew), my commentary on divorce and the recession was nevertheless featured in an NPR story by Shankar Vedantam. Further inspired, I sent a new version of the paper (with new data) to Demographic Research, which also rejected it. I presented on the work a couple of times in 2012, getting feedback each time. By August 2012, with the paper still not “published,” I was quoted describing my “divorce/recession lull-rebound hypothesis” in New York magazine.

The news media pieces were not simply my work appearing in the news, in a one-directional manner, or me commenting on other people’s research, but rather me bringing data and informed commentary to stories the reporters were already working on. Their work influenced my work. And all along that news coverage was generating on- and offline conversations, as I found and shared work by other people working on these topics (like the National Center for Marriage and Family Research, and the Pew Research Center). Looking back over my tweets about divorce, I see that I covered divorce and religion, disabilities, economics, and race/ethnic inequality, and also critiqued media coverage. (Everything also got discussed on Facebook, in a smaller semi-private circle.)

By 2014 I finally got the paper — now with even newer data — published in a paywalled peer-reviewed journal, in Population Research and Policy Review. This involved writing the dreaded phrase, “Thank you very much for the opportunity to revise this paper again.” (Submitted October 2012, revision submitted August 2013, second revision submitted January 2014, final revision April 2014.) The paper, eventually titled, “Recession and Divorce in the United States, 2008-2011,” did improve over this time as new data provided better leverage on the question, and the reviewers actually made some good suggestions.

Also in 2014 the descriptive analysis was published in my textbook. The results were reported here and there, and expanded into the general area of family-recession studies, including this piece in the Conversation. I also developed a method of projecting lifetime divorce odds (basically 50%), for which I shared the data and code, which was reported on here. Along the way I also did some work on job characteristics and divorce (data and code, working paper). When I posted technical notes, I got interesting responses from people like economist Marina Adshade, whom I’ve never met.

From my presentation, “The open scholarship media strategy for maximum success.”

So that’s an engagement story that includes teaching, the blogosphere and social media, news media, peer-reviewed publishing, conference presentations and colloquium talks. I did research, but also argued about politics and inequality, and taught and learned demography. It’s not a story of how I used social media, or the news media, to get the word out about my research, although that happened, too. The work product, not just the “publications,” were all public to varying degrees, and the discussions included all manner of students, sociologists, reporters, and interested blog or Twitter readers, most of whom I didn’t know or wouldn’t have met any other way.

So I can’t draw a line dividing the “engagement” and the “research,” because they weren’t separate processes.

Philip Cohen’s ASA Publications Committee platform

[with addendum at the end]

The American Sociological Association’s Committee on Nominations invited me to stand for election to the ASA’s Committee on Publications. I accepted the invitation reluctantly, because I am pessimistic about the opportunities to make progress on the most pressing issues before ASA, which I discuss below. I think it’s likely we’ll be in better shape for making big decisions in a few years, after the economics of publishing have gotten worse and (hopefully) alternatives such as SocArXiv and Sociological Science have gotten stronger.

But I appreciate the invitation, and I think there is something to be gained even if all I can do is raise the issues, and help disseminate important information (and my opinions) to the membership. And of course I can’t predict the future, and what the other present and future committee members will do, so who knows. Plus, it’s as a chance to spend time with the interesting and dedicated members of the committee and ASA staff.

Note on conflict of interest

I am the director of SocArXiv, a non-profit, open source, open access paper server for the social sciences, which I founded with a group of sociologists and library community leaders. I don’t draw a salary as director of SocArXiv, but if I can get grants for the project I might use them to buy off some of my time (meaning my salary doesn’t go up, but I work on SocArXiv instead of teaching), so you could say I stand to gain from promoting SocArXiv. And one of my ambitions in that role is to have ASA work with SocArXiv, for example in the dissemination of preprints and postprints from ASA journals, conference papers, and possibly innovative new projects like open peer review. I hope serving on this committee will help me advance SocArXiv’s agenda as well as ASA’s. So if that makes you uncomfortable you might not want to vote for me. In that case you might also consider the similar conflicts of interest held by the committee members who are, were, or hope to be editors of journals Sage publishes for ASA, or who otherwise benefit in their careers from the paywalled publishing status quo in academia. We’re all trying to do what we think is right and get paid and at the same time.

Basic perspective

In 2016 I wrote a short essay about ASA and publishing, and my basic perspective hasn’t much changed. Here is the gist from that essay.

The scholarly communication system is broken, and ASA lives off the money that brokenness creates. According to the 2016 budget report (I don’t see an update to this), 35% of total revenue comes from journal operations. That is $2.2 million that came from institutional subscriptions (mostly paid by the libraries of colleges and universities where ASA members work), under the contract with Sage. Increasingly, these subscriptions are part of big “bundles” of journals, in which individual libraries have little say over what they’re actually buying. Publishing the journals, in turn, costs 11% of total expenses, or about $717,000. So journal publishing produces money for other things the association does.

At the same time, ASA — like other paywall publishers — is in an increasingly defensive position, as open access alternatives spread and the cost of technologically and legally defending the paywall increases under pressure from Sci-Hub (which I wrote about here) and various other breaches. In a quasi-official statement from the ASA, publications director Karen Edwards wrote that Sci-Hub, “threatens the well-being of ASA and our sister associations as well as the peer assessment of scholarship in sociology and other academic disciplines.” Without the paywall, she implied, peer review itself cannot survive. I disagree.

More generally, ASA staff has raised alarms about the sustainability of the current model. From the Publications Committee minutes in spring 2016:  “The possibility exists that the journal world may not be as profitable in the future as it is now. The journal marketplace is shifting, and will continue to do so, so Council and EOB should keep an eye on this revenue source.”

We know that free journals could be published for a fraction of what ASA and Sage now spend and reap (one of the major expenses of any paywall publisher is developing and maintaining the technology to keep publications out of the hands of non-paying customers). That would mean giving up a substantial share of the association’s current income. And of course, it’s not a simple task. The basic goal is a future in which scholarly societies, academic libraries, and granting agencies together pay for scholarly publication, and cut out the for-profit publishers. That is instead of universities and federal agencies paying for research twice — once for the researchers, and once again for their published output — and all of it gets to be open access. This requires some institutional mechanism for collecting and distributing the funds used to produce open-access research output. Such solutions will require creativity, collaboration, and hard work. Designing a new system is relatively easy, but moving today’s institutional actors in that direction is not.

Platform

Here’s what I’d like to do, more and less implausible. These are overlapping and not mutually exclusive.

1. Adopt TOP guidelines for ASA journals

The Center for Open Science has published Transparency and Openness Promotion (TOP) Guidelines. The guidelines are incremental.  With almost 5,000 signatories — journals and associations — they include eight standards, each with three levels of stringency. Journals select which standards they will adopt and a level of implementation for each, ranging from disclosure requirements to verified replications. For example, we could say ASA journals will encourage data and code for analyses be posted publicly and require a statement about whether they are or not. We could encourage publication of replication studies. And so on. Even at low levels of implementation, they set an important tone and direction. This could be done for all of ASA, or individual journal editorial boards could implement them. In addition, by assigning openness badges to journals, authors, or papers, we can symbolically reward open scholarship practices.

2. Adopt preprints and data sharing

When people submit papers for ASA journals, they should be encouraged or required to post them on a public server (such as SocArXiv), along with data and code. When papers are published they can be linked to the preprints, ensuring readers are let to the final versions, while disseminating the research earlier, and free. As a model, consider the American Psychological Association, which has designated a preferred preprint server (PsyArXiv) and data archive (the Center for Open Science, which hosts SocArXiv and PsyArXiv).

3. Open conference papers

ASA requires complete papers be submitted for presentation at the annual meetings. We should make these papers available in a public archive, properly identified and preserved as part of the scholarly record. This is a key step in developing the working paper culture in sociology.

4. No new paywall journals

I will oppose the creation of any new paywalled journals, by ASA or its sections.

5. Flip ASA journals

The Holy Grail. Flipping journals refers to transitioning them from paywalled to open access. We can disseminate more sociology, better and faster, for less money. University libraries and university presses, and some foundations, can be mobilized to raise the money needed to launch sustainable models. For example, if a few hundred libraries would agree to give ASA what they spend on subscriptions to our journals, we could produce the same journals and open them to the public, cutting out Sage’s profits. My goal is for the ASA membership to task our association’s staff with figuring out how to pay for the journals without paywalls, which might require research and grants, and then bring proposals to the membership for approval.

6. Reduce revenue

ASA should make less money. Instead of making money from publications, we should just raise the money we need to publish journals, with the appropriate academic partners (universities and funding agencies). The many laudable things ASA does should be paid for voluntarily, by institutions or people who want to support them, rather than being paid for by publication profits from consumers (our employers) who have no choice but to subsidize ASA operations (and Sage profits).

7. Disseminate information

Too many sociologists don’t know how ASA works, how academic publishing works, and how we all fit into the overall system of research production and dissemination, the “scholarly communications ecosystem.” I would like to see us (starting with students) learning more about the politics and economics of this system, to have a better grasp of the big issues we face. On the publications committee I would hope to learn a lot, and help communicate this to our members and the public.

Conclusion

Here are some relevant experiences I’ve had. As editor, with Syed Ali (who is also running for publications committee!) of Contexts for three years (just completed), I was a non-voting member of the publications committee and got to see how it works. This included participating on a task force on the future of Contexts, which resulted in Sage agreeing to allow free downloads of the journal for the first 30 days of each issue, and in perpetuity after the first year. I’m the current chair of the Family section, and secretary treasurer of the Population section. I also successfully agitated for a new rule requiring ASA-award-winning dissertations to be publicly available. (For previous posts and activity related to ASA, visit the tag on this blog.) All that gives me some useful experience and knowledge.

I’d love for this election to help spur a more widespread discussion and debate over these questions. Without a majority on the publications committee or the ASA Council (and even with such a majority) this is a pretty daunting set of goals. I couldn’t pretend to promise results on any of these, but I pledge to at least promote these ideas, in the service of more research transparency and openness at lower cost.

The ASA election ballots will go out in April. To see what the Committee on Publications does, check out the committee records here.

---

Addendum: I got elected. Also, I added a plank on revising the ASA/Sage journal author agreement, described here. Follow the ASA tag for updates.