Reply to Neveu and Neveu: Inference in an information-restricted environment

We thank Neveu and Neveu for their commentary ( 1 ) on our study ( 2 ).We agree that it would be worthwhile to examine the role of reviewers in the proliferation of fraud in scientific publishing.Indeed, paper mills have been previously implicated in compromising review processes and fabricating reviewer identities ( 3 , 4 ).However, we dispute other assertions made in their commentary.Neveu and Neveu contend that reviewers are betterequipped and have more opportunities to detect fraud than editors.We agree with Neveu and Neveu that analyzing patterns in journals where both editor and reviewers are named would provide better insights.However, as noted by Neveu and Neveu themselves, "[editors] of scientific journals make central decisions in the publication process" ( 5 ).Although reviewers often wield more expertise than editors, editors always wield more authority than reviewers.Moreover, submissions are typically passed through layers of editorial oversight ( 6) and editors see many more submissions than reviewers, situating editors well to detect suspicious submission behavior, such as an influx of unusually similar manuscripts.We maintain that editors and reviewers both play active and complementary roles in maintaining the integrity of the scholarly record ( 7 ).Principally, Neveu and Neveu protest that we have insufficient evidence to imply that some editors play a causal role in large-scale publication fraud.They are correct that our analysis uses markers that are variously suggestive of misconduct by any party, from weakest (implausibly short turnaround times) to strongest (retractions).Even our strongest indicator is not a surefire signal, as retractions can and do occur for a number of reasons unrelated to misconduct ( 8 ).However, attempting to infer underlying patterns from relatively weak indicators and rare events is well-trodden statistical territory.The existence of alternative explanations for these flags does not preclude their use for associative analysis.Our analysis, despite its basis in limited data, recovers numerous instances where a majority of a flagged editor's handled articles have since been retracted.For each of the Hindawi journals that we analyze, the single most frequent named editor of retracted articles handled between 9.0% (40 articles, Journal of Healthcare Engineering) and 61.6% (154 articles, Journal of Environmental and Public Health) of all retracted articles in the journal [note that Hindawi published a majority of articles in special issues with guest editors ( 9)].Although we are ignorant to the exact circumstances surrounding these cases, it is difficult to imagine that these editors in particular do not bear some responsibility for these breaches of publication integrity.Moreover, evidence abounds that some editors have a direct role in publication fraud ( 10 , 11 ).Ascertaining the causal role of editors and reviewers in publication fraud on any scale requires knowledge of every step of the process: editor identities, author identities, reviewer identities, relationships and prior interactions among them, review content, manuscript versions, turnaround times, manuscript provenance, etc.Even where this information is recorded, it is often kept closely guarded by publishers.We demonstrate that even in the absence of most of this information, actors that are likely to have played central roles can still be identified.