Report on contextually sensitive framework for defining QRPs

Questionable Research Practices (henceforth, QRPs) are a foundational concept in research integrity, but also fraught with inherent ambiguities. They are by definition a grey area of practices that may or may not be problematic depending on the context, and for this reason are hard to define, quantify and study. One of the core tasks of WP3 was to address these limitations, by offering a theoretical framework to better define and contextualise QRPs, and empirically assess the possible effects of the identified contextual features.This deliverable reports the results of three studies that met these objectives. These studies are currently under review or in preparation, and this reports presents an excerpt and synthesis of them.The first study collected the definitions of QRPs from surveys about research misconduct, and reclassified these definitions according to a principled scheme. Results yield 27 distinctive types of QRP and show that the QRPs most commonly self-reported in surveys consists in either the selective reporting of data, covariates, analyses, results, and publications, or in HARKing, a QRP in which hypotheses obtained after viewing the results are presented as being tested and confirmed by those results. These QRP are also the most context-sensitive, and thus constituted the basis for the second and third study.The second study produced a framework to contextualise QRPs. This was based on Information Compression Theory (ICT), a theoretical framework that proposes to understand metascientific problems through the lenses of information theory. Focusing on QRPs that (1) introduce bias in a study and (2) are not fully reported in the study, the ICT analysis suggests that a QRP study is most likely to affect the literature, positively or negatively, in fields that have a high information yield (e.g. fields characterised by simpler phenomena and more mature methods), high replicability (e.g. fields characterised by greater transparency and greater codification of methods) and higher biases (e.g. fields in which studies already systematically over- or underestimate the result, due to QRPs). These three dimensions, and the possible factors that affect them, constituted the basis for the third study included in this report.The third study aimed at assessing if and how researchers take contextual factors into account when evaluating the severity of a QRP. Focusing on HARKing as a common and context-sensitive QRP, we conducted a randomised online experiment in which researchers were shown a vignette reporting a fictitious study and were asked to express their preference between two summaries, one of which embodied HARKing. Five dimension of context were manipulated. Results show that researchers on average rated the nonquestionable summary higher, and that none of the five contextual factors significantly modulated their judgement. However, researchers who believed that these contextual factors were important for evaluating a QRP judged the HARKing vignette less negatively