Pandora's Box

Nature on a new, psychologist-driven movement that proposes a strategy to minimise errors in publications.The idea was initiated by Elson, a psychologist at the University of Bern, following the incidental finding of a number of errors in a highly influential paper, casting some doubt on its findings.The paper, published in 2010 by respected researchers from Harvard University in Cambridge, Massachusetts, on the subject of economics, 'promoted austerity measures to reduce national debt', and had a major influence on financial policies in Europe.The errors were only identified 3 years later, when a PhD student attempted to replicate the publication's findings.The authors of the paper cooperated in providing their data and admitted to the errors, although they maintained that their conclusions were sound.This triggered the idea of Elson and colleagues to propose what they call the ERROR project, which will pay reviewers to check highly cited psychology papers, searching for errors in code, statistics and references cited.Reviewers will be paid for each paper reviewed and will in addition get a bonus for any errors they find, with the bonus payment being higher for bigger errors.This approach was modelled on the 'bug bounty' programmes that Microsoft, Google and other technology companies offer to hackers they employ to find and report possible weaknesses in their programs.Payments are also offered to the authors of publications for answering reviewers' questions and for making their data available; extra payments are offered to them if no errors are found, or if any errors are only minor.So far, only a few authors have agreed to have their studies reviewed.Finding reviewers is not straightforward, either.Despite the offer of reasonably attractive remuneration, many potential reviewers are likely to be in the early-career stage and may be wary of getting involved, owing to concerns about repercussions from senior academics in their field.However, the ERROR project has received funding and is gathering momentum, and it is hoped that it will be expanded to other disciplines in addition to psychology in the near future.Courses are being developed to teach error detection at a Master's level at psychology department of the University of Bern, and also at the Institute of Psychiatry, Psychology and Neuroscience at King's College, London.Elson and his colleagues hope that their project will flourish if funding bodies can be persuaded to pay for error reviews of publications they fund, arguing that this will be in everybody's interest, including that of the funders.Let's wait and see.In the meantime: researchers, do double-check your work; Big Brother may be watching.