by Stephen Curry, Professor of Structural Biology, Imperial College (@Stephen_Curry) As the song goes – and I have in mind the Beatles’ 1963 cover version of Money (that’s all I want) – “the best things in life are free.” But is peer review one of them? The freely given service that many scientists provide as validation […]
Do you want to host @PREreview_ journal club? support from @MozillaScience @SloanFoundation will get you snacks to rope in the grad students and postdocs https://twitter.com/jessicapolka/status/929994930587668480 …
Fake reviews continue to be a serious concern in medical publishing, putting data integrity and trust in the scientific community at risk. As recently reported by Retraction Watch, a new tool designed by Clarivate Analytics will be available in December 2017 to help journals identify fake reviews and prevent publication of articles that rely on them.
Fake peer review has been responsible for the retraction of over 500 articles to date and the issue has caused some journals to review their policy of requesting reviewer nominations from authors. However, many journals still retain this policy as recruiting peer reviewers is becoming increasingly difficult and time-consuming. While some fake reviewers may be easy to identify, in some cases it is more difficult. The new fraud prevention tool can be used at multiple points during the submission and review process, and looks at 30 different factors that can help to identify fake profiles, impersonators and unusual activity.
Upon identification of a possible fake review, the journal is alerted and the editor or publisher is then able to decide whether to investigate further and whether to accept the article for publication. It is anticipated that early identification of possible fake reviews during the submission and peer review process will reduce the number of retractions and help to protect the reputation of medical publishing.
Subscribe to Blog via Email
Enter your email address to subscribe to this blog and receive notifications of new posts by email.
The post New tool to identify fakes in the peer review process appeared first on The Publication Plan for everyone interested in medical writing, the development of medical publications, and publication planning.
Earlier this year, an American Geophysical Union analysis of peer review in its journals revealed evidence of gender bias, with women being less likely to be invited to review than men despite being more likely to be the first author of an accepted paper. In this interview, Brooks Hanson (Senior Vice President, Publications) and former Data Analyst, Jory Lerback describe the original study and the AGU’s efforts to address this bias.
The post Gender Bias in Peer Review: An Interview with Brooks Hanson and Jory Lerback appeared first on The Scholarly Kitchen.
Valerie Spezi, Simon Wakeling, Stephen Pinfield, Jenny Fry, Claire Creaser, Peter Willett, (2017) ““Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals”, Journal of Documentation, https://doi.org/10.1108/JD-06-2017-0092
“Findings suggest that in reality criteria beyond technical or scientific soundness can and do influence editorial decisions. Deviations from the original OAMJ model are both publisher supported (in the form of requirements for an article to be “worthy” of publication) and practice driven (in the form of some reviewers and editors applying traditional peer review criteria to OAMJ submissions). Also publishers believe post-publication evaluation of novelty, significance and relevance remains problematic.”
About that peer-review crisis … There isn’t one
The Center for Scientific Review at the National Institutes of Health (NIH) has signed an agreement with , the information analytics company specializing in science and health, to support improving the peer review process of NIH grant applications by using Expert Lookup, Elsevier’s tool that identifies scientific experts. Expert Lookup use Elsevier’s powerful semantic Fingerprinting algorithms, the Scopus abstract and citation database of over 69 million records, and 10 disciplinary and multidisciplinary thesauri to ensure the recommended reviewers are relevant and thought leaders in their fields.
Most journals follow peer review process to assess and select manuscripts for publication. Peer reviews can provide you with information on the strengths and weaknesses of your paper. The reviewers are either chosen by the publishers or suggested by the author. They should be unbiased and expert in the subject area they are reviewing. Reviewers…
Science communication is very important. As researchers increasingly forge global collaborations in biological research, the scientific community will need more collaborative tools. To meet that need, Authorea developed a collaborative document editor service. It helps collaborators edit shared documents. Furthermore, it facilitates the process of archiving preprints or publication-ready manuscripts (not peer reviewed) and submitting … Read more
are there any journals that have a more interactive peer review a la github pull-requests for code?
I recently participated in a workshop hosted by the University of Kent Business School – the subject was whether metrics or peer review are the best tools to support research assessment. Thankfully, we didn’t get embroiled in the sport of ‘metric bashing’, but instead agreed that one size does not fit all and that whatever research assessment we do, while taking account of context, needs to be proportionate.
There are many reasons why we want to assess research – to identify success in relation to goals, to allocate finite resources, to build capacity, to reward and incentivise researchers, as a starting point for further research – but these are all different questions, and the information you need to answer them is not always going to be the same.
What do we know about peer review?
In recent years, while researchers and evaluators have started to swim with the metric tide and explore how new metrics have value in different contexts, ‘peer review’, i.e., the qualitative way that research and researchers are assessed, is (a) still described as if it is one thing, and (b) remains a largely unknown ‘quantity’. I am not sure if this is ironic (or intentional?) or not, but there remains dearth of information on how peer review works (or doesn’t).
Essentially, getting an expert’s view on a piece of research – be that in a grant application, a piece submitted for publication to a journal, or work already published – can be helpful to science. However, there is now significant body of evidence that suggests that how the scientific community organises, requests and manages its expert input may not be as optimum as many consumers of its output assume. A 2011 UK’s House of Commons report on the state of peer review concluded that while it “is crucial to the reputation and reliability of scientific research” many scientists believe the system stifles innovation and “there is little solid evidence on its efficacy.” Indeed, during the production of the HEFCE commissioned 2015 Metric Tide report, we found ourselves judging the value of quantitative metrics based on the extent to which they replicated the patterns of choices made by ‘peers’. This was done without any solid evidence to support the veracity and accuracy of the peer review decisions themselves; following a long-established tradition for reviews on the mechanics of peer review to cite reservations about the process, before eventually concluding that ‘it’ remains the gold standard. As one speaker at the University of Kent workshop surmised, “people talking about the gold standard [of peer review] maybe don’t want to open up their black boxes.” However, things might be changing.
Bringing in the experts at right time
In grant assessment, there is increasing evidence that how and when we use experts in the grant selection and funding process may be inefficient and lack precision, see for example: Nature; NIH; Science and RAND. Several funding agencies are now experimenting with approaches that use expert input at different stages in the grant funding cycle and to different degrees – the aim being to encourage innovation, while bringing efficiencies to the process, including by reducing the opportunity for bias and practically, reducing the burden on peers, examples of this are Wellcome Trust Investigator Award grants; HRC Explorer grants; Volkswagenstiftung Experiment grants; and Velux Foundation Villum experiment.
Opening peer review in publishing
In the publishing world, there is considerable momentum towards the adoption of models in where research is shared much earlier and more openly. Preprint repositories such as bioRxiv and post-publication peer review platforms, such as F1000Research, Wellcome Open Research, and soon to be launched Gates Open Research and UCL Child Health Open Research, enable open commenting and open peer review respectively as the default. Such models not only provide transparency and accelerate access to research findings and data to all users but they fundamentally change the role of experts – to one focused on providing constructive feedback and helping research to advance – even if they don’t like or agree with what they see! Furthermore, opening up access to what experts have said about others’ work is an important step towards reducing the selection bias of what is published and allowing readers more autonomy to reach their own conclusions about what they see.
Creating a picture of the workload
Perhaps the most obvious ways in which ‘peer review’ is currently broken is under the sheer weight of what publishers, funding agencies and institutions are asking experts to do. Visibility around a contribution presents the opportunity for experts to receive recognition for the effort and contributions they have made to the research enterprise in its broadest sense – as is already underway with ORCID – thus providing an incentive to get involved. And for funding agencies, publishers and institutions, more information about who is providing the expert input, and therefore where the burden lies, can help them to consider who, when and how they approach experts, maximising the chance of a useful response, and bringing efficiencies and effectiveness to the process.
The recent acquisition of Publons by Clarivate is a clear indication of the current demand and likely potential for more information about expert input to research – and should go some way to addressing the dearth of intelligence on how ‘peer review’ is working – and actually works.
Synlett, an international chemistry journal published by Thieme, has successfully tried a new form of peer review aimed to make manuscript assessment faster and fairer. The journal’s editor in chief, Benjamin List of the Max Planck Institute for Coal Research in Germany, and his PhD student Denis Höfler have called the new method “intelligent crowd” … Read more
Anna Ehler Society Marketing We sat down with Chris Graf, Wiley’s new Director of Research Integrity and Publishing Ethics, to learn just how much published research needs a second look.
At Wellcome Open Research, we operate a model of post-publication open peer review . We believe this will encourage constructive feedback from experts focused on helping the authors improve their work.
There are many other models of open peer review out there that work in different ways. In most models, the reviewer is named and it is seen as a way of crediting them for their work. We go a step further by not only naming them, but we also include their full reports as part of published article. Each peer review report also has its own DOI that can be added to ORCiD profiles, which also ensures peer reviewers get credit for their work.
Open peer review as a two-way conversation
Open peer review could also be described as a way of giving reviewers a voice as their critique and insight often helps shape what is the final article. Although the peer reports and reviewers’ names are readily available, we don’t often hear from reviewers, so were interested to explore what the conversation between author and reviewer looks like.
CRISPR for the community
Jürg Bähler, María Rodríguez-López and their team decided to try to refine the CRISPR/Cas9 gene-editing technique for yeast based on questions that were raised on a community email distribution list. They saw their work as being a valuable resource in helping others in their research and were keen to get it out there quickly. This was one of the main reasons that they decided to publish their Method Article on Wellcome Open Research.
Peer reviewed by the community
Once published, Jürg, María and colleagues then needed to decide who had the most appropriate expertise to review their article. This can be particularly important in niche fields as authors are best placed to know who should review their work. In this instance, Jürg and María thought it would be good to invite Damien Helmand to review as they knew his work, and also knew he was interested in this specific technique from questions he raised on the email distribution list. Damien agreed and invited two of his PhD students, Carlo Yague-Sanz and Olivier Finet, to review with him as a way of gaining experience in peer review. Carlo and Olivier are also named alongside Damien as reviewers of the article. Credit where credit is due.
Exploring the living article’s publishing process
After the article passed peer review, we went to meet with Jürg, María, Damien and Carlo to hear their views on the publication process, open peer review and how versioning has helped make a living article.