Should reviewers be expected to review supporting datasets and code?

by John Helliwell, Emeritus Professor of Chemistry University of Manchester and DSc Physics University of York (@HelliwellJohn) Introduction For the meeting entitled “Transparency, Reward, and Innovation in Peer Review in the Life Sciences” to be held on Feb. 7-9, 2018 at the Howard Hughes Medical Institute in Chevy Chase, Maryland ( I have been asked by […]

Should scientists receive credit for peer review?

by Stephen Curry, Professor of Structural Biology, Imperial College (@Stephen_Curry) As the song goes – and I have in mind the Beatles’ 1963 cover version of Money (that’s all I want) – “the best things in life are free.” But is peer review one of them? The freely given service that many scientists provide as validation […]

New tool to identify fakes in the peer review process

Combating fake peer review

Fake reviews continue to be a serious concern in medical publishing, putting data integrity and trust in the scientific community at risk. As recently reported by Retraction Watch, a new tool designed by Clarivate Analytics will be available in December 2017 to help journals identify fake reviews and prevent publication of articles that rely on them.

Fake peer review has been responsible for the retraction of over 500 articles to date and the issue has caused some journals to review their policy of requesting reviewer nominations from authors. However, many journals still retain this policy as recruiting peer reviewers is becoming increasingly difficult and time-consuming. While some fake reviewers may be easy to identify, in some cases it is more difficult. The new fraud prevention tool can be used at multiple points during the submission and review process, and looks at 30 different factors that can help to identify fake profiles, impersonators and unusual activity.

Upon identification of a possible fake review, the journal is alerted and the editor or publisher is then able to decide whether to investigate further and whether to accept the article for publication. It is anticipated that early identification of possible fake reviews during the submission and peer review process will reduce the number of retractions and help to protect the reputation of medical publishing.


Summary by Philippa Flemming, PhD from Aspire Scientific

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

The post New tool to identify fakes in the peer review process appeared first on The Publication Plan for everyone interested in medical writing, the development of medical publications, and publication planning.

Gender Bias in Peer Review: An Interview with Brooks Hanson and Jory Lerback

Earlier this year, an American Geophysical Union analysis of peer review in its journals revealed evidence of gender bias, with women being less likely to be invited to review than men despite being more likely to be the first author of an accepted paper. In this interview, Brooks Hanson (Senior Vice President, Publications) and former Data Analyst, Jory Lerback describe the original study and the AGU’s efforts to address this bias.

The post Gender Bias in Peer Review: An Interview with Brooks Hanson and Jory Lerback appeared first on The Scholarly Kitchen.

The vision and reality of soundness-only peer review in open-access mega-journals

Valerie Spezi, Simon Wakeling, Stephen Pinfield, Jenny Fry, Claire Creaser, Peter Willett, (2017) ““Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals”, Journal of Documentation,

“Findings suggest that in reality criteria beyond technical or scientific soundness can and do influence editorial decisions. Deviations from the original OAMJ model are both publisher supported (in the form of requirements for an article to be “worthy” of publication) and practice driven (in the form of some reviewers and editors applying traditional peer review criteria to OAMJ submissions). Also publishers believe post-publication evaluation of novelty, significance and relevance remains problematic.”

The NIH Center of Scientific Review in deal with Elsevier to support the peer review process of grant applications

The Center for Scientific Review at the National Institutes of Health (NIH) has signed an agreement with , the information analytics company specializing in science and health, to support improving the peer review process of NIH grant applications by using Expert Lookup, Elsevier’s tool that identifies scientific experts. Expert Lookup use Elsevier’s powerful semantic Fingerprinting algorithms, the Scopus abstract and citation database of over 69 million records, and 10 disciplinary and multidisciplinary thesauri to ensure the recommended reviewers are relevant and thought leaders in their fields.

DOI for Peer Reviews

Peer ReviewsMost journals follow peer review process to assess and select manuscripts for publication. Peer reviews can provide you with information on the strengths and weaknesses of your paper. The reviewers are either chosen by the publishers or suggested by the author. They should be unbiased and expert in the subject area they are reviewing. Reviewers…
read more

BioRxiv and Authorea Partner to Streamline Preprint Submission


Science communication is very important. As researchers increasingly forge global collaborations in biological research, the scientific community will need more collaborative tools. To meet that need, Authorea developed a collaborative document editor service. It helps collaborators edit shared documents. Furthermore, it facilitates the process of archiving preprints or publication-ready manuscripts (not peer reviewed) and submitting  … Read more

Opening up the black box of peer review

I recently participated in a workshop hosted by the University of Kent Business School – the subject was whether metrics or peer review are the best tools to support research assessment. Thankfully, we didn’t get embroiled in the sport of ‘metric bashing’, but instead agreed that one size does not fit all and that whatever research assessment we do, while taking account of context, needs to be proportionate.

There are many reasons why we want to assess research – to identify success in relation to goals, to allocate finite resources, to build capacity, to reward and incentivise researchers, as a starting point for further research – but these are all different questions, and the information you need to answer them is not always going to be the same.


What do we know about peer review?

In recent years, while researchers and evaluators have started to swim with the metric tide and explore how new metrics have value in different contexts, ‘peer review’, i.e., the qualitative way that research and researchers are assessed, is (a) still described as if it is one thing, and (b) remains a largely unknown ‘quantity’.  I am not sure if this is ironic (or intentional?) or not, but there remains dearth of information on how peer review works (or doesn’t).

Essentially, getting an expert’s view on a piece of research – be that in a grant application, a piece submitted for publication to a journal, or work already published –  can be helpful to science.  However, there is now significant body of evidence that suggests that how the scientific community organises, requests and manages its expert input may not be as optimum as many consumers of its output assume.  A 2011 UK’s House of Commons report on the state of peer review concluded that while it “is crucial to the reputation and reliability of scientific research” many scientists believe the system stifles innovation and “there is little solid evidence on its efficacy.” Indeed, during the production of the HEFCE commissioned 2015 Metric Tide report, we found ourselves judging the value of quantitative metrics based on the extent to which they replicated the patterns of choices made by ‘peers’. This was done without any solid evidence to support the veracity and accuracy of the peer review decisions themselves; following a long-established tradition for reviews on the mechanics of peer review to cite reservations about the process, before eventually concluding that ‘it’ remains the gold standard. As one speaker at the University of Kent workshop surmised, “people talking about the gold standard [of peer review] maybe don’t want to open up their black boxes.” However, things might be changing.


Bringing in the experts at right time

In grant assessment, there is increasing evidence that how and when we use experts in the grant selection and funding process may be inefficient and lack precision, see for example: Nature; NIH; Science and RAND. Several funding agencies are now experimenting with approaches that use expert input at different stages in the grant funding cycle and to different degrees – the aim being to encourage innovation, while bringing efficiencies to the process, including by reducing the opportunity for bias and practically, reducing the burden on peers, examples of this are Wellcome Trust Investigator Award grants; HRC Explorer grants; Volkswagenstiftung Experiment grants; and Velux Foundation Villum experiment.


Opening peer review in publishing

In the publishing world, there is considerable momentum towards the adoption of models in where research is shared much earlier and more openly.  Preprint repositories such as bioRxiv and post-publication peer review platforms, such as F1000Research, Wellcome Open Research, and soon to be launched Gates Open Research and UCL Child Health Open Research, enable open commenting and open peer review respectively as the default. Such models not only provide transparency and accelerate access to research findings and data to all users but they fundamentally change the role of experts – to one focused on providing constructive feedback and helping research to advance – even if they don’t like or agree with what they see! Furthermore, opening up access to what experts have said about others’ work is an important step towards reducing the selection bias of what is published and allowing readers more autonomy to reach their own conclusions about what they see.


Creating a picture of the workload

Perhaps the most obvious ways in which ‘peer review’ is currently broken is under the sheer weight of what publishers, funding agencies and institutions are asking experts to do. Visibility around a contribution presents the opportunity for experts to receive recognition for the effort and contributions they have made to the research enterprise in its broadest sense – as is already underway with ORCID – thus providing an incentive to get involved. And for funding agencies, publishers and institutions, more information about who is providing the expert input, and therefore where the burden lies, can help them to consider who, when and how they approach experts, maximising the chance of a useful response, and bringing efficiencies and effectiveness to the process.

The recent acquisition of Publons by Clarivate is a clear indication of the current demand and likely potential for more information about expert input to research – and should go some way to addressing the dearth of intelligence on how ‘peer review’ is working – and actually works.

Intelligent Crowd Peer Review Successfully Tested

Synlett, an international chemistry journal published by Thieme, has successfully tried a new form of peer review aimed to make manuscript assessment faster and fairer. The journal’s editor in chief, Benjamin List of the Max Planck Institute for Coal Research in Germany, and his PhD student Denis Höfler have called the new method “intelligent crowd”  … Read more

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑