Peer review as practised at Wellcome Open Research

In November 2016 Wellcome became the first research funder to launch a publishing platform for the exclusive use of its grantholders. Wellcome Open Research, run on behalf of Wellcome by F1000, uses a model of immediate publication followed by invited, post-publication peer review. All reviews are citable and posted to the platform along with the identities of the reviewers. Continue reading “Peer review as practised at Wellcome Open Research”

Top 5 blog posts for 2017

2017 has been a good year for Wellcome Open Research! As we step into the holiday season we thought it a good time to have look back at what we has been happening on our blog this year. We share our top five blog posts for 2017, covering various topics from benefits of our open peer review model for creating conversations opening up channels of collaboration to the various milestones that we have reached. We would like to take this opportunity to thank all our authors and reviewers who have contributed to Wellcome Open Research’s success this year.

Continue reading “Top 5 blog posts for 2017”

Gateways – dedicated spaces for publishing Wellcome-funded research

Today, we’re happy to announce the launch of Gateways on Wellcome Open Research. This new functionality will enable Wellcome centres, institutions and funded projects to create their own dedicated space on our platform and collate any research published relating to it in one searchable domain.

With Wellcome Open Research being an author-led platform, the Gateways are a natural extension that gives all Wellcome-funded researchers and communities the opportunity to create their own branded “publishing home” for their research outputs designed to suit their needs.

The first Gateway to be launched on the platform is for one of the Africa programmes, the KEMRI-Wellcome Trust Research Programme (KWTRP), Kenya. Since the launch of Wellcome Open Research, KWTRP have embraced the platform to publish different research types of research (original research, methodologies, systematic reviews) recognising the advantages of publishing in this way. With a growing number of published articles, they felt that bringing all these together in one place would help to showcase their research even further.

WOR Advisory board member and Executive director of KWTRP, Philip Bejon said:

“We’ve enjoyed the ability to rapidly share research findings, analysis and data through the Wellcome Open Research platform.  A specific gateway to connect and recognize our work provides a sense of ownership and enhances the experience for all.”

WOR author and KEMRI Gateway Advisor, Ifedayo Adetifa said:

“I recently tried out Wellcome Open Research as an author and enjoyed the novelty of playing a more active role in selection of reviewers. More importantly, the whole process from submission to publication was easy, transparent and quick. So, I look forward to more publications and contributing as a gateway advisor to promote KEMRI’s gateway.”

As well as providing a central place to host research relating to a specific centre/project, the Gateways also enables their owners to showcase the identity of their centre/project. Readers are made aware from the article level that the work is related to other related research on the platform. By exploring this, the reader will find more research from a specific community and can then easily search through relevant content using the dedicated filters specific to the gateway. This allows for further engagement, whilst also giving the reader easy accessibility to more related research.

The Gateways also enable their owners to put across more information about their projects and links to their latest news helping provide more context to what they are trying to achieve to the reader. All in all, the Gateways have been created to help support and build new research communities in an open and collaborative way.

We will be launching more Gateways on the platform in the near future. If you are a Wellcome-funded researcher who would like to discuss the idea of using a Gateway for your own research then please do get in contact with us – we would love to hear from you!

Bringing the peer review conversation to life

At Wellcome Open Research, we operate a model of post-publication open peer review . We believe this will encourage constructive feedback from experts focused on helping the authors improve their work.

There are many other models of open peer review out there that work in different ways. In most models, the reviewer is named and it is seen as a way of crediting them for their work. We go a step further by not only naming them, but we also include their full reports as part of published article. Each peer review report also has its own DOI that can be added to ORCiD profiles, which also ensures peer reviewers get credit for their work.

Open peer review as a two-way conversation

Open peer review could also be described as a way of giving reviewers a voice as their critique and insight often helps shape what is the final article. Although the peer reports and reviewers’ names are readily available, we don’t often hear from reviewers, so were interested to explore what the conversation between author and reviewer looks like.

CRISPR for the community

Jürg Bähler, María Rodríguez-López and their team decided to try to refine the CRISPR/Cas9 gene-editing technique for yeast based on questions that were raised on a community email distribution list. They saw their work as being a valuable resource in helping others in their research and were keen to get it out there quickly. This was one of the main reasons that they decided to publish their Method Article on Wellcome Open Research.

Peer reviewed by the community

Once published, Jürg, María and colleagues then needed to decide who had the most appropriate expertise to review their article. This can be particularly important in niche fields as authors are best placed to know who should review their work. In this instance, Jürg and María thought it would be good to invite Damien Helmand to review as they knew his work, and also knew he was interested in this specific technique from questions he raised on the email distribution list.  Damien agreed and invited two of his PhD students, Carlo Yague-Sanz and Olivier Finet, to review with him as a way of gaining experience in peer review. Carlo and Olivier are also named alongside Damien as reviewers of the article. Credit where credit is due.

 

 

Exploring the living article’s publishing process

After the article passed peer review, we went to meet with Jürg, María, Damien and Carlo to hear their views on the publication process, open peer review and how versioning has helped make a living article.

Wellcome Open Research author survey results

Author survey shows that publication speed and the ability to share a variety of research outputs are the primary reasons why authors publish on the Wellcome Open Research publishing platform. Michael Markie, Publisher at F1000 and Robert Kiley, Head of Open Research, Wellcome discuss the survey results and what actions will taken based on them.

Wellcome Open Research has now been publishing for just over 6 months and to-date has published 63 articles.

The platform was specifically developed for Wellcome-funded researchers to explore the benefits of immediate publication of articles and other research outputs with no editorial bias, followed by an author-led, transparent, peer review process. As this approach differs somewhat from the traditional publishing model we were keen to reach out to those authors who had used this platform to understand their motivations for publishing here, what they liked and which aspects could be improved.

Consequently, in April 2017 we invited the first 50 submitting authors on Wellcome Open Research to participate in a short survey. We received an impressive 84% response rate and access to the survey results can be found here. Below is a summary of the major findings and what we have learnt from our authors.

What have we learnt?

The author experience with the editorial office and their experience of the overall publication process was very positive. A clear majority said that the submission process was efficient and they were very satisfied with the level of support, speed and responsiveness of the editorial team. Due to this positive experience, most the authors said they would recommend publishing on the platform to a colleague, and said they would also be inclined to publish again.

Speed and variety of article types

We asked our authors explicitly what was their primary reason for submitting to the platform. The two stand out reasons were, one, the speed of publication and two, the fact that the platform publishes all research outputs – not just traditional research articles.

With regard to speed of publication, the median time from submission to publication is 19 days, whilst the median time from publication to when an article has passed peer review and is indexed in PubMed, PMC and Europe PMC is currently 31 days. The speed in which research findings are not only accessible but also discoverable through these major online platforms is a key factor for our authors and one which is driving new submissions.

We are also very pleased that the platform is carving out a niche of publishing a variety of research types that the authors believe should be made publicly available. Currently half the articles we have published are not traditional research articles, but rather a rich mix of research outputs such as software tools, methods, protocols and data notes. Our authors have made it clear there is much research they would like to share with the community but can’t necessarily do so in a traditional journal; Wellcome Open Research is providing a useful venue to facilitate this.

Perception of peer review

The open peer review process, which is author led – suggesting reviewers, and engaging with them in an open, transparent way –  is probably the biggest difference that our authors experience whilst publishing on the platform and, not surprisingly, this aspect of the process is where we have had suggestions of how we can improve.

We received valuable feedback that our competing interest’s criteria about co-authors may be too stringent, as in some cases previous co-authors and collaborators are the most appropriate people to review a certain article and so shouldn’t be automatically excluded. This is a valid point and something we will look more closely at.  Ultimately, we need to balance the need of ensuring we receive an unbiased review against an ambition to allow the author to select the right reviewer for an article, which in some cases might be someone they have previously worked with in the past.

The survey also highlighted an interesting dilemma around attitudes to open peer review.  So, whereas only 14% of respondents disagreed with the statement that “the ability to select the referees improves the publication process”, a third of respondents felt that author-driven selections would result in reviews being less critical.  Whether this is the case is impossible to determine, but it is worth noting that reviewers have been prepared to “not approve” papers and that the reviews – all publicly available – are on occasions highly critical.

Responding to survey results

Our authors also felt that the information about the peer review process could be clearer, especially with regard to how and when to respond to their online reviews and at what point they should make their revisions. In light of this, we intend to streamline the authors user experience so they are fully aware what steps are needed and at what point to follow them. With the author having more autonomy in the peer review process and in the absence of an editor, it is important that the instructions and tools we provide to the author enables them to navigate the process in a simple and intuitive way.

Finally, our referee finder tool was well received, though only half the authors made use of it.  For those who did use it, it not only helped find potential reviewers, but also helped to identify new collaborators by bringing the authors attention to research groups they were previously unaware of.  In the words of one researcher:

“We chose referees relevant to the project, from the selector tool. In fact, one of them is now coming to do a seminar at my institution, so the process has also led to networking and potentially collaboration opportunities for us.”

With this in mind, we will work on making this tool more integrated and visible at the point where authors are selecting the reviewers for their article as it seems to be a very good complement to their own suggestions, and it is helping ensure the correct reviewers are being selected.

We will continue to survey our authors as more of them publish on the platform. We thank those who participated this time around and through this community feedback will make the necessary changes to keep improving the Wellcome Open Research platform.

SciLite – an open annotation platform for sustainable curation

Scientific publications are the main medium for sharing scientific results and assertions supported by observational data. Consequently, bioinformatics resources depend on research literature to keep the content updated; a task carried out by curators, who extract information from articles and transfer its essence to the corresponding resources.

The advances made in high-throughput technology have resulted in a tremendous growth of biological data, increasing the number of research papers being published. It provides a great challenge for manual curation that relies on finding the right articles and assimilating facts described in them. Therefore, services that support researchers and curators in browsing the content and identifying key biological concepts with minimal effort would be beneficial for the community.

 

What is SciLite?

 

We at the literature services group, EMBL-EBI, host Europe PMC, a database for life science literature, a partner in PubMed Central International. Europe PMC hosts a large variety of content and provides free access to over 32 million abstracts (27 million from PubMed) and 4 million full-text articles.

Our goal is to develop Europe PMC as an open community platform for new developments that improve our interaction with the scientific literature. As a part of this effort we have recently launched a new Europe PMC tool – SciLite, which we present in our Software Tool Article published on Wellcome Open Research. SciLite presents an opportunity for text miners to showcase their work to a wider public. SciLite exposes text-mined annotations and provides deep links with related data to a wide audience of scientists and curators, as well as other interested stakeholders.

 

How does SciLite work?

 

SciLite links text mined annotations from literature to the corresponding data resource and highlights those outputs on full text articles and abstracts in Europe PMC. Using the checkboxes on the right-hand side of article pages, readers can select the type of concepts that they are interested in, and matching annotations for that article will be highlighted on the article text as below. Clicking on the highlighted terms in the text opens a popup with information about the given annotation, such as a link to related database record and the source of the annotation.

 

What types of annotations are available?

 

SciLite annotates articles by identifying concepts, such as gene/protein names, organisms, diseases, Gene Ontology terms, chemicals, and accession numbers, as well as biological events (e.g. phosphorylation). The latter annotations are provided by the National Centre for Text Mining. SciLite also displays gene function annotations (GeneRIF – Gene Reference into Function) contributed by the Bibliomics and Text Mining group at the University of Applied Sciences, Geneva.

 

Are all annotations correct?

 

Although text-mining algorithms have greatly improved over the years and are being actively used in real-world applications, inaccuracies do occur. To counteract that we have introduced a user-driven mechanism to refine the annotations. While reading a paper, users of Europe PMC can validate or report an erroneous annotation (see example below). Such feedback ensures the quality of provided annotations and improves the text-mined outputs.

 

How is SciLite useful?

 

For the reader SciLite makes it very easy to skim-read articles, focusing on highlighted terms and concepts and helping to quickly understand what a given article is about. Those annotated entities are linked to the corresponding resources, so the reader can comfortably get to the underlying data in a straightforward way. In addition, SciLite could be useful for fetching related concepts from the text, as annotations highlighted in close proximity might signal a functional relationship between those terms, e.g., gene-disease association.

 

What are the future plans for SciLite?

 

We believe SciLite has the potential to further enhance the reading experience of scientific articles by developing applications that improve full text searching, filtering and integration with biological data. We have taken an initial step towards this for Protein Data Bank (PDB) accession numbers with the BioJS application. For a given PDB accession number it fetches the coordinate information and displays the corresponding 3D molecular structure, serving as an interactive visualiser (see below). Similar applications could be developed to display relevant information for a given annotation type in the context of the article.

 

How can you contribute to SciLite?

 

We encourage sharing annotations from text-mining and other associated communities on the SciLite platform. We have set up a participation page to assist interested groups to submit annotation data. Furthermore, the annotations on SciLite are modelled based on the Web Annotation Data Model specification, and the open nature of the format allows other platforms, such as journal publisher websites or other content aggregators, to fetch these annotations from SciLite to be reflected on their resource.

 

If you would like to find out more about developments at Europe PMC, visit https://europepmc.org/Roadmap or follow @EuropePMC_news on Twitter.

A month of Wellcome Open Research

A review of the first month of Wellcome Open Research by Michael Markie, Publisher, F1000, and Robert Kiley, Head of Digital Services, Wellcome.

It’s been just over a month since we launched Wellcome Open Research, and already we are seeing researchers embrace the opportunities that the platform presents to enable them to share their work openly and without delay. The first set of articles published are a testament to the initial goals we set ourselves: making research outputs available faster while supporting reproducibility and transparency.

The story so far (you can also view as a PDF here):

Authors

At the time of writing we have published 31 papers from 232 authors who represent 62 institutions – including Wellcome research centres and institutes and major overseas programmes  – covering 17 different countries. The demographic of authors spans a wide breadth of career stages ranging from Masters students all the way to Senior Investigators.

As we are keen to enhance the impact of our authors’ work, we are using our blog for researchers to tell their own story about their research and why they chose to publish on the platform. For example, we have highlighted two Postdoctoral Fellows, María Rodríguez-López and Cristina Cotobal from University College London, who describe their new CRISPR/Cas9-based protocol and primer design tool. We also recently featured Charles Bangham, Head of the Division of Infectious Diseases, Imperial College London, who describes his article on the association of free serum haemoglobin with brain atrophy in secondary progressive multiple sclerosis and also his positive experience using the platform:

 “I would strongly encourage other Wellcome grant-holders to publish on Wellcome Open Research. I think it has real potential to improve the standard, speed and fairness of scientific publishing.”

Charles Bangham, Imperial College London

 Articles

There are currently 31 published articles on Wellcome Open Research. These cover a wide range of subject areas, including cell biology, genetics and genomics, infectious diseases, public health and science education. So far, the articles have attracted considerable attention with nearly 12,000 views and over 1000 downloads.  These metrics are further enriched by Altmetric data which shows the level of community engagement through Twitter, news outlets and other services.

The published research also covers a range of article types from the traditional “research article” through to study protocols and data notes.  In total the breakdown of publications by article type is as follows:

  • 10 research articles
  • 8 method articles
  • 4 research notes (shorter articles with a few descriptive figures/tables)
  • 3 data notes (descriptions of datasets that include details of why and how the data were created)
  • 3 software tool articles
  • 3 study protocols

As the platform requires that the source data underlying the results are made available the published articles make clear where these data and code can be accessed.  An analysis of the data and software availability statements – a mandatory piece of metadata for all articles – reveals that there are 41 open datasets in public repositories such as FigShare and the Open Science Framework, 15 datasets available in established, field-specific repositories such as the ENA, Genbank, NCBI GEO and PRIDE, and seven cases of software source code being made permanently available in Zenodo. Making the data and software code available enables readers and users to reanalyze, replicate and reuse the data from each article.

Peer Review

At the time of writing there are 44 open peer review reports online which have been collectively viewed 904 times. Reviews are accompanied by the reviewers’ names and are individually citable as they all receive a DOI. Reviewers for Wellcome Open Research are also taking up the option to add their reports directly to their ORCID account to show a record of their report and enable them to get credit for their time and expertise:

 

Peer-review displayed on an ORCID account

The most impressive aspect of the peer review so far is the speed. The median time to the first referee report for an article is 8.5 days and the median time for two referee reports is 16 days; this is remarkably quick compared to the traditional peer review system.

Funding

Wellcome Open Research is centrally funded by Wellcome, so authors do not have to deal with any article processing charges (APCs); the costs are automatically covered allowing authors to quickly and efficiently submit their research without this burden.

We also know that most research is the result of collaboration and funded from multiple sources; just over half of the articles published on the platform include non-Wellcome funding information. Linking funding information directly to research outputs helps to improve grant-related impact tracking; these funding data are important pieces of an article’s meta-data and are deposited with CrossRef.

All in all, it’s been a hugely positive start. We hope to continue with this success next year with more Wellcome-funded researchers seeking to benefit from the platform to share their work in an open and reproducible way. There will also be some exciting new features added to the platform in 2017, so to be kept abreast about these new developments please do sign up for more information via the Wellcome Open Research homepage.

Happy Holidays.

“Hepitopes: representing why I am an enthusiastic pioneer of the Wellcome Open Research platform.”

Hepitopes’ is a new online resource, comprising a database of immune responses to Hepatitis B virus (HBV). We believe this will be an important resource for the HBV research community, with implications that range from characterising the basic science of virology and immunology, through to informing vaccine design and understanding clinical outcomes of infection. The database is designed as a live interactive resource that will evolve and develop over time, with improvements in the quality and content of the data, links to other tools and resources, and the potential to underpin scientific dialogue and new collaborations. Continue reading ““Hepitopes: representing why I am an enthusiastic pioneer of the Wellcome Open Research platform.””

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑