Peer review is the characteristic that defines academic communication. In fact, in a survey conducted in 2018 to more than 11,000 members of the scientific community, 98% considered it important or extremely important to ensure the quality and integrity of academic communication. Despite this, “research on peer review is scarce” and “fragmented, with few connections and with limitations when it comes to sharing knowledge”, as noted in the article «Unlock ways to share data on peer review». How is the evaluation process managed? How do you assess the quality and usefulness of individual reviews? What factors can bias editorial decisions? These are some of the issues to be resolved.

Francisco Grimaldo, deputy director of the School of Engineering (ETSE-UV) and researcher in the state PREWAIT project (which analyses peer review with advanced information tools), highlights the need to share all knowledge and “look at the hidden part of the iceberg”. That is to say, discover all those comments and changes hidden in the eyes of the readers from the moment an original about a scientific investigation is received until the final article, the visible product, is published.

As also noted in the Nature publication, work has been done to achieve this goal for decades, but this subject is “poorly studied”. Access to data on the review process is difficult and often depends on personal connections with the journals, the signatories regret. Therefore, while it is true that they have been working for years, Grimaldo is aware that “there is much more to do” and believes that “the fact that Nature publishes it gives strength to this idea”.

The publication describes a pilot project to encourage a comprehensive systematic review of peer review and cites some positive experiences in which data sharing has improved processes. In addition, the importance of interdisciplinary teams that investigate the peer review both quantitatively and qualitatively is defended, in order to understand what evaluation models work best in different circumstances and why.

In short, the note in Nature asks that all these data on peer review can be collected, ensuring careful and responsible management and within the European legal framework. However, still, publishers, both private and non-profit, consider it a risky policy to show how they work.

The role of ESTE-UV in data management

In 2014, academic staff, scientific publishers and entities to finance research began to collaborate in the European PEERE project. The members of the group (among which there were many of the authors who have now signed this note in Nature), considered that in Europe the evaluation by experts was not being studied in a global way [specifically, PEERE is a European Cooperation action in Science and Technology, COST]. In 2017, a protocol was signed to share and study the data generated in the review processes between the University of Valencia and Brescia, in collaboration with the Royal Society and the publishing houses Elsevier, Springer Nature and Wiley.

It is the ETSE-UV that hosts a repository with this information and manages access. From the analysed data, a dozen articles have been published in impact journals such as Nature, Nature Communications, Journal of Informetrics, PLOS ONE or Scientometrics.

International conference in September

Parallel to the publication of this note, the ETSE-UV finalises the preparations for the second edition of the international PEERE conference, which will be held in Valencia next September. Taking over the first edition, held in Rome in 2018, it will deal, among others, with several evaluation models by experts; good editorial practices; possible biases in reviews; of the evaluation of works not only for scientific journals, or opening – or at least, sharing – these data. In summary, over three days it is intended to create a forum in which scientific and professional staff and other interested parties share their knowledge, research and experiences, but also concerns, about peer review in various scientific fields. And, also, to promote an investigation on the more systematic subject.

The organisation of the second PEERE conference is one of the activities that make up the state project PREWAIT: Advanced information tools about peer review of scientific manuscripts [RTI2018-095820-B-I00]. In this project research staff of the University of Valencia participates as Grimaldo himself, and also Emilio Soria Olivas, Emilia López Iñesta, José Rafael Magdalena Benedicto, Daniel García Costa and Joan Vila-Francés, belonging to the IDAL (Intelligent Data Analysis Laboratory) research group.

Article:

Squazzoni, Flaminio et al. «Unlock ways to share data on peer review». Nature 578(7796):512-514. February 2020. DOI: 10.1038/d41586-020-00500-y

Enlace: https://www.nature.com/articles/d41586-020-00500-y