Today peer review is the cornerstone of the scholarly publication process, demarcating science published following peer review as purportedly more rigorous, robust, and reliable than non-peer-reviewed research. However, reviewers can also be biased, petty, or uninformed, and even when done correctly, the peer-review process delays publication by months to years.

Leveraging two new developments: the increasing popularity of preprints, and the advent of artificial intelligence tools in the field of natural-language processing (NLP), a project directed by Uri Maoz at Chapman University will use a machine-learning-based approach to explore whether the benefits of peer review for scientific publications outweigh its costs.

The project team aims to: 

• Construct a database of at least 10,000 preprint-paper pairs. The database will be constructed according to FAIR principles (Findability, Accessibility, Interoperability, and Reuse of digital assets), published, and made freely available to the scientific community and the general public.
   
• Build two NLP models. The first will be trained to tell apart non-peer-reviewed preprints from their corresponding peer-reviewed publications based on their textual content. The second model will make use of the metadata of each preprint-publication pair to test for potential bias based on the gender and ethnicity of the authors, the name recognition of the university, the journal, and so on. 

With this, they will investigate the value of peer review in scientific publications on a relatively large scale. They will also study the effect of belonging to underrepresented populations in science on the peer-review process. Their study, therefore, promises to shed new light both on the value of the peer-review process as a whole, and in particular how fair it is toward populations that are underrepresented in science.