Abstract
The phenomenon of plagiarism in peer-review evaluation reports remained surprisingly unrecognized, despite a notable rise of such cases in recent years. This study reports multiple cases of peer-review plagiarism recently detected in 50 different scientific articles published in 19 journals. Their in-depth analysis reveals that such reviews tend to be nonsensical, vague and unrelated to the actual manuscript. The analysis is followed by a discussion of the roots of such plagiarism, its consequences and measures that could counteract its further spreading. In addition, we demonstrate how increased availability and access to AI technologies through recent emergence of chatbots may be misused to write or conceal plagiarized peer-reviews. Plagiarizing reviews is a severe misconduct that requires urgent attention and action from all affected parties.
This is a preview of subscription content, log in via an institution to check access.
Data availability
All data supporting this article are included as Supplementary Materials.
References
Aghemo, A., Forner, A., & Valenti, L. (2023). Should artificial intelligence-based language models be allowed in developing scientific manuscripts? A debate between ChatGPT and the editors of Liver International. Liver International, 43, 956–957. https://doi.org/10.1111/liv.15580
Amano, T., Ramírez-Castañeda, V., Berdejo-Espinola, V., Borokini, I., Chowdhury, S., Golivets, M., & Veríssimo, D. (2023). The manifold costs of being a non-native English speaker in science. PLoS Biology, 21(7), e3002184. https://doi.org/10.1371/journal.pbio.3002184
Anonymous. (2012). How to stop plagiarism. Nature, 481, 21–23. https://doi.org/10.1038/481021a
Anonymous. (2020). Nature will publish peer review reports as a trial. Nature, 578, 8. https://doi.org/10.1038/d41586-020-00309-9
Anonymous. (2022). Transparent peer review for all. Nature Communications, 13, 6173. https://doi.org/10.1038/s41467-022-33056-8
Anonymous. (2023). Tools such as ChatGPT threaten transparent science; Here are our ground rules for their use. Nature, 613, 612. https://doi.org/10.1038/d41586-023-00191-1
Beck, E., Jr. (2003). Anonymous reviews: Self-serving, counterproductive, and unacceptable. Eos, Transactions American Geophysical Union, 84(26), 249. https://doi.org/10.1029/2003EO260005
Butler, D. (2010). Journals step up plagiarism policing. Nature, 466, 167. https://doi.org/10.1038/466167a
Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchi, G. (2021). AI-assisted peer review. Humanities and Social Sciences Communications, 8, 25. https://doi.org/10.1057/s41599-020-00703-8
Csomós, G., & Farkas, J. Z. (2023). Understanding the increasing market share of the academic publisher “multidisciplinary digital publishing institute” in the publication output of central and eastern European countries: A case study of hungary. Scientometrics, 128, 803–824. https://doi.org/10.1007/s11192-022-04586-1
D’Andrea, R., & O’Dwyer, J. P. (2017). Can editors save peer review from peer reviewers? PLoS ONE, 12(10), e0186111. https://doi.org/10.1371/journal.pone.0186111
Eini, M. R., Rahmati, A., Salmani, H., Brocca, L., & Piniewski, M. (2022). Detecting characteristics of extreme precipitation events using regional and satellite-based precipitation gridded datasets over a region in Central Europe. Science of the Total Environment, 852, 158497. https://doi.org/10.1016/j.scitotenv.2022.158497
Epstein, D., Wiseman, V., Salaria, N., & Mounier-Jack, S. (2017). The need for speed: The peer-review process and what are we doing about it? Health Policy and Planning, 32(10), 1345–1346. https://doi.org/10.1093/heapol/czx129
Fiala, C., & Diamandis, E. P. (2017). The emerging landscape of scientific publishing. Clinical Biochemistry, 50(12), 651–655. https://doi.org/10.1016/j.clinbiochem.2017.04.009
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Yuan, L., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. Npj Digital Medicine, 6, 75. https://doi.org/10.1038/s41746-023-00819-6
Halsted, G. B. (1896). Compliment or plagiarism. Science, 4(102), 877–878.
He, Y., Tian, K., & Xu, X. (2023). A validation study on the factors affecting the practice modes of open peer review. Scientometrics, 128, 587–607.
Helgesson, G., & Eriksson, S. (2015). Plagiarism in research. Medicine, Health Care and Philosophy, 18, 91–101. https://doi.org/10.1007/s11019-014-9583-8
Jarić, I. (2016). High time for a common plagiarism detection system. Scientometrics, 106, 457–459. https://doi.org/10.1007/s11192-015-1756-6
Knoepfler, P. (2015). Reviewing post-publication peer review. Trends in Genetics, 31(5), 221–223. https://doi.org/10.1016/j.tig.2015.03.006
Koshy, K., Fowler, A. J., Gundogan, B., & Agha, R. A. (2018). Peer review in scholarly publishing part A: Why do it? International Journal of Surgery Oncology, 3(2), e56. https://doi.org/10.1097/IJ9.0000000000000056
Kundzewicz, Z. W., & Koutsoyiannis, D. (2005). Editorial—The peer-review system: Prospects and challenges. Hydrological Sciences Journal, 50(4), 577–590.
Lykkesfeldt, J. (2016). Strategies for using plagiarism software in the screening of incoming journal manuscripts: Recommendations based on a recent literature survey. Basic & Clinical Pharmacology & Toxicology, 119, 161–164. https://doi.org/10.1111/bcpt.12568
Maurer, H., Kappe, F., & Zaka, B. (2006). Plagiarism—A survey. Journal of Universal Computer Science. https://doi.org/10.3217/jucs-012-08-1050
Mercer J. G. (2022). Excellence in peer-review: The hallmark and fundamental of a good society-owned journal. Journal of Neuroendocrinology. 34(9):e13173. https://doi.org/10.1111/jne.13173
Resnik, D. B., & Elmore, S. A. (2016). Ensuring the quality, fairness, and integrity of journal peer review: A possible role of editors. Science and Engineering Ethics, 22, 169–188. https://doi.org/10.1007/s11948-015-9625-5
Roig, M. (2015). Avoiding plagiarism, self-plagiarism, and other questionable writing practices: A guide to ethical writing. The Office of Research Integrity (ORI)
Rogerson, A. M., & McCarthy, G. (2017). Using internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism? International Journal for Educational Integrity, 13, 1–15. https://doi.org/10.1007/s40979-016-0013-y
Ross-Hellauer, T. (2017). What is open peer review? A systematic review. Reserach, 6, 588. https://doi.org/10.12688/f1000research.11369.2
Ross-Hellauer, T., Deppe, A., & Schmidt, B. (2017). Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PLoS ONE, 12(12), e0189311. https://doi.org/10.1371/journal.pone.0189311
Squazzoni, F., Grimaldo, F., & Marušić, A. (2017). Publishing: Journals could share peer-review data. Nature, 546, 352. https://doi.org/10.1038/546352a
Srivastava, M. (2023). A day in the life of ChatGPT as an academic reviewer: Investigating the potential of large language model for scientific literature review. https://doi.org/10.31219/osf.io/wydct
Teixeira da Silva, J. A., & Nazarovets, S. (2022). The role of publons in the context of open peer review. Publishing Research Quarterly, 38, 760–781. https://doi.org/10.1007/s12109-022-09914-0
Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science. https://doi.org/10.1126/science.adg7879
Venegas-Cordero, N., Cherrat, C., Kundzewicz, Z. W., Singh, J., & Piniewski, M. (2023). Model-based assessment of flood generation mechanisms over Poland: The roles of precipitation, snowmelt, and soil moisture excess. Science of the Total Environment, 891, 164626. https://doi.org/10.1016/j.scitotenv.2023.164626
Weber-Wulff,. (2019). Plagiarism detectors are a crutch, and a problem. Nature, 567, 435. https://doi.org/10.1038/d41586-019-00893-5
Wolfram, D., Wang, P., & Abuzahra, F. (2021). An exploration of referees’ comments published in open peer review journals: The characteristics of review language and the association between review scrutiny and citations. Research Evaluation, 30(3), 314–322. https://doi.org/10.1093/reseval/rvab005
Wolfram, D., Wang, P., Hembree, A., & Park, H. (2020). Open peer review: Promoting transparency in open science. Scientometrics, 125, 1033–1051. https://doi.org/10.1007/s11192-020-03488-4
Acknowledgements
The authors thank an anonymous reviewer for providing helpful comments and suggestions that improved the manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Piniewski, M., Jarić, I., Koutsoyiannis, D. et al. Emerging plagiarism in peer-review evaluation reports: a tip of the iceberg?. Scientometrics 129, 2489–2498 (2024). https://doi.org/10.1007/s11192-024-04960-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-024-04960-1