Skip to main content
SpringerLink
Log in

(Mis-)Classification of 17,721 Journals by an Artificial Intelligence Predatory Journal Detector

  • Published:
Publishing Research Quarterly Aims and scope

Abstract

The SCImago Journal Rank (SJR) ranks journals into four quartiles (Q1–Q4). SJR serves as a safelist for journal selection, when trying to avoid predatory journals, as journals that have been indexed by SJR are seen as having stringent publishing standards. An AI-based tool, the Academic Journal Predatory Checking System (AJPC), claims to be able to differentiate suspected predatory journals from normal journals. In this study, we asked (25 March 2023) AJPC to classify the top 2500 journals (based on SJR ranking). We found that 65.64% of these journals were classified as “suspected predatory”, 33.28% were classified as “normal”, while 1.08% of the journals could not be classified. A follow-up run (30 March 2023) returned different results (89.20%, 10.16% and 0.64%). This set of results is worrying, not only because of the number of SJR journals that are thought to be “predatory”, but also because of the difference in results when using the same tool a matter of days apart. We extended our analysis and looked at seven stand-alone publishers (Elsevier, Frontiers, MDPI, OMICS, Springer Nature (incl. Nature Portfolio), Taylor & Francis and Wiley). In total, we asked AJPC to classify 17,721 journals. The results confirm that we should be concerned about the classifications provided by AJPC. For example, 100% (4756) of Elsevier journals were classified as “suspected predatory”. Even Springer Nature (which includes the journal that published the AJPC paper) has journals which were classified at “suspected predatory”. We thus urge caution against relying on AJPC at present. We also urge the AJPC’s authors to provide an explanation as to why a journal is classified in a particular way, as part of their user interface. We are willing to conduct further analyses should AJPC be revised and updated as it could provide an important service to the scholarly community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://140.113.207.51:8000/

  2. We suspect that this limit was imposed following our heavy usage, for which we apologize.

  3. https://beallslist.net/standalone-journals/. Last accessed: 30 March 2023.

  4. https://predatoryjournals.com/journals. This is the URL provided on the AJPC website, but the link is broken, when we attempted to access it (30 March 2023). The last archived memento at the Internet Archive is from 20 December 2021: https://web.archive.org/web/20211220083526/https://predatoryjournals.com/journals/

  5. http://s-quest.bihealth.org:3838/OAWhitelist/. This is the URL provided on the AJPC website, but the link is broken, when we attempted to access it (30 March 2023). The last archived memento at the Internet Archive is from 3 March 2021: https://web.archive.org/web/20210303113839/http://s-quest.bihealth.org:3838/OAWhitelist/. We also found the following possibly related websites: https://github.com/quest-bih; https://github.com/quest-bih/OpenAccessPositiveList

  6. https://webscraper.io/. Last accessed: 30 March 2023.

References

  1. Grudniewicz A, Moher D, Cobey KD, Bryson GL, Cukier S, Allen K, Ardern C, Balcom L, Barros T, Berger M, Ciro JB, Cugusi L, Donaldson MR, Egger M, Graham ID, Hodgkinson M, Khan KM, Mabizela M, Manca A, Milzow K, Mouton J, Muchenje M, Olijhoek T, Ommaya A, Patwardhan B, Poff D, Proulx L, Rodger M, Severin A, Strinzel M, Sylos-Labini M, Tamblyn R, van Niekerk M, Wicherts JM, Lalu MM. Predatory journals: no definition, no defence. Nature. 2019;576(7786):210–2. https://doi.org/10.1038/d41586-019-03759-y.

    Article  Google Scholar 

  2. Frandsen TF. Are predatory journals undermining the credibility of science? a bibliometric analysis of citers. Scientometrics. 2017;113(3):1513–28. https://doi.org/10.1007/s11192-017-2520.

    Article  Google Scholar 

  3. Eriksson S, Helgesson G. The false academy: predatory publishing in science and bioethics. Med Health Care Philos. 2017;20(2):163–70. https://doi.org/10.1007/s11019-016-9740-3.

    Article  Google Scholar 

  4. Manley S. Predatory journals on trial. Allegations, responses, and lessons for scholarly publishing from FTC v. OMICS. J Sch Publ. 2019;50(3):183–200. https://doi.org/10.3138/jsp.50.3.02.

    Article  Google Scholar 

  5. Manley S. On the limitations of recent lawsuits against Sci-Hub, OMICS, ResearchGate, and Georgia State University. Learn Publ. 2019;32(4):375–81. https://doi.org/10.1002/leap.1254.

    Article  Google Scholar 

  6. Siler K, Vincent-Lamarre P, Sugimoto CR, Larivière V. Predatory publishers’ latest scam: bootlegged and rebranded papers. Nature. 2021;598(7882):563–5. https://doi.org/10.1038/d41586-021-02906-8.

    Article  Google Scholar 

  7. Teixeira da Silva JA, Moradzadeh M, Adjei KOK, Owusu-Ansah CM, Balehegn M, Faúndez EI, Janodia MD, Al-Khatib A. An integrated paradigm shift to deal with “predatory” publishing. J Acad Librariansh. 2022;48(1):102481. https://doi.org/10.1016/j.acalib.2021.102481.

    Article  Google Scholar 

  8. Teixeira da Silva JA, Moradzadeh M, Yamada Y, Dunleavy DJ, Tsigaris P. Cabells’ predatory reports criteria: assessment and proposed revisions. J Acad Librariansh. 2023;49(1):102659. https://doi.org/10.1016/j.acalib.2022.102659.

    Article  Google Scholar 

  9. Teixeira da Silva JA, Tsigaris P. Issues with criteria to evaluate blacklists: an epidemiological approach. J Acad Librariansh. 2020;46(1):102070. https://doi.org/10.1016/j.acalib.2019.102070.

    Article  Google Scholar 

  10. Tsigaris P, Teixeira da Silva JA. Why blacklists are not reliable: a theoretical framework. J Acad Librariansh. 2021;47:102266. https://doi.org/10.1016/j.acalib.2020.102266.

    Article  Google Scholar 

  11. Teixeira da Silva JA, Dobránszki J, Tsigaris P, Al-Khatib A. Predatory and exploitative behaviour in academic publishing: an assessment. J Acad Librariansh. 2019;45(6):102071. https://doi.org/10.1016/j.acalib.2019.102071.

    Article  Google Scholar 

  12. Yamada Y, Teixeira da Silva JA. A psychological perspective towards understanding the objective and subjective gray zones in predatory publishing. Qual Quant. 2022;56(6):4075–87. https://doi.org/10.1007/s11135-021-01307-3.

    Article  Google Scholar 

  13. Chen L-X, Su S-W, Liao C-H, Wong K-S, Yuan S-M. An open automation system for predatory journal detection. Sci Rep. 2023;13(1):2976. https://doi.org/10.1038/s41598-023-30176-z.

    Article  Google Scholar 

  14. Ateeq WMB, Al-Khalifa HS. Intelligent framework for detecting predatory publishing venues. IEEE Access. 2023;11:20582–618. https://doi.org/10.1109/ACCESS.2023.3250256.

    Article  Google Scholar 

  15. Jaafar R, Pereira V, Saab SS, El-Kassar A-N. Which journal ranking list? a case study in business and economics. EuroMed J Bus. 2021;16(4):361–80. https://doi.org/10.1108/EMJB-05-2020-0039.

    Article  Google Scholar 

  16. Beall J. What I learned from predatory publishers. Biochem Med. 2017;27(2):273–8. https://doi.org/10.11613/BM.2017.029.

    Article  Google Scholar 

  17. Kendall G. Beall’s legacy in the battle against predatory publishers. Learn Publ. 2021;34(3):379–88. https://doi.org/10.1002/leap.1374.

    Article  Google Scholar 

  18. Bloudoff-Indelicato M. Backlash after frontiers journals added to list of questionable publishers. Nature. 2015;526:613. https://doi.org/10.1038/526613f.

    Article  Google Scholar 

  19. Oviedo-García MÁ (2021) Journal citation reports and the definition of a predatory journal: The case of the Multidisciplinary Digital Publishing Institute (MDPI). Res Eval 2021;30(3):405–419. https://doi.org/10.1093/reseval/rvab020; expression of concern: Res Eval 2021;30(3):420. https://doi.org/10.1093/reseval/rvab030; retracted and replaced: Res Eval (in press). https://doi.org/10.1093/reseval/rvad014

  20. Csomós G, Farkas JZ. Understanding the increasing market share of the academic publisher “multidisciplinary digital publishing institute” in the publication output of Central and Eastern European countries: a case study of Hungary. Scientometrics. 2023;128(1):803–24. https://doi.org/10.1007/s11192-022-04586-1.

    Article  Google Scholar 

  21. Kendall G, Linacre S. Predatory journals: revisiting beall’s research. Publ Res Q. 2022;38(3):530–43. https://doi.org/10.1007/s12109-022-09888-z.

    Article  Google Scholar 

  22. Teixeira da Silva JA. Reflections on the disappearance of Dolos list, a now-defunct “predatory” publishing blacklist. Open Inf Sci. 2022;6(1):136–42. https://doi.org/10.1515/opis-2022-0136.

    Article  Google Scholar 

  23. Teixeira da Silva JA, Tsigaris P, Moussa S. Can AI detect predatory journals? the case of FT50 journals. SSRN (Preprint, Not Peer Reviewed). 2023. https://doi.org/10.2139/ssrn.4391108.

    Article  Google Scholar 

  24. Yamada Y, Teixeira da Silva JA. A measure to quantify predatory publishing is urgently needed. Account Res (in press). 2023. https://doi.org/10.1080/08989621.2023.2186225.

    Article  Google Scholar 

  25. Teixeira da Silva JA, Daly T. The diagnostic accuracy of AI-based predatory journal detectors: an analogy to diagnosis. Diagnosis (in press). 2023. https://doi.org/10.1515/dx-2023-0039.

    Article  Google Scholar 

Download references

Funding

No funding was received by either author for this research.

Author information

Authors and Affiliations

  1. Independent Researcher, Ikenobe 3011-2, Kagawa-ken, 761-0799, Japan

    Jaime A. Teixeira da Silva

  2. University of Nottingham (UK and Malaysia), University Park, Nottingham, NG7 2RD, UK

    Graham Kendall

Authors
  1. Jaime A. Teixeira da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Graham Kendall
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

GK conducted the web scraping and automated AJPC tests. Apart from that, both authors contributed equally to all other aspects of the paper.

Corresponding authors

Correspondence to Jaime A. Teixeira da Silva or Graham Kendall.

Ethics declarations

Conflict of interest

Neither author declares any relevant conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 1037 kb)

Supplementary file2 (DOCX 22 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Teixeira da Silva, J.A., Kendall, G. (Mis-)Classification of 17,721 Journals by an Artificial Intelligence Predatory Journal Detector. Pub Res Q 39, 263–279 (2023). https://doi.org/10.1007/s12109-023-09956-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12109-023-09956-y

Keywords

Search

Navigation