Skip to main content
SpringerLink
Log in

Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold

  • Original Article
  • Published:
European Journal of Orthopaedic Surgery & Traumatology Aims and scope Submit manuscript

Abstract

Background

The impact factor (IF) is the most commonly used bibliometric method for rating academic journals. However, the practice of journals’ self-citation may artificially elevate the IF. Additional bibliometric methods including Eigenfactor scale, SCImago Journal Ranking (SJR), and corrected IF (cIF) have been created. Comparing general-interest and specialized orthopaedic journals, the aims of this study were to assess: (1) the effect of journal´s self-citation on IF; (2) differences in bibliometric analysis; and (3) to determine thresholds for monitoring self-citation practices by defining the self-citation in orthopaedic research (SCOR) Threshold.

Methods

The journal citation reports and SCImago Journal and Country Rank databases were queried for orthopaedic journals from 1997 to 2017. The following bibliometrics were compared between general-interest and specialized journals: IF, cIF, Eigenfactor, self-citation rates, and SJR. A novel metric, the cIF ratio, was proposed to represent the relationship between a journal’s IF and cIF. Thresholds for cIF were based on statistical outliers of cIF ratio within general-interest and specialized journals were calculated. Outliers were defined as data points that were greater than the third quartile by 1.5 times the interquartile range using the last 10 years studied (2007–2017).

Results

Specialized orthopaedic journals had a higher median self-citation rates compared to general-interest journals (11.85% vs. 6.36%, p < 0.001). Overall, cIF ratio declined over study period, and general-interest journals had a lower cIF ratio than specialized journals (8.77% vs. 19.54%, p < 0.001). Overall, general-interest journals had more favourable values for the bibliometric indices studied compared to specialized journals The SCOR threshold for cIF ratio was determined as 25.4% for general-interest journals and 53.3% for specialized journals.

Conclusion

Overall, self-citation occurs at a higher rate in specialized versus general-interest orthopaedic journals. We propose the use of a cIF ratio along with the SCOR threshold as a tool to evaluate and monitor journal self-citation practices in orthopaedic research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295:90–93. https://doi.org/10.1001/jama.295.1.90

    Article  PubMed  CAS  Google Scholar 

  2. Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178:471–479

    Article  CAS  Google Scholar 

  3. Garfield E (1996) How can impact factors be improved? BMJ Br Med J 313:411–413

    Article  CAS  Google Scholar 

  4. Butler JS, Sebastian AS, Kaye ID et al (2017) Understanding traditional research impact metrics. Clin Spine Surg 30:164–166. https://doi.org/10.1097/BSD.0000000000000530

    Article  PubMed  Google Scholar 

  5. Brown T, Gutman SA (2018) Impact factor, eigenfactor, article influence, scopus SNIP, and SCImage journal rank of occupational therapy journals. Scand J Occup Ther. https://doi.org/10.1080/11038128.2018.1473489

    Article  PubMed  Google Scholar 

  6. van der Pol CB, McInnes MDF, Petrcich W et al (2015) Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates? PLoS ONE 10:e0119892. https://doi.org/10.1371/journal.pone.0119892

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  7. Garner RM, Hirsch JA, Albuquerque FC, Fargen KM (2018) Bibliometric indices: defining academic productivity and citation rates of researchers, departments and journals. J Neurointerv Surg 10:102–106. https://doi.org/10.1136/neurintsurg-2017-013265

    Article  PubMed  Google Scholar 

  8. Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS ONE 5:e10204. https://doi.org/10.1371/journal.pone.0010204

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  9. Kurmis AP (2003) Understanding the limitations of the journal impact factor. J Bone Joint Surg Am 85:2449–2454

    Article  Google Scholar 

  10. Brown T (2011) Journal quality metrics: options to consider other than impact factors. Am J Occup Ther 65:346–350

    Article  Google Scholar 

  11. Miro O, Brown AFT, Graham CA et al (2015) Relationship between category size and journals’ impact factor: implications for emergency medicine journals and researchers. Eur J Emerg Med 22:355–362. https://doi.org/10.1097/MEJ.0000000000000184

    Article  PubMed  Google Scholar 

  12. Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314:498–502

    Article  CAS  Google Scholar 

  13. Clarivate Analytics Title Suppressions: Editorial Expression of Concern. In: InCites J. Cit. Reports Help. http://ipscience-help.thomsonreuters.com/incitesLiveJCR/JCRGroup/titleSuppressions.html

  14. Siebelt M, Siebelt T, Pilot P et al (2010) Citation analysis of orthopaedic literature; 18 major orthopaedic journals compared for impact factor and SCImago. BMC Musculoskelet Disord. https://doi.org/10.1186/1471-2474-11-4

    Article  PubMed  PubMed Central  Google Scholar 

  15. (1997) Science Citation Index, Journal Citation Reports. Philadelphia, Pennsylvania, USA

  16. Hoaglin DC (2003) John W. Tukey and data analysis. Stat Sci 18:311–318. https://doi.org/10.1214/ss/1076102418

    Article  Google Scholar 

  17. Hawkinson MP, Krueger CA, Carroll J (2018) Self-citation does not appear to artificially inflate orthopaedic journal ranking. J Surg Orthop Adv 27:131–135

    Article  Google Scholar 

  18. Moverley R, Rankin KS, McNamara I et al (2013) Impact factors of orthopaedic journals between 2000 and 2010: trends and comparisons with other surgical specialties. Int Orthop 37:561–567. https://doi.org/10.1007/s00264-012-1769-1

    Article  PubMed  PubMed Central  Google Scholar 

  19. Mohamed NS, Gwam CU, Etcheson JI et al (2018) Impact factors of orthopaedic journals between 2010 and 2016: trends and comparisons with other surgical specialties. Ann Transl Med 6:114. https://doi.org/10.21037/atm.2018.03.02

    Article  PubMed  PubMed Central  Google Scholar 

  20. Fassoulaki A, Paraskeva A, Papilas K, Karabinis G (2000) Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth 84:266–269

    Article  CAS  Google Scholar 

  21. Miyamoto S (2018) Self-citation rate and impact factor in the field of plastic and reconstructive surgery. J Plast Surg Hand Surg 52:40–46. https://doi.org/10.1080/2000656X.2017.1319847

    Article  PubMed  Google Scholar 

  22. Mimouni M, Segal O (2014) Self-citation rate and impact factor in ophthalmology. Ophthalmic Res 52:136–140. https://doi.org/10.1159/000366284

    Article  PubMed  Google Scholar 

  23. Henderson AH, Upile T, Pilavakis Y, Patel NN (2016) Reporting guidelines and journal quality in otolaryngology. Clin Otolaryngol 41:461–466. https://doi.org/10.1111/coa.12546

    Article  PubMed  CAS  Google Scholar 

  24. Yuen J (2018) Comparison of Impact Factor, Eigenfactor Metrics, and SCImago Journal Rank Indicator and h-index for Neurosurgical and Spinal Surgical Journals. World Neurosurg. https://doi.org/10.1016/j.wneu.2018.07.144

    Article  PubMed  Google Scholar 

  25. Slobogean G, Bhandari M (2012) Introducing levels of evidence to the journal of orthopaedic trauma: implementation and future directions. J Orthop Trauma 26:127–128. https://doi.org/10.1097/BOT.0b013e318247c931

    Article  PubMed  Google Scholar 

  26. Bergstrom CT (2007) Eigenfactor: measuring the value and prestige of scholarly journals. Coll Res Libr News 68:314–316. https://doi.org/10.5860/crln.68.5.7804

    Article  Google Scholar 

  27. Guerrero-Bote VP, Moya-Anegón F (2012) A further step forward in measuring journals’ scientific prestige: the SJR2 indicator. J Informetr 6:674–688. https://doi.org/10.1016/j.joi.2012.07.001

    Article  Google Scholar 

  28. Lancho-Barrantes BS, Guerrero-Bote VP, Moya-Anegón F (2010) What lies behind the averages and significance of citation indicators in different disciplines? J Inf Sci 36:371–382. https://doi.org/10.1177/0165551510366077

    Article  Google Scholar 

  29. Janssens F, Zhang L, De Moor B, Glänzel W (2009) Hybrid clustering for validation and improvement of subject-classification schemes. Inf Process Manag 45:683–702. https://doi.org/10.1016/j.ipm.2009.06.003

    Article  Google Scholar 

Download references

Funding

No financial support was used for the conductance of this study. Outside the submitted work, Dr. Piuzzi reports board or committee membership in the Orthopaedic Research Society.

Author information

Authors and Affiliations

  1. Department of Orthopedic Surgery, Cleveland Clinic, 9500 Euclid Ave/A41, Cleveland, OH, 44195, USA

    Kavin Sundaram, Jared Warren, Hiba K. Anis, Alison K. Klika & Nicolas S. Piuzzi

Authors
  1. Kavin Sundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jared Warren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiba K. Anis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alison K. Klika
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicolas S. Piuzzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolas S. Piuzzi.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest related to the submitted work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sundaram, K., Warren, J., Anis, H.K. et al. Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold. Eur J Orthop Surg Traumatol 30, 629–635 (2020). https://doi.org/10.1007/s00590-019-02616-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00590-019-02616-y

Keywords

Search

Navigation