Skip to main content
SpringerLink
Log in

ChatGPT as an information tool in rhinology. Can we trust each other today?

  • Short Communication
  • Published:
European Archives of Oto-Rhino-Laryngology Aims and scope Submit manuscript

Abstract

Purpose

ChatGPT (Chat-Generative Pre-trained Transformer) has proven to be a powerful information tool on various topics, including healthcare. This system is based on information obtained on the Internet, but this information is not always reliable. Currently, few studies analyze the validity of these responses in rhinology. Our work aims to assess the quality and reliability of the information provided by AI regarding the main rhinological pathologies.

Methods

We asked to the default ChatGPT version (GPT-3.5) 65 questions about the most prevalent pathologies in rhinology. The focus was learning about the causes, risk factors, treatments, prognosis, and outcomes. We use the Discern questionnaire and a hexagonal radar schema to evaluate the quality of the information. We use Fleiss's kappa statistical analysis to determine the consistency of agreement between different observers.

Results

The overall evaluation of the Discern questionnaire resulted in a score of 4.05 (± 0.6). The results in the Reliability section are worse, with an average score of 3.18. (± 1.77). This score is affected by the responses to questions about the source of the information provided. The average score for the Quality section was 3.59 (± 1.18). Fleiss's Kappa shows substantial agreement, with a K of 0.69 (p < 0.001).

Conclusion

The ChatGPT answers are accurate and reliable. It generates a simple and understandable description of the pathology for the patient's benefit. Our team considers that ChatGPT could be a useful tool to provide information under prior supervision by a health professional.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Data availability

Not applicable.

References

  1. Biswas SS (2023) Role of chat GPT in Public Health. Ann Biomed Eng 51(5):868–869

    Article  PubMed  Google Scholar 

  2. Kim JK, Chua M, Rickard M, Lorenzo A (2023) ChatGPT and large language model (LLM) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicine. J Pediatric Urol 19(5):598–604

    Article  Google Scholar 

  3. Májovský M, Černý M, Kasal M, Komarc M, Netuka D (2023) Artificial intelligence can generate fraudulent but authentic-looking scientific medical articles: Pandora’s box has been opened. J Med Internet Res 25

  4. Szczesniewski JJ, Tellez Fouz C, Ramos Alba A, Diaz Goizueta FJ, García Tello A, Llanes González L (2023) ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patients. World J Urol 41(11):3149–3153

    Article  PubMed  Google Scholar 

  5. Bellinger JR, De La Chapa JS, Kwak MW, Ramos GA, Morrison D, Kesser BW (2023) BPPV information on google versus AI (ChatGPT). Otolaryngol Head Neck Surg. https://doi.org/10.1002/ohn.506

    Article  PubMed  Google Scholar 

  6. Nielsen JPS, von Buchwald C, Grønhøj C (2023) Validity of the large language model ChatGPT (GPT4) as a patient information source in otolaryngology by a variety of doctors in a tertiary otorhinolaryngology department. Acta Oto-Laryngol

  7. Dave T, Athaluri SA, Singh S (2023) ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023:6:1169595

    Article  PubMed  PubMed Central  Google Scholar 

  8. De Angelis L, Baglivo F, Arzilli G, Privitera GP, Ferragina P, Tozzi AE et al (2023) ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front Public Health 11:1166120

    Article  PubMed  PubMed Central  Google Scholar 

  9. Riestra-Ayora J (2023) Answers of ChatGPT to questions about rhinology diseases [Internet] https://osf.io/dashboard

  10. Roxbury CR, Ishii M, Richmon JD, Blitz AM, Reh DD, Gallia GL (2016) Endonasal endoscopic surgery in the management of sinonasal and anterior skull base malignancies. Head Neck Pathol 10(1):13–22

    Article  PubMed  PubMed Central  Google Scholar 

  11. Valero A, Navarro AM, Del Cuvillo A, Alobid I, Benito JR, Colás C et al (2018) Position paper on nasal obstruction: evaluation and treatment. J Investig Allergol Clin Immunol 28(2):67–90

    Article  CAS  PubMed  Google Scholar 

  12. Fokkens WJ, Lund VJ, Hopkins C, Hellings PW, Kern R, Reitsma S et al (2020) Executive summary of EPOS 2020 including integrated care pathways. Rhinology 58(2):82–111

    Article  CAS  PubMed  Google Scholar 

  13. Rimmer J, Hellings P, Lund VJ, Alobid I, Beale T, Dassi C et al (2019) European position paper on diagnostic tools in rhinology. Rhinology 57(Suppl S28):1–41

    CAS  PubMed  Google Scholar 

  14. Charnock D, Shepperd S, Needham G, Gann R (1999) DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 3(2):105-111

    Article  Google Scholar 

  15. Temsah MH, Aljamaan F, Malki KH, Alhasan K, Altamimi I, Aljarbou R et al (2023) ChatGPT and the future of digital health: a study on healthcare workers’ perceptions and expectations. Healthcare (Switzerland) 11(13):1812

    Google Scholar 

  16. Frosolini A, Franz L, Benedetti S, Vaira LA, de Filippis C, Gennaro P et al (2023) Assessing the accuracy of ChatGPT references in head and neck and ENT disciplines. Eur Arch Otorhinolaryngol 280(11):5129-5133

    Article  PubMed  Google Scholar 

  17. Lechien JR, Maniaci A, Gengler I, Hans S, Chiesa-Estomba CM, Vaira LA (2023) Validity and reliability of an instrument evaluating the performance of intelligent chatbot: the Artificial Intelligence Performance Instrument (AIPI). Euro Arch Oto-Rhino-Laryngol. https://doi.org/10.1007/s00405-023-08219-y

    Article  Google Scholar 

Download references

Funding

This research did not receive any specific grant or funding.

Author information

Authors and Affiliations

  1. Department of Medicine, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, Villaviciosa de Odón, 28670, Madrid, Spain

    Juan Riestra-Ayora, Cristina Vaduva, Jonathan Esteban-Sánchez, Carolina Sánchez-Rodríguez & Eduardo Martin-Sanz

  2. Department of Otolaryngology-Head and Neck Surgery, Hospital Universitario de Getafe, Carretera de Toledo, Km 12.500, Getafe, 28905, Madrid, Spain

    Juan Riestra-Ayora, Cristina Vaduva, Jonathan Esteban-Sánchez, María Garrote-Garrote, Carlos Fernández-Navarro & Eduardo Martin-Sanz

Authors
  1. Juan Riestra-Ayora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cristina Vaduva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan Esteban-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. María Garrote-Garrote
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Fernández-Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carolina Sánchez-Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Eduardo Martin-Sanz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Riestra-Ayora.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest in this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Riestra-Ayora, J., Vaduva, C., Esteban-Sánchez, J. et al. ChatGPT as an information tool in rhinology. Can we trust each other today?. Eur Arch Otorhinolaryngol 281, 3253–3259 (2024). https://doi.org/10.1007/s00405-024-08581-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00405-024-08581-5

Keywords

Search

Navigation