Dear Editor,
The National Academy of Medicine defines health literacy as the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions. Most patients use the Internet as their initial source of health information (Swoboda et al. 2018). Currently, there is an established body of misinformation surrounding coronavirus disease 2019 (COVID-19) (Ioannidis 2020). Therefore, it is paramount that online health resources for COVID-19 are at an appropriate reading level; we aimed to assess their readability.
In a cache-cleared and location-disabled web browser, “coronavirus information” was entered in three popular search engines (Google, Bing, Yahoo!) on March 24, 2020, using five geographically representative virtual private network (VPN) locations in the United States and Canada (San Francisco, Miami, New York, Vancouver, and Toronto). Only websites in the first three search engine pages of each unique VPN location were included, as people are unlikely to proceed past this point (Eysenbach and Kohler 2002). Public health, governmental (including regional and national), and foundational webpages which pertained to COVID-19 information for the general public were included. Webpages under 100 words, non-English articles, photos, videos, advertisements, news articles, and webpages not aimed at the general public (e.g., for healthcare professionals or businesses) were excluded. Each webpage was assessed by two independent reviewers. Any differences in data extraction were resolved by re-evaluation until consensus.
The American Medical Association (AMA) and National Institutes of Health (NIH) have both recommended that reading levels of online health material should be at or below a sixth-grade level to be comprehensible to the general public (Fahimuddin et al. 2019). Thus, five scores were used to estimate the grade level appropriate for reading each website: Flesch-Kincaid Grade Level (FKGL), Automated Readability Index (AMI), SMOG Index (SMOGI), Coleman-Liau Index (CLI), and Gunning Fog Score (GFS). The general grade level (GGL) was calculated by averaging these five scores. Flesch Reading Ease Score (FRES) applies a scale ranging from 0 to 100 with higher scores indicating easier readability (Abu-Heija et al. 2019). Last, websites were categorized based on reading level: a GGL < 7 was considered easy; between 7 and 10, intermediate; and > 10, hard.
Of 428 webpages, 371 were excluded (330 were duplicates, 11 were < 100 words, 19 were not for the general public, and 11 were news articles). The mean GGL of the remaining 57 included webpages was 10.8. By stratifying readability of webpages with GGL, 70% were considered hard and 30% intermediate (Table 1). Based on FRES, 82% were considered hard to read. None of the webpages were considered easy to read based on either GGL or FRES.
In conclusion, our results show that the readability of online resources for COVID-19 exceeds thresholds previously set by the AMA and NIH. Readability may be improved by limiting sentences to 8–10 words and using simple words, instead of medical terminology (Boztas et al. 2017). We recommend that both government and health information websites re-evaluate their online resources to match an appropriate reading level for the general public.
References
Abu-Heija, A. A., Shatta, M., Ajam, M., Abu-Heija, U., Imran, N., & Levine, D. (2019). Quantitative readability assessment of the internal medicine online patient information on Annals.org. Cureus, 11(3), e4184. https://doi.org/10.7759/cureus.4184.
Boztas, N., Omur, D., Ozbılgın, S., Altuntas, G., Piskin, E., Ozkardesler, S., & Hanci, V. (2017). Readability of internet-sourced patient education material related to "labour analgesia". Medicine (Baltimore), 96(45), e8526. https://doi.org/10.1097/MD.0000000000008526.
Eysenbach, G., & Köhler, C. (2002). How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324(7337), 573–577. https://doi.org/10.1136/bmj.324.7337.573.
Fahimuddin, F. Z., Sidhu, S., & Agrawal, A. (2019). Reading level of online patient education materials from major obstetrics and gynecology societies. Obstet Gynecol, 133(5), 987–993. https://doi.org/10.1097/AOG.0000000000003214.
Ioannidis, J. P. A. (2020). Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures. Eur J Clin Invest, e13223. https://doi.org/10.1111/eci.13223.
Swoboda, C. M., Van Hulle, J. M., McAlearney, A. S., & Huerta, T. R. (2018). Odds of talking to healthcare providers as the initial source of healthcare information: updated cross-sectional results from the health information National Trends Survey (HINTS). BMC Fam Pract, 19(1), 146. https://doi.org/10.1186/s12875-018-0805-7.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
The authors have not published, posted, or submitted any related papers from the same study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Treanor, L., Radonjic, A. Assessing the reading level of online resources on COVID-19. Can J Public Health 111, 484–485 (2020). https://doi.org/10.17269/s41997-020-00363-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.17269/s41997-020-00363-w