Skip to main content
SpringerLink
Log in

Objectively Assessing and Comparing the User Experience of Two Thousand Digital Health Apps

  • Conference paper
  • First Online:
Design, User Experience, and Usability (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14034))

Included in the following conference series:

Abstract

There are more than 350,000 digital health apps in the app stores today. There is an average of around 250 digital health apps being added daily to the app stores. This indicates a public appetite to consume digital health apps, and a potential to decrease the pressure on healthcare providers by promoting digital technology. However, for this to be a viable option, digital health apps must be effective and safe to use. The quality, defined as “compliance with best practice standards”, of such apps must be of the highest standard for these to be recommended by healthcare professionals. One crucially important aspect of the quality of an app is user experience (UX). The UX of digital health apps is important to assure that the technology is being used safely and as intended. The objective of this study was to describe common practices related to the UX for digital health app design. This study analysed a sample of 2,053 digital health apps with a focus on the UX practices in the digital health app design. The data included in this study was collected using the Organisation for the Review of Care and Health Applications (ORCHA) assessment tool. ORCHA is a United Kingdom (UK) based digital health compliance company that specialises in assessing the quality of digital health apps, which includes UX. The ORCHA UX assessment consists of 15 polar questions (Yes/No) and 3 multiple selection questions. A score of 65 is considered a ‘threshold score’ and a starting point, based on answers this score will increase, decrease, or stay the same.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Data Sharing

This study conducted secondary data analysis on the ORCHA dataset (Excel format). The data from this dataset (and for more DHIs) is freely available to registered users (registration is free) at ORCHA library [17]. However, the data will be presented for individual DHIs and not in Excel format.

References

  1. Kern, J., et al.: Written consent of IQVIA and the IQVIA Institute. Digital Health Trends (2021)

    Google Scholar 

  2. ORCHA. COVID-19: Digital Health Trends Report - ORCHA (2020). https://orchahealth.com/covid19-digital-health-trends-report/

  3. Alqahtani, F., Orji, R.: Insights from user reviews to improve mental health apps. Health Informatics J. 26(3), 2042–2066 (2020). https://doi.org/10.1177/1460458219896492

    Article  Google Scholar 

  4. Hunt Sophie. Review Documentation - Review Development & Resources | Exte. Accessed 13 Mar 2022. https://confluence.external-share.com/content/b6055aac-83e4-4947-be0e-ebb8c39559ef

  5. Brooke, J.: (PDF) SUS: A quick and dirty usability scale. ResearchGate (1995). https://www.researchgate.net/publication/228593520_SUS_A_quick_and_dirty_usability_scale

  6. Richardson, B., Campbell-Yeo, M., Smit, M.: Mobile application user experience checklist: a tool to assess attention to core UX principles. Int. J. Human- Comput. Interact. 37(13), 1283–1290 (2021). https://doi.org/10.1080/10447318.2021.1876361

    Article  Google Scholar 

  7. ORCHA Privacy Policy. Accessed 11 Aug 2022. https://appfinder.orcha.co.uk/privacy-policy/

  8. Torous, J., et al.: Towards a consensus around standards for smartphone apps and digital mental health. World Psychiat. 18(1), 97 (2019). https://doi.org/10.1002/WPS.20592

    Article  Google Scholar 

  9. ISO - ISO/TS 82304-2:2021 - Health software—Part 2: Health and wellness apps—Quality and reliability. Accessed 7 Feb 2023. https://www.iso.org/standard/78182.html

  10. Hensher, M., et al.: Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers. J. Am. Med. Informat. Assoc. 28(6), 1318–1329 (2021). https://doi.org/10.1093/JAMIA/OCAB041

    Article  Google Scholar 

  11. Henson, P., David, G., Albright, K., Torous, J.: Deriving a practical frame-work for the evaluation of health apps. Lancet Dig/ Health 1(2), e52–e54 (2019). https://doi.org/10.1016/S2589-7500(19)30013-5

    Article  Google Scholar 

  12. Lagan, S., Sandler, L., Torous, J.: Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps. BMJ Open 11, 47001 (2021). https://doi.org/10.1136/bmjopen-2020-047001

    Article  Google Scholar 

  13. Torous, J.B., et al.: A hierarchical framework for evaluation and in-formed decision making regarding smartphone apps for clinical care. Psychiat. Serv. 69(5), 498–500 (2018). https://doi.org/10.1176/APPI.PS.201700423

    Article  Google Scholar 

  14. Hyzy, M., et al.: System usability scale benchmarking for digital health apps: meta-analysis. JMIR Mhealth Uhealth 10(8), E37290 (2022). https://Mhealth.Jmir.Org/2022/8/E37290. https://doi.org/10.2196/37290

  15. Patel, B., Thind, A.: Usability of mobile health apps for postoperative care: systematic review. JMIR Perioper Med 3(2), E19099 (2020). https://Periop.Jmir.Org/2020/2/E19099. https://doi.org/10.2196/19099

  16. Biduski, D., Bellei, E.A., Rodriguez, J.P.M., Zaina, L.A.M., de Marchi, A.C.B.: Assessing long-term user experience on a mobile health application through an in-app embedded conversation-based questionnaire. Comput. Hum. Behav. 104, 106169 (2020). https://doi.org/10.1016/J.CHB.2019.106169

    Article  Google Scholar 

  17. Digital Health Libraries - ORCHA. Accessed 19 Jan 2023. https://orchahealth.com/services/digital-health-libraries/

Download references

Acknowledgements

This research is done in partnership with ORCHA, a UK-based digital health compliance company. This work is supported by a Northern Ireland DfE CAST award / PhD scholarship. We would like to acknowledge the contribution of the many digital health app reviewers and developers who worked with ORCHA that allowed for the review of digital health apps and consented for their data to be used for the purposes of research. Without their contribution and consent this research would not have been possible.

Author information

Authors and Affiliations

  1. School of Computing, Ulster University, York St, Belfast, BT15 1ED, UK

    Maciej Hyzy, Raymond Bond, Maurice D. Mulvenna & Lu Bai

  2. ORCHA, Sci-Tech Daresbury, Violet V2, Keckwick Lane, Daresbury, WA4 4AB, UK

    Maciej Hyzy, Robert Daly & Simon Leigh

  3. Warwick Medical School, University of Warwick, Coventry, CV4 7AL, UK

    Simon Leigh

Authors
  1. Maciej Hyzy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond Bond
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maurice D. Mulvenna
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lu Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert Daly
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Simon Leigh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maciej Hyzy .

Editor information

Editors and Affiliations

  1. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  2. World Usability Day and Bubble Mountain Consulting, Newton Center, MA, USA

    Elizabeth Rosenzweig

  3. Southern University of Science and Technology – SUSTech, Shenzhen, China

    Marcelo M. Soares

Ethics declarations

This study is funded by a DfE Cast award and ORCHA. Simon Leigh and Robert Daly are employees at ORCHA.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hyzy, M., Bond, R., Mulvenna, M.D., Bai, L., Daly, R., Leigh, S. (2023). Objectively Assessing and Comparing the User Experience of Two Thousand Digital Health Apps. In: Marcus, A., Rosenzweig, E., Soares, M.M. (eds) Design, User Experience, and Usability. HCII 2023. Lecture Notes in Computer Science, vol 14034. Springer, Cham. https://doi.org/10.1007/978-3-031-35705-3_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35705-3_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35704-6

  • Online ISBN: 978-3-031-35705-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation