A framework for evaluating eHealth research
Introduction
Health care is in the midst of a consumer-oriented technology explosion. Individuals of all ages and backgrounds have discovered eHealth. For example, we can now gain access to our own medical records through web portals, “chat” online with others individually or in virtual communities, transmit health data using the telephone or wireless technologies, and surf the Internet to find information about health and health services. But the challenges of implementing and evaluating eHealth are just beginning to surface, and, as technology changes, new challenges emerge. Evaluation is critical to the future of eHealth. Unfortunately, few blueprints for effective evaluation methodologies come with this 21st century technology. This article addresses four dimensions of eHealth evaluation: (1) design and methodology issues; (2) challenges related to the technology itself; (3) environmental issues that are not specific to eHealth but pose special problems for eHealth researchers; and (4) logistic or administrative concerns of the evaluation methodology selected.
This article describes challenges associated with each dimension, using examples from actual eHealth interventions; proposes that thorough planning for each dimension is “necessary but not sufficient” for sound eHealth research; and suggests that the four dimensions must be integrated to provide a holistic framework for designing and implementing eHealth research projects, as well as for understanding the totality of the eHealth intervention. The framework must be flexible enough to adapt to a variety of end users, regardless of whether the end user is a healthcare organization, a for-profit business, a community organization, or an individual. The framework is depicted as a puzzle with four interlocking pieces (Fig. 1).
This article does not endorse specific designs, methods, or approaches for conducting eHealth research; these decisions should be dependent on the question being asked, the nature of the technology, and the specific needs of the organization. Nor does it address sustainability issues; these topics are beyond the scope of this document and have been addressed in the literature (Maheu, Whitten, & Allen, 2001). Rather, the contribution of this paper is to explore the challenges and opportunities in eHealth research from a holistic perspective, in order to advance our understanding of consumer technologies that may have an impact on health status.
Section snippets
The eHealth explosion
In The Wizard of Oz, Dorothy dreams that her farmhouse is tossed in the vortex of a tornado and lands squarely on the Wicked Witch. With the help of the Scarecrow, the Tin Man, and the Lion, she skips along the yellow brick road in search of answers from the Great Oz. The Land of Oz is a good metaphor for our healthcare system today. Like Dorothy's farmhouse, our healthcare system has been uprooted by advances in telecommunications and electronics, and keeping pace with these advances can be
eHealth research challenges
The explosion in eHealth use has generated a corresponding increase in the literature describing these technologies. A call for papers to the British Medical Journal on eHealth issues resulted in nearly 100 submissions—more than any submitted for a BMJ theme issue (Jadad & Delamothe, 2004). But the editors noted that many of the submissions were descriptive in nature or single-case studies. They question whether traditional research methods are adequate, even appropriate, to investigate eHealth
A call for innovative eHealth research
eHealth offers a new lens through which to view the delivery of health services, including preventive and acute care. But with this opportunity comes a challenge to lay the groundwork for a meaningful research agenda. We have presented an overview of eHealth research issues, drawing on our perspectives from two very different projects. Although the patient populations, technologies, and approaches differ, they share common concerns that can provide valuable lessons to others who are interested
Acknowledgments
The authors acknowledge, with appreciation and thanks, the financial support provided by the Health e-Technologies Initiative, and the assistance provided by the National Program Staff of the Robert Wood Johnson Foundation.
References (15)
- et al.
Self-care behaviors among patients with heart failure
Heart and Lung
(2002) - et al.
The Fun, Food and Fitness Project (FFFP): The Baylor GEMS pilot study
Ethnicity & Diseases
(2003) - Dansky, K.H. (2005). Technology enablers: Management practices that promote end-user acceptance of technology in health...
- et al.
Clinical outcomes of telehomecare for diabetic patients
Journal on Information Technology in Healthcare
(2003) eHealth research and evaluation: Challenges and opportunities
Journal of Health Communication
(2002)- et al.
The first generation of e-patients
British Medical Journal
(2004) Qualitative research and the profound grasp of the obvious
Health Services Research
(1998)
Cited by (56)
Revealing the role of intellectual capital in digitalized health networks. A meso‑level analysis for building and monitoring a KPI dashboard
2022, Technological Forecasting and Social ChangeCitation Excerpt :On the other hand, Pagliari (2007) stated that a multidisciplinary team of experts is irreplaceable in order to capture the potential of digital innovation. Dansky et al. (2006) argued that multidisciplinary elements and skills are aimed at developing, organizing, monitoring, and reshaping the digital healthcare system. Hamid and Skarmad (2008) shed light on the necessity to adopt a multiple perspective in the evaluation phase.
Dissemination and implementation science in program evaluation: A telemental health clinical consultation case example
2017, Evaluation and Program PlanningCitation Excerpt :For instance, logistical delays, as well as the need for additional training in using the technological equipment, can delay program start and may need to be attended to in order to secure quality data collection. For a good review on the topic, see Dansky, Thompson, and Sanner (2006), who outlined four main challenges to engaging in TMH research, many examples of which were relevant in the case of the current example: 1) design and methodology issues; 2) challenges related to the technology; 3) environmental issues not specific TMH but nonetheless relevant; and 4) logistical and administrative issues. Evaluators will need to be increasingly familiar with these challenges and potential methods to overcome such barriers when engaging in program evaluation efforts incorporating the use of technology.
Principles for the evaluation of telemedicine applications: Results of a systematic review and consensus process
2016, Zeitschrift fur Evidenz, Fortbildung und Qualitat im GesundheitswesenScheme of the Software Engineering Elements that Are Part of the Development of eHealth-Oriented Models and Frameworks
2024, Communications in Computer and Information ScienceFactors Affecting the Transition from Paper to Digital Data Collection for Mobile Tuberculosis Active Case Finding in Low Internet Access Settings in Pakistan
2022, Tropical Medicine and Infectious Disease