Leveraging user's performance in reporting patient safety events by utilizing text prediction in narrative data entry
Introduction
Many attempts have been made to investigate the difficulties in clinical data entry in order to promote the acceptance and quality-in-use of clinical information systems [1], [2], [3], [4]. With the advance of non-narrative entry templates and natural language processing techniques, the application of structured entries in clinical information systems is increasing because of the merit of interoperability and reuse. However, given the complexity of healthcare, failure to include essential fields and lack of options in structure entries are still common. Structured entries also received critiques due to lack of flexibility and expressiveness in clinical communication. Therefore, unstructured narrative data entry plays an indispensable role in clinical data entry.
For patient safety event reporting, narrative entry has been a prevalent and dominant format. However, it is now in a transition of becoming a supplementary format to non-narrative forms released by national and international organizations [5], [6]. Narrative comment field is intended to collect case details beyond structured entries. Previous studies showed that voluntary reporters usually ignored this field or described events with inaccurate and incomplete terms and sentences [7], [8]. Two major barriers are identified due to the multitasking and busy nature of healthcare and the lack of languages or knowledge for reporters to describe patient safety events in detail [9]. To help remove the barriers and achieve quality-in-use, it requires a user-centered design process which embraces cultural, strategic and technical considerations [10]. We proposed to examine the utilization of text prediction applied in the narrative fields in reporting patient safety events.
Text prediction, also known as word, sentence or context prediction originated in augmentative and alternative communication (AAC) to increase text generation rates for people with the disabilities of motor or speech impairment [11], [12]. The advance of natural language processing techniques has brought text prediction into a broader scope of daily computing activities, such as mobile computing [13] and radiography reports [14]. Studies on optimization [15] and impact [16] of text prediction have received much attention. This study focusing on the impact evaluation, would respond three basic concerns regarding text prediction in reporting patient safety events. These concerns include whether text prediction would increase (1) reporter's engagement in narrative comment field; (2) quality of narrative entry, which is highly valuable in generating actionable knowledge yet received little attention; (3) efficiency of reporting patient safety events, which remains unclear based on mixed results across the fields [16]. In this study, we employed a two-group randomized experiment to examine the impact of text prediction in the three aspects.
Section snippets
Background
This study is grounded in a user-centered design of patient safety event reporting systems. Reporting systems in use show the problems of underreporting [17] and low quality of reports [8], [18], though efforts from all levels are made to improve the systems [19], [20]. Prior to this study, we have consecutively conducted heuristic evaluation, cognitive task analysis and think-aloud user testing [4], [7], [21], [22] which revealed interface representational issues and identified that the
Participants
Potential candidates who were nurses and experienced in reporting and analyzing patient safety events in Tianjin First Central Hospital (TFCH) in Tianjin, China were identified and invited to participate in the study. Two candidates were on leave of absence during the study period, and three candidates felt not confident with operating computers. As a result, the remaining 52 nurses from 21 clinical departments enrolled in the study. All nurses were female between the ages of 30 and 52 years.
Results
All participants, with each reported five cases, successfully completed the experiment and thus generated 260 reports. There were 25 participants allocated in the control and 27 in the in the treatment group, accounting for 125 and 135 reports, respectively. The means of participants' ages were 43.6 ± 5.8 versus 41.1 ± 6.6 years in the control and treatment group. The 260 reports contained 2849 MCQ answers and 238 unstructured narrative comments. The completion times of MCQs were 131.0 ± 50.0
Discussion
Clinicians working under time constraints are usually expected to enter data for documentation in a timely and efficient manner [31], [32]. The comprehensiveness of entered data is critical to clinicians' decision-making and creation of actionable knowledge. To accommodate the expectation in patient safety event reporting, we introduced two text prediction functions of CL and AS, which are attached to narrative comment field pervasively used in patient safety reporting systems. A two-group
Conclusion
Text data entry, as an indispensable format in clinical information systems, was demonstrated to be enhanced by the text prediction functions in terms of efficiency, data quality and engagement in a two-group randomized experiment. This study disclosed the necessity of the text prediction functions in facilitating experienced domain users in text data entry when reporting patient safety events. The quality of event reporting plays a key role in learning from the events. Simply counting numbers
Authors' contributions
Yang Gong: Responsible for study design, data collection and analysis, and coauthored the manuscript with review and revision responsibilities.
Lei Hua: Responsible for the conceptualization, study design, prototype development, testing session observation, data collection and analysis, coauthored the manuscript.
Shen Wang: Organized and coordinated with the study participants, assisted in preparing testing cases, data analysis and had review responsibilities.
Conflicts of interest
We declare there are no conflicts of interest involved in the research.
Summary table
Known on the topic
- •
Text data entry is indispensable toward the quality care delivery but suffered from low quality and user's engagement.
- •
Text prediction is widely used in mobile computing.
Our contribution
- •
Text prediction can increase the efficiency, data quality and user's engagement of text data entry if the functions were properly designed.
- •
Text prediction would be still helpful for experienced users to enhance their performance.
- •
An initial investigation about the text prediction functions in
Acknowledgments
The authors express heartfelt thanks to Xiao Liu, RN and Ying Sun, RN at the Office of Nursing at Tianjin First Central Hospital in China who made numerous efforts helping review and edit the testing cases, derived proper text predictor responses, scheduled the testing sessions, and graded the subjects' commentary data. This project was in part supported by a grant on patient safety from the University of Texas System (grant number #156374) and a grant from AHRQ (grant number 1R01HS022895).
References (39)
- et al.
Results of a survey on medical error reporting systems in Korean hospitals
Int. J. Med. Inform
(2006) - et al.
Properties of permuted-block randomization in clinical trials
Control. Clin. Trials
(1988) - et al.
The impact of electronic health records on time efficiency of physicians and nurses: a systematic review
J. Am. Med. Inform. Assoc
(2005) The barriers to electronic medical record systems and how to overcome them
J. Am. Med. Inform. Assoc
(1997)Reducing barriers to physician data entry for computer-based patient records
Top. Health Inf. Manage
(1994)The clinician's perspective on electronic health records and how they can affect patient care
BMJ
(2004)- et al.
Text prediction on structured data entry in healthcare: a two-group randomized usability study measuring the prediction impact on user performance
Appl. Clin. Inform
(2014) Users' Guide AHRQ Common Formats for Patient Safety Organizations
(2008)Towards an international classification for patient safety: the conceptual framework
- et al.
Design of a user-centered voluntary patient safety reporting system: understanding the time and response variances by retrospective think-aloud protocols
Stud. Health Technol. Inform
(2013)
Data consistency in a voluntary medical incident reporting system
J. Med. Syst
A review of medical error reporting system design considerations and a proposed cross-level systems research framework
Hum. Factors
Sources of Power: How People Make Decisions
Augmentative and alternative communication
Text prediction systems: a survey
Univ. Access Inf. Soc
Text entry for mobile computing: models and methods, theory and practice
Human-Computer Interaction
Informatics in radiology (infoRAD): radiology report entry with automatic phrase completion driven by language modeling
Radiographics
Improving word prediction using Markov models and Heuristic methods
Augment. Altern. Commun
Learning and performance of able-bodied individuals using scanning systems with and without word prediction
Assist. Technol
Cited by (6)
Challenges and Opportunities of Patient Safety Event Reporting
2022, Studies in Health Technology and InformaticsRoutine Health Information System (RHIS) improvements for strengthened health system management
2020, Cochrane Database of Systematic ReviewsWords prediction based on N-gram model for free-text entry in electronic health records
2019, Health Information Science and SystemsAn information enhanced framework for reporting medication events
2018, Studies in Health Technology and InformaticsEnhancing patient safety event reporting: A systematic review of system design features
2017, Applied Clinical InformaticsClinical Text Prediction with Numerically Grounded Conditional Language Models
2016, EMNLP 2016 - 7th International Workshop on Health Text Mining and Information Analysis, LOUHI 2016 - Proceedings of the Workshop