Summary points
What was known before this research?
- •
The problem
The last decade has witnessed a substantial growth in the amount of medical data recorded for a given patient, along with an increasing pressure to improve the quality of healthcare and reduce medical errors. The problem-oriented, Electronic Health Record (EHR), centered on the problem list, is seen by many as a possible answer to these growing challenges. The problem list is a central place for clinicians to have a concise view of all of a patient's medical problems. The problem list also encourages an orderly process of clinical problem solving, prevents redundant actions [1], and supports the clear documentation of patient condition and clinical decision-making and improves communication among caregivers.
At Intermountain Health Care (IHC), a health maintenance organization serving Utah, the problem list is an important piece of the medical record, and a central component of the clinical information system called HELP2 [2]. To enable its potential benefits, the problem list has to be as accurate, complete and timely as possible. Unfortunately, problem lists are usually incomplete and inaccurate, and are often totally unused. To address this deficiency, we have created an application using Natural Language Processing (NLP) to harvest potential problem list entries from the multiple free-text electronic documents available in a patient's EHR [3], [4]. The medical problems identified are then proposed to the physicians for addition to the official problem list. This system, referred to as the Automatic Problem List (APL) system, is evaluated here. We hypothesize that the use of NLP to automatically provide potential medical problems will improve the completeness, accuracy and timeliness (decreased time between problems identification and their addition to the list) of this Automated Problem List.
The problem list in a Problem-Oriented Medical Record (POMR) was proposed more than three decades ago by Weed [5], [6] as an answer to the complexity of medical knowledge and clinical data, and to address weaknesses in the documentation of medical care. In recent years, the problem-oriented, Computer-based Patient Record (CPR) and the problem list have seen renewed interest as an organizational tool [1], [7], [8], [9], [10], [11], [12], [13]. Advantages to this approach are that the problem
As mentioned earlier, the Automated Problem List system extracts potential medical problems from free-text medical documents, and uses NLP to achieve this task. The two main components comprising the system are a background application for problem discovery and the problem list management application mentioned above. The background application is responsible for the text processing and analysis and stores extracted problems in the central clinical database. These problems can then be accessed
Ten different reviewers, all physicians, reviewed clinical documents to create the reference standard. Eight were board-certified physicians (most of them in internal medicine), and two were residents with at least 2 years of training. Each reviewer examined an average of 686 different documents with the web-based application described above. The time spent for a review was between 48 and 216 s per document. Reviewers’ overall agreement was very good, with a Finn's R of 0.897 when reviewing
This evaluation of our Automated Problem List system suggests that the addition of NLP to improve accuracy and timeliness of the problem list was successful. We measured a significantly increased sensitivity. Clearly, enhancing the problem list management application with NLP made the problem list more complete. We also measured a significantly improved timeliness of the problem list, with an average time difference between a medical problem's first mention in text and its addition to the
The Automated Problem List system that we developed to extract potential medical problems from free-text documents in a patient's EHR has shown satisfying results. This system's goal to improve the problem list's quality by increasing its completeness and timeliness was met, showing higher sensitivity and better timeliness in the intervention group. This was achieved only when the problem list was used. By encouraging the use of a problem list of better quality, this system could potentially
This work is supported by a Deseret Foundation Grant (Salt Lake City, Utah). We would like to thank Min Bowman for her help with the modified Problems module. We would also like to thank Greg Gurr for his advices and his help. Scott Narus and Stan Huff also gave us helpful advice and guidance for which we are grateful. Finally, we are especially grateful to Terry Clemmer whose enthusiasm for the problem list made this study possible. Summary points What was known before this research? The problem
Unfortunately, this is haphazardly performed, leading to inaccurate, incomplete and duplicative lists. Efforts to automate the creation of the problem list based on EHR data have been made, but were limited to 80 prespecified problems and utilized techniques that may not scale to all possible problems [10–12]. With the advent of advanced natural language processing (NLP) and machine learning techniques [13–15], it is possible for automatic creation and updating of a patients’ problem list based on the data in the EHR.
Limited research has been conducted on this topic. A clinical alerting system [15] and natural language processing [16] have been used for improving the completeness of problem lists. The support systems efficiently improve the completeness of problem lists; however, they can also be a source of tremendous clinical documentation errors, which may affect patient care [17,18].
Data extraction from electronic medical records requires less labor on the back end, but ultimately depends on reliable assessment and data input from clinical staff, which may not consistently occur. Electronic problems lists, for instance, are frequently inaccurate and incomplete [32]. In addition, the paucity of replication studies for these models, with the exception of Rudolph [25] developing an electronic version of Inouye's 1993 model, is an important limitation of this literature.
For example, a study of the Veteran's Administrations EMR [11] found that only 49% of patients with hypertension had the diagnosis on the problem list. Further, a study from Intermountain Healthcare reported that their problem lists are “usually incomplete and inaccurate, and are often totally unused” [12]. The completeness and accuracy of electronic problem lists can be improved using clinical decision support (CDS) to semi-automate the creation of problem list entries via indication based prescribing [13,14], or, as recently reported by Wright, using pop-up screens independent from orders [15].
This search led to a pruned ANN architecture which minimized the number of diagnostic factors used during the training phase and, therefore, also decreased the number of nodes in the ANN input and hidden layer, as well as in the Mean Square Error of the trained ANN at the testing phase. In Meystre and Haug (2008) an Automated Problem List system using Natural Language Processing (NLP) was developed to extract potential medical problems from free-text documents in a patient’s Electronic Health Record (EHR). The system significantly increased the sensitivity of the problem lists in the intensive care unit, from about 9–41%, and even 77% if problems were automatically proposed.