What can be done to increase the use of diagnostic decision support systems?

Introduction

With the increased interest in preventing diagnostic errors, the use of diagnostic decision support systems (DDSS) seems an obvious solution to the problem of diagnostic errors. The use of clinical decision support systems is part of the criteria for receiving incentives under the meaningful use regulation, which provides an additional reason for health care systems to implement them [1]. Yet despite the potential value of DDSS in assisting with diagnoses, over the decades that DDSS have been available, they have not been widely used [2–5]. This essay will examine the factors that could lead to lack of use and will propose some potential solutions. Some of the ideas have been previously discussed by the author in other publications and presentations [6–10], but they are integrated in this essay and supplemented with support from the recent research literature.

DDSS have traditionally been used when a clinician experiences a puzzling diagnostic problem and seeks a consultation from the system. Usually the physician needs to enter relevant patient data into the system and the DDSS will use that patient information, compare the data to its knowledge base and present the physician with a list of potential diagnoses. The physician then reviews the list for relevance to the case at hand.

Although different DDSS use different knowledge sources, algorithms, and user interfaces, the following features are common to all DDSS: (1) the physician experiences a diagnostic puzzle that is sufficient to motivate consultation with the DDSS, (2) the system provides suggestions of possible diagnoses, and (3) the physician evaluates the suggestions. It is the author’s contention that each of these features common to DDSS (and certainly the combination) can lead to underuse of DDSS, but that they also provide a framework for better design of systems to promote optimal use.

Physician perception of diagnostic puzzles

For physicians to be motivated to seek out the assistance of DDSS, they need to feel that they are faced with a case for which they either do not have, or are not sure of, the answer. The assumption that such uncertainty happens with every patient is clearly not an accurate picture of medical practice. Although it is difficult to find good empirical support, what is known as the 80–20 rule has been said to apply to the cases an experienced physician sees in his or her practice. That is, about 80% of the patients have conditions that are fairly easily diagnosed. For this large majority of cases, not only are DDSS not needed, using them would be an inefficient use of the physician’s time and might even lead to expensive and unnecessary additional diagnostic testing. However, it is a challenge to determine which are the truly easy to diagnose cases and which only seem that way.

Of the remaining patients, those that the physician recognizes as having puzzling presentations would be an appropriate use of DDSS, but often the physician uses other means to deal with them, such as ordering additional tests, consulting experts or seeking out more information. The other patient cases, which look like routine cases, but are not, are probably the ones most likely to be misdiagnosed [11, 12], but because these appear to the physician to be like the easy-to-diagnose cases, the clinician may not recognize their difficulty initially, leading to inappropriate treatment and delay. In fact, often the physician will work by trial and error and if the patient gets better after the second or third trial of a diagnosis/treatment, the previous trials may not even be perceived as errors. Thus, it is understandable that many physicians do not perceive a frequent need for DDSS based on their own experience. Even though there are cases that would be appropriate for using a DDSS (either because they are perceived as challenging or because the physician erroneously thinks they are easy), the treating physician may not perceive the need to use the system and it is this perception, rather than the actual need, that will determine use of the DDSS.

One might expect that the publicity around the extent of diagnostic errors would lead physicians to use DDSS, even if their own experience did not prompt such use. However, the data on the extent of diagnostic errors [9], while greatly concerning to those who have experienced the errors and alarming to the public, have not led physicians to embrace DDSS to any great degree. In part this may be because the data are discrepant from the physicians’ experiences, especially because the studies on errors use errors per the number of patients over a particular time interval to calculate the error rate, while the physician uses errors over the total number of encounters over a lifetime to assess his/her own sense of the extent of errors. And by that calculation, the physician’s own error rate can seem miniscule and may not be worth the effort to learn to use DDSS for what seems like such a small problem.

For the above reasons, even physicians who are very good at recognizing what they know and do not know could tend to see little need for DDSS. However, we also know that many physicians are not well-calibrated in regard to confidence and accuracy of their diagnoses, meaning that they often think they are correct when they are not. Podbregar et al. found that physician confidence in their clinical diagnoses was unrelated to their accuracy as shown on autopsy [13]. Friedman et al. examined physician confidence when using diagnostic decision support systems and also found frequent overconfidence [14]. In a recent study, Meyer et al. found that physicians were almost as confident about their diagnoses on difficult cases as they were with easy ones, but were much less accurate. On these difficult cases the physicians were also not interested in seeking additional information to assist them [11]. If one does not perceive that a case is a diagnostic challenge and if one is confident in one’s diagnosis, there is little motivation to seek out the assistance of DDSS.

The cognitive biases that influence diagnoses – in particular, premature closure, availability bias and confirmation bias – have been well-documented [15–17]. These biases as well as other heuristics are difficult to eradicate because much of the time they are effective [18]. That is, the typical approach of generating and confirming diagnoses fairly rapidly, will, most of the time, as described above, yield a correct diagnosis. Wears and Nemeth have commented that errors, diagnostic and otherwise, are often obvious only in hindsight [19]. In fact, if a physician arrives at a diagnosis quickly with only a few pieces of the potentially available patient data, if the diagnosis is correct, the physician is hailed as a great diagnostician, but if it is wrong, it is considered premature closure. The problem is at the time of the initial diagnostic formulation it is difficult to tell which is the case and unfortunately, feedback on patient outcomes, which could prompt re-examination of the problem, is rare [20, 21].

If there were no time pressures, perhaps, even if they do not think there is a problem, physicians might use diagnostic decision support systems to provide suggestions for “what else it could be,” but given the time pressures common in clinical care, and the lack of a perceived need, exhortations to reconsider one’s initial thinking or to use DDSS often fail to elicit changes in behavior.

Design of DDSS and physician evaluation of suggestions

One might argue that rather than requiring physicians to seek out the advice of a DDSS one could design DDSS to push the diagnostic advice to the physician. Currently most DDSS require, at least to some extent, that physicians seek out the advice. Research on the use of other types of clinical decision support where advice is pushed, such as the drug interaction programs, has shown that a phenomenon known as alert fatigue often occurs [22]. Alert fatigue occurs when physicians get so many alerts that are not valid that they begin to ignore all of the alerts, even the important ones. There is often information that a physician knows about the patient that could potentially rule out some diagnoses but this information may not be able to be accommodated by the DDSS. For this reason the DDSS suggestions usually aim to be sensitive and comprehensive, rather than highly specific, relying on the physician to ignore the obviously wrong suggestions.

The problem of lack of specificity has been shown in many DDSS where the system provides a large number of suggestions, many of which are either blatantly incorrect or are so obvious that the physician has already thought of them (and either dismissed them, or is already working them up). In fact, it is because “common diseases are common”, that students are advised “when you hear hoofbeats, think horses not zebras.” Like a good medical student, a good DDSS will also list the common diseases first, but the impact of this may be that the more obscure diseases will be very far down the list of potential diagnoses. This position will make it likely that they will be dismissed or the busy physician will not take the time to review many either obvious or extremely unlikely diseases to find the buried valuable suggestion.

In addition, if the DDSS are stand-alone systems, there is a need to enter the patient data into the DDSS and separately into the electronic health record (EHR). Fortunately, systems are becoming more integrated into electronic health record systems, but because the DDSS vocabulary might not match the EHR vocabulary there is likely to be at least some additional data entry, even if some duplication can be eliminated. Thus, the extra time for data entry coupled with the time needed to review a lengthy list of suggestions, many of which are perceived as not relevant, lead to a lack of motivation to use the DDSS even if a diagnostic puzzle is perceived.

To summarize the discussion thus far, for a variety of reasons (1) the average physician does not feel the need to seek out the advice of a DDSS on a routine basis; (2) time and cost pressures as well as the perception that the DDSS has a low likelihood of suggesting something that would make a major difference in treatment, lead to low use of DDSS; and (3) in ambulatory care there are rarely feedback systems in place that can compensate for an initially incorrect diagnosis and prompt DDSS consultation. Yet the problem of diagnostic error continues to need addressing. In the next section the author will provide some suggestions on what is needed.

Suggestions

The first suggestion is to develop systems that are better integrated into the EHR rather than relying on double data entry of diagnostically relevant data. As we move toward standards for representing clinical data and have more structured clinical notes and reports, it may be easier to have DDSS operate in the background and use the data from the EHR to formulate initial diagnostic hypotheses. The diagnoses can be updated as more data are accumulated and the DDSS should be accessible to the user at any time, should the user desire consultation.

Second, rather than waiting for the physician to perceive uncertainty and ask for information, the DDSS should also provide unsolicited suggestions, but only at certain times. Those suggestions should only be provided if it clear from the diagnostic work-up and orders that the physician is going in a direction where a potentially likely diagnosis will be missed. As an example, if the same laboratory test or procedure that the physician is ordering for one diagnosis would also discover another diagnosis that the DDSS found, but the physician did not recognize, it may not be necessary to remind the physician of the other diagnosis, since it will be obvious in the test results. However, if a test were needed to confirm or rule out a key diagnosis that the DDSS had as high priority and the physician did not order that test, only then should the DDSS alert the physician to the other possible diagnosis. What this means is DDSS need to include both diagnoses and work-up strategies in their knowledge bases, DDSS need to interact with the CPOE system, and the alerts should be provided just before work-up orders are finalized. This very targeted advice at the time of ordering would be a departure from the way we typically think of the use of DDSS.

In addition, the DDSS should be refined to be more specific than most DDSS in the past have been, or at least display their results in ways that make it easier for physicians to review and assess the diagnostic suggestions. Coupled with the above suggestion for limited and targeted diagnostic advice, this may be easier in the future as there is more knowledge of, and interest in, usability of health information technology [23]. Such an approach would provide, as Osheroff and his colleagues have advocated, the right information in the right format at the right time [24] and would most likely avoid the problem of alerting unnecessarily.

Finally, as has been advocated by several sources, there needs to be more automated follow-up and feedback to physicians [25, 26]. The right type of system would have user-friendly ways of documenting a follow-up interval when the patient should have improved if the diagnosis and treatment were appropriate. The designated time to follow-up would trigger automated ways of contacting the patient, checking on outcomes, and alerting the physician if the patient were not improving as expected. Not only should the physician be alerted if there is no improvement, but there should be an automated link to the DDSS or other resources to provide alternative diagnostic hypotheses.

If such a system were able to be implemented it would provide a mechanism for modifying diagnoses and therapy in a timely manner, it could mitigate the harm to patients from diagnostic errors, and it might even help physicians become better calibrated, increase the physician’s knowledge and lead to more reflective practice. There would still be problems that might slip through such a system, but hopefully they will be few and with the feedback system in place they should be able to be caught early.

Lobach, in a recent editorial, has said that we still do not really know how to effectively deploy clinical decision support systems [27]. Diagnostic decision support systems have had even less research done on them than therapeutic systems and are less frequently used in practice. Given the greater recognition of the problem of diagnostic errors, it is time to re-examine the role diagnostic decision support systems can play in reducing diagnostic errors, how the systems should be designed, and what type of research is needed to determine if we have succeeded.