Abstract
Objectives
To test a structured electronic health record (EHR) case review process to identify diagnostic errors (DE) and diagnostic process failures (DPFs) in acute care.
Methods
We adapted validated tools (Safer Dx, Diagnostic Error Evaluation Research [DEER] Taxonomy) to assess the diagnostic process during the hospital encounter and categorized 13 postulated e-triggers. We created two test cohorts of all preventable cases (n=28) and an equal number of randomly sampled non-preventable cases (n=28) from 365 adult general medicine patients who expired and underwent our institution’s mortality case review process. After excluding patients with a length of stay of more than one month, each case was reviewed by two blinded clinicians trained in our process and by an expert panel. Inter-rater reliability was assessed. We compared the frequency of DE contributing to death in both cohorts, as well as mean DPFs and e-triggers for DE positive and negative cases within each cohort.
Results
Twenty-seven (96.4%) preventable and 24 (85.7%) non-preventable cases underwent our review process. Inter-rater reliability was moderate between individual reviewers (Cohen’s kappa 0.41) and substantial with the expert panel (Cohen’s kappa 0.74). The frequency of DE contributing to death was significantly higher for the preventable compared to the non-preventable cohort (56% vs. 17%, OR 6.25 [1.68, 23.27], p<0.01). Mean DPFs and e-triggers were significantly and non-significantly higher for DE positive compared to DE negative cases in each cohort, respectively.
Conclusions
We observed substantial agreement among final consensus and expert panel reviews using our structured EHR case review process. DEs contributing to death associated with DPFs were identified in institutionally designated preventable and non-preventable cases. While e-triggers may be useful for discriminating DE positive from DE negative cases, larger studies are required for validation. Our approach has potential to augment institutional mortality case review processes with respect to DE surveillance.
Funding source: Agency for Healthcare Research and Quality
Award Identifier / Grant number: R18-HS026613
-
Research funding: This study was funded by the Agency for Healthcare Research and Quality (AHRQ) R18-HS026613.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: Authors state no conflict of interest.
-
Informed consent: A waiver of informed consent was approved for retrospective review of electronic health record.
-
Ethical approval: The local Institutional Review Board approved this study.
References
1. Patient Safety Network. Patient safety 101. Agency for healthcare research and quality. 2019. Available from: https://psnet.ahrq.gov/primer/patient-safety-101.Search in Google Scholar
2. Shojania, KG, Burton, EC, McDonald, KM, Goldman, L. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA 2003;289:2849–56. https://doi.org/10.1001/jama.289.21.2849.Search in Google Scholar PubMed
3. Singh, H, Sittig, DF. Advancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework. BMJ Qual Saf 2015;24:103–10. https://doi.org/10.1136/bmjqs-2014-003675.Search in Google Scholar PubMed PubMed Central
4. Raffel, KE, Kantor, MA, Barish, P, Esmaili, A, Lim, H, XueF, et al.. Prevalence and characterisation of diagnostic error among 7-day all-cause hospital medicine readmissions: a retrospective cohort study. BMJ Qual Saf 2020. https://doi.org/10.1136/bmjqs-2020-010896.Search in Google Scholar PubMed
5. Graber, ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22:ii21–7. https://doi.org/10.1136/bmjqs-2012-001615.Search in Google Scholar PubMed PubMed Central
6. Singh, H, Meyer, AN, Thomas, EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf 2014;23:727–31. https://doi.org/10.1136/bmjqs-2013-002627.Search in Google Scholar PubMed PubMed Central
7. Singh, H, Thomas, EJ, Wilson, L, Kelly, PA, Pietz, K, Elkeeb, D, et al.. Errors of diagnosis in pediatric practice: a multisite survey. Pediatrics 2010;126:70–9. https://doi.org/10.1542/peds.2009-3218.Search in Google Scholar PubMed PubMed Central
8. Medford-Davis, L, Park, E, Shlamovitz, G, Suliburk, J, Meyer, AN, Singh, H. Diagnostic errors related to acute abdominal pain in the emergency department. Emerg Med J 2016;33:253–9. https://doi.org/10.1136/emermed-2015-204754.Search in Google Scholar PubMed
9. Hautz, WE, Kämmer, JE, Hautz, SC, Sauter, TC, Zwann, L, Maier, V, et al.. Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scand J Trauma Resuscitation Emerg Med 2019;27:54. https://doi.org/10.1186/s13049-019-0629-z.Search in Google Scholar PubMed PubMed Central
10. Singh, H, Schiff, GD, Graber, ML, Onakpoya, I, Thompson, MJ. The global burden of diagnostic errors in primary care. BMJ Qual Saf 2017;26:484–94. https://doi.org/10.1136/bmjqs-2016-005401.Search in Google Scholar PubMed PubMed Central
11. Perrem, LM, Fanshawe, TR, Sharif, F, Plüddemann, A, O’Neill, MB. A national physician survey of diagnostic error in paediatrics. Eur J Pediatr 2016;175:1387–92. https://doi.org/10.1007/s00431-016-2772-0.Search in Google Scholar PubMed
12. Hussain, F, Cooper, A, Carson-Stevens, A, Donaldson, L, Hibbert, P, Hughes, T, et al.. Diagnostic error in the emergency department: learning from national patient safety incident report analysis. BMC Emerg Med 2019;19:77. https://doi.org/10.1186/s12873-019-0289-3.Search in Google Scholar PubMed PubMed Central
13. Singh, H, Giardina, TD, Meyer, AN, Forjuoh, SN, Reis, MD, Thomas, EJ. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med 2013;173:418–25. https://doi.org/10.1001/jamainternmed.2013.2777.Search in Google Scholar PubMed PubMed Central
14. Gunderson, CG, Bilan, VP, Holleck, JL, Nickerson, P, Cherry, BM, Chui, P, et al.. Prevalence of harmful diagnostic errors in hospitalised adults: a systematic review and meta-analysis. BMJ Qual Saf 2020;29:1008–18. https://doi.org/10.1136/bmjqs-2019-010822.Search in Google Scholar PubMed
15. Bergl, PA, Taneja, A, El-Kareh, R, Singh, H, Nanchal, RS. Frequency, risk factors, causes, and consequences of diagnostic errors in critically ill medical patients: a retrospective cohort study. Crit Care Med 2019;47:e902–10. https://doi.org/10.1097/ccm.0000000000003976.Search in Google Scholar PubMed
16. Zwaan, L, Singh, H. Diagnostic error in hospitals: finding forests not just the big trees. BMJ Qual Saf 2020;12:961–4. https://doi.org/10.1136/bmjqs-2020-011099.Search in Google Scholar PubMed
17. Liberman, AL, Newman-Toker, DE. Symptom-disease pair analysis of diagnostic error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data. BMJ Qual Saf 2018;27:557–66. https://doi.org/10.1136/bmjqs-2017-007032.Search in Google Scholar PubMed PubMed Central
18. Thammasitboon, S, Cutrer, WB. Diagnostic decision-making and strategies to improve diagnosis. Curr Probl Pediatr Adolesc Health Care 2013;43:232–41. https://doi.org/10.1016/j.cppeds.2013.07.003.Search in Google Scholar PubMed
19. Cifra, CL, Custer, JW, Singh, H, Fackler, JC. Diagnostic errors in pediatric critical care: a systematic review. Pediatr Crit Care Med 2021;22:701–12. https://doi.org/10.1097/pcc.0000000000002735.Search in Google Scholar PubMed PubMed Central
20. Zwaan, L, Singh, H. The challenges in defining and measuring diagnostic error. Diagnosis 2015;2:97–103. https://doi.org/10.1515/dx-2014-0069.Search in Google Scholar PubMed PubMed Central
21. Singh, H, Khanna, A, Spitzmueller, C, Meyer, AND. Recommendations for using the revised Safer Dx instrument to help measure and improve diagnostic safety. Diagnosis 2019;6:315–23. https://doi.org/10.1515/dx-2019-0012.Search in Google Scholar PubMed
22. Al-Mutairi, A, Meyer, AN, Thomas, EJ, Etchegaray, JM, Roy, KM, Davalos, MC, et al.. Accuracy of the safer dx instrument to identify diagnostic errors in primary care. J Gen Intern Med 2016;31:602–8. https://doi.org/10.1007/s11606-016-3601-x.Search in Google Scholar PubMed PubMed Central
23. El-Kareh, R, Hasan, O, Schiff, GD. Use of health information technology to reduce diagnostic errors. BMJ Qual Saf 2013;22:ii40–51. https://doi.org/10.1136/bmjqs-2013-001884.Search in Google Scholar PubMed PubMed Central
24. Singh, H, Giardina, TD, Forjuoh, SN, Forjuoh, SN, Reis, MD, Kosmach, S, et al.. Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21:93–100. https://doi.org/10.1136/bmjqs-2011-000304.Search in Google Scholar PubMed PubMed Central
25. Murphy, DR, Meyer, AN, Sittig, DF, Meeks, DW, Thomas, EJ, Singh, H. Application of electronic trigger tools to identify targets for improving diagnostic safety. BMJ Qual Saf 2019;28:151–9. https://doi.org/10.1136/bmjqs-2018-008086.Search in Google Scholar PubMed PubMed Central
26. Shenvi, EC, El-Kareh, R. Clinical criteria to screen for inpatient diagnostic errors: a scoping review. Diagnosis 2015;2:3–19. https://doi.org/10.1515/dx-2014-0047.Search in Google Scholar PubMed PubMed Central
27. Mendu, ML, Lu, Y, Petersen, A, Tellez, MG, Beloff, J, Fiumara, K, et al.. Reflections on implementing a hospital-wide provider-based electronic inpatient mortality review system: lessons learnt. BMJ Qual Saf 2020;29:304–12. https://doi.org/10.1136/bmjqs-2019-009864.Search in Google Scholar PubMed
28. Schiff, GD, Kim, S, Abrams, RI, Cosby, K, Lambert, B, Elstein, AS, et al.. Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project. In: Advances in patient safety: from research to implementation (volume 2: concepts and methodology). Rockville, MD: Agency for Healthcare Research and Quality (US); 2005.Search in Google Scholar
29. Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf 2009;42:377–81. https://doi.org/10.1016/j.jbi.2008.08.010.Search in Google Scholar PubMed PubMed Central
30. Griffin, JA, Carr, K, Bersani, K, Piniella, N, Motta-Calderon, D, Malik, M, et al.. Analyzing diagnostic errors in the acute setting: a process-driven approach. Diagnosis 2021;9:77–88. https://doi.org/10.1515/dx-2021-0033.Search in Google Scholar PubMed PubMed Central
31. Schiff, GD, Hasan, O, Kim, S, Abrams, R, Cosby, K, Lambert, BL, et al.. Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881–7. https://doi.org/10.1001/archinternmed.2009.333.Search in Google Scholar PubMed
32. Pippins, JR, Gandhi, TK, Hamann, C, Ndumele, CD, Labonville, SA, Diedrichsen, EK, et al.. Classifying and predicting errors of inpatient medication reconciliation. J Gen Intern Med 2008;23:1414–22. https://doi.org/10.1007/s11606-008-0687-9.Search in Google Scholar PubMed PubMed Central
33. Kripalani, S, Roumie, CL, Dalal, AK, Cawthon, C, Businger, A, Eden, SK, et al.. Effect of a pharmacist intervention on clinically important medication errors after hospital discharge: a randomized trial. Ann Intern Med 2012;157:1–10. https://doi.org/10.7326/0003-4819-157-1-201207030-00003.Search in Google Scholar PubMed PubMed Central
34. Singh, H. Editorial: helping health care organizations to define diagnostic errors as missed opportunities in diagnosis. Joint Comm J Qual Patient Saf 2014;40:99–101. https://doi.org/10.1016/s1553-7250(14)40012-6.Search in Google Scholar PubMed
35. Zikos, D, Shrestha, A, Fegaras, L. Estimation of the mismatch between admission and discharge diagnosis for respiratory patients, and implications on the length of stay and hospital charges. AMIA Jt Summits Transl Sci Proc 2019;2019:192–201.Search in Google Scholar
36. Williams, T, Szekendi, M, Pavkovic, S, Clevenger, W, Cerese, J. The reliability of AHRQ common format harm scales in rating patient safety events. J Patient Saf 2015;11:52–9. https://doi.org/10.1097/PTS.0b013e3182948ef9.Search in Google Scholar PubMed
37. Bergl, PA, Zhou, Y. Diagnostic error in the critically ill: a hidden epidemic? Crit Care Clin 2022;38:11–25. https://doi.org/10.1016/j.ccc.2021.09.005.Search in Google Scholar PubMed
38. Winters, B, Custer, J, Galvagno, SMJr, Colantuoni, E, Kapoor, SG, Lee, H, et al.. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012;21:894–902. https://doi.org/10.1136/bmjqs-2012-000803.Search in Google Scholar PubMed
39. Vaghani, V, Wei, L, Mushtaq, U, Sittig, DF, Bradford, A, Singh, H. Validation of an electronic trigger to measure missed diagnosis of stroke in emergency departments. J Am Med Inform Assoc 2021;28:2202–11. https://doi.org/10.1093/jamia/ocab121.Search in Google Scholar PubMed PubMed Central
40. Bhise, V, Sittig, DF, Vaghani, V, Wei, L, Baldwin, J, Singh, H. An electronic trigger based on care escalation to identify preventable adverse events in hospitalised patients. BMJ Qual Saf 2018;27:241–6. https://doi.org/10.1136/bmjqs-2017-006975.Search in Google Scholar PubMed PubMed Central
41. Mahajan, P, Pai, CW, Cosby, KS, Mollen, CJ, Shaw, KN, Chamberlain, JM, et al.. Identifying trigger concepts to screen emergency department visits for diagnostic errors. Diagnosis 2021;8:340–6. https://doi.org/10.1515/dx-2020-0122.Search in Google Scholar PubMed
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/dx-2022-0032).
© 2022 Walter de Gruyter GmbH, Berlin/Boston
