Delirium screening for patients in the intensive care unit: A prospective validation study of the iCAM-ICU by nurse researchers and bedside nurses in routine practice

https://doi.org/10.1016/j.ijnurstu.2021.103886Get rights and content

Abstract

Background

Daily delirium assessment using the Confusion Assessment Method for the Intensive Care Unit was recommended for patients in the intensive care unit, yet implementation may be difficult because of lack of simple and standardized data collection schemes which may result in low sensitivities when used by bedside nurses.

Objective

This study was to validate the diagnostic accuracy of the intelligent Confusion Assessment Method for the Intensive Care Unit (iCAM-ICU) used by both nurse investigators and bedside nurses in Chinese patients in the intensive care unit.

Setting

A university affiliated tertiary hospital in China.

Participants

A total of 373 hospitalized patients (181 in phase I and 192 in phase II) in the intensive care units met the inclusion criteria and participated in the study. There were two nursing researchers (phase I) and 24 bedside nurses (phase II) used the iCAM-ICU to assess delirium among patients.

Methods

Two prospective cohort studies were conducted to validate the diagnostic accuracy of iCAM-ICU on delirium screening when it was used by nurse investigators and bedside nurses in the intensive care unit. Using the Diagnostic and Statistical Manual of Mental Disorders as the gold standard, the sensitivity, specificity, predictive values along with the likelihood ratios were determined to estimate the performance of the iCAM-ICU in patients in the intensive care setting. The Kappa consistency test was examined to determine the inter-rater consistency. Subgroup analysis in terms of different age, level of education, severity of illness and cognitive status were also conducted to evaluate potential variations of the iCAM-ICU performance in different patient groups.

Results

A total of 373 patients were included in the validation studies. In comparing with the gold standard, the sensitivities of the iCAM-ICU demonstrated by the two nurse investigators were 95.2 % and 93.7%, while the specificities of the iCAM-ICU were 93.3% and 93.2%. The Kappa consistency between two nurse investigators was 0.96. The sensitivity and specificity of the iCAM-ICU demonstrated by bedside nurses in intensive care patients were 86.7% and 97.7%, respectively. Subgroup analysis also revealed that the sensitivities and specificities in those different subgroups were acceptable, with all statistics being above 80%.

Conclusions

The iCAM-ICU, an information technology enabled delirium screening tool, showed highly acceptable accuracy in detecting delirium in the intensive care units. It can assist bedside nurses to detect delirium reliably and identify potential patients with delirium accurately.

Registration number

ChiCTR-OCH-13003050.

Section snippets

Background

Delirium is one of the most common complications among patients in the intensive care unit (ICU). The reported incidence for delirium ranged about 20% to 50% for nonmechanically ventilated patients, and 60% to 80% for mechanically ventilated critically ill patients in ICU settings (Krewulak et al., 2018; Ely et al., 2001). ICU patients complicated with delirium are predisposed to adverse outcomes, including prolonged ICU and hospital stay, impaired cognitive function, increased in-hospital and

Methods

This study employed a two-phase prospective cohort study design. The institutional review committee of the study hospital (university affiliated hospital) approved this study before it was initiated. The study was registered at the Chinese Clinical Trial website (ChiCTR-OCH-13003050), and written informed consent was provided from patient themselves before their enrollment.

Basic patient characteristics in phase I

In phase I, a total of 240 patients admitted to the 4 ICUs were screened and 181 patients were finally enrolled in this phase (See Fig. 1).

As shown in Table 1, among 181 patients enrolled in phase I, 63 patients were identified as delirium-positive while 118 were as delirium-negative according to the results made by experts against the reference standard. The mean age of patients included in phase I were 70 (SD = 10) years old, and there were about half of patients being male (52.4%) and with

Discussion

In this study, we conducted a two-phase diagnostic testing of the iCAM-ICU in ICU patients among both nurse investigators (phase I) and bedside nurses (phase II) in against the gold standard. To the best of our knowledge, this was the first study in which an interactive mobile application based on the CAM-ICU algorithm with decision-making support was created and tested for accuracy in ICU settings, among nursing investigators and ICU bedside nurses. The results of our study showed good

Strengths and limitations

In our study, we incorporated information technology assisted decision-making support to assist ICU nurses identifying delirium correctly during routine practice. As indicated in previous research, the CAM-ICU paper version has been reported with low sensitivities among bedside nurses (Nishimura et al., 2016; Boettger et al., 2018), this limited its application and reduced its beneficial effect for patients in routine practice. Our study utilized information technology to overcome these

Conclusions

In conclusion, the iCAM-ICU, an intelligent and interactive screening tool for delirium based on the CAM-ICU, can assist bedside nurses to detect delirium reliably. It is identified with high sensitivity and specificity in ICU patients including those with different clinical characteristics. Future studies are needed to evaluate the effect of routine delirium assessment using iCAM-ICU on delirium related clinical outcomes and explore its effect for delirium prevention.

CRediT authorship contribution statement

All authors approved the final version of the manuscript for submission.

Ying Wu setup the protocol, conceptualization, carried out the design and development of the “iCAM-ICU”, oversaw the collection, analysis and interpretation of the data, and critically reviewed and revised the manuscript. Fangyu Yang drafted the protocol, carried out the design and development of “iCAM-ICU”, participated in data collection, analysis and performed statistical analysis, as well as development of the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

In the development stage of iCAM-ICU, interface outline, flow chart and process based on CAM-ICU were provided by our team, and the group from Beijing Jian Yue Times Technology Co., Ltd. assisted us in the actual development of the application, such as programming, developing and refining the application, but had no

Acknowledgment

We would like to acknowledge the following group or people who have made contributions to the current study. In the process of system development and clinical research, Beijing Jian Yue Times Technology Co., Ltd. provided technology support and realized the iCAM-ICU design. iPads used for delirium assessment were provided by the Shenda group. During our study, we received support from Chaoyang Hospital and Xuanwu Hospital (affiliated hospitals with Capital Medical University) by giving us

Funding

This study was supported by the National Natural Science Foundation of China (#30871049, Principal Investigator: Dr. Ying Wu).

References (29)

  • Confusion Assessment Method for the ICU (CAM-ICU): The Complete Training Manual [Internet]

    (2002)
  • JW Devlin et al.

    Clinical practice guidelines for the prevention and management of pain, agitation / sedation, delirium, immobility, and sleep disruption in adult patients in the ICU

    Crit. Care Med.

    (2018)
  • PC Dykes et al.

    Fall prevention in acute care hospitals: a randomized trial

    JAMA

    (2010)
  • EW Ely et al.

    Delirium in mechanically ventilated patients: validity and reliability of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)

    JAMA

    (2001)
  • Cited by (11)

    View all citing articles on Scopus
    1

    Authors contributed equally to the publication of the manuscript.

    View full text