Most Canadian non-critical care residency programs require their trainees to undertake rotations through an intensive care unit (ICU), in part to learn how to assess, manage, and appropriately refer (for further care) their sickest of patients.1 We propose that the educational value of these ICU rotations—as with any core and off-service rotations—relies on three fundamental assumptions:

  1. 1.

    A few weeks of ICU experience provide adequate exposure to key learning opportunities;

  2. 2.

    Exposure to learning opportunities helps residents to develop and retain essential clinical abilities;

  3. 3.

    Resident clinical abilities are directly observed and assessed by supervising ICU clinicians to confirm learning.

In this commentary, we reflect on the challenges of these assumptions on two fronts: 1) a lack of supporting evidence, and 2) clinical and educational constraints inherent to the ICU environment. In addition, we argue for the importance of high-quality data to advance our understanding of ICU learning processes and outcomes, especially as residency programs transition to competency-based medical education (CBME).

The ICU environment offers unparalleled learning opportunities.2,3 Residents can partake in many clinical activities uniquely encountered during ICU rotations, including procedures (e.g., central venous catheter insertion and airway management), first-hand assessment and stabilization of deteriorating patients—e.g., as part of Rapid Response (Outreach) Teams,4 goals-of-care discussions with patients or relatives, or team-based resuscitations requiring crisis resource management skills. Certain specialty programs (e.g., internal medicine) expect their residents to become competent in specific skills (e.g., procedures) by the end of their ICU rotations. Nevertheless, data are lacking on how frequently and reliably residents encounter these highly valued experiences during a typical ICU rotation.5

Several barriers are likely to compromise resident access to key learning opportunities in the ICU.6 Critical care practice is highly heterogeneous with clinical problems and patient populations that can vary greatly from day to day or between hospitals. Accordingly, where and when residents complete their ICU rotations can impact what they see and do.5 Yet, residents’ learning needs are rarely the sole determinant of their rotation schedule.7 The capacity of individual ICUs to both accommodate multiple rotation requests and to meet educational expectations of different programs is limited. Heterogeneous resident groups are therefore distributed across a small number of ICUs based on availabilities. Certain specialty programs also have rules in terms of duration and timing of ICU rotations. As a result of these various constraints, resident ICU rotations often lack educational continuity, such as when rotations are short and distributed over time and place (e.g., two separate months over two years in two different ICUs) or are at risk of educational redundancy when rotations are completed continuously in the same location (e.g., three consecutive months in one ICU). Either way, rotating residents do not experience a unifying ICU curriculum that is tailored to their needs and delivered longitudinally across rotations. A resident’s learning experience is largely determined by local patient population, work organization, and educational practices.7 In addition, learning opportunities can be lost in many other circumstances such as when trainees are not invited to actively participate in certain clinical activities by other ICU team members. It can also occur when residents choose not to engage based on a perceived lack of relevance for their future practice,8 or when safety concerns for critically ill patients hinder residents’ direct involvement in patient care.9 The ICU environment is therefore fraught with educational challenges, some of which are shared in part by other clinical environments receiving specialty residents during off-service rotations. Learning opportunities cannot be assumed to be equally and consistently available for all residents, and data are needed to develop realistic learning objectives for ICU rotations.

Clinical experience and exposure to learning opportunities do not necessarily translate into learning and performance improvement.10 Even under optimal learning conditions, it is possible that adequate performance may not be achieved for each ICU-relevant competency and by each rotating ICU resident.11 Learning is consolidated through repeated, deliberate practice.12 When chances to learn do arise in the ICU, they may be random, unique, or not substantial enough for residents to derive sustainable benefit.2 A limited number of studies indicate that residents can gain certain skills during short ICU rotations,13,14,15,16 especially when simulation-based medical education is used to complement clinical learning.10,15,16 While these studies demonstrate short-term benefits, learning outcomes have been measured using simulation or self-report tools,16 and are inconsistent among trainees.14 It remains unclear what level of competence is achieved in clinical practice, and if long-term retention occurs. To better understand the impact of short ICU rotations on specialty resident learning outcomes, these outcomes need to be measured consistently and longitudinally. The respective contribution of out-of-the-ICU teaching activities (e.g., simulation-based training) and real ICU clinical activities on learning outcomes should also be further explored.

A valid assessment strategy is required to determine if residents achieve targeted learning outcomes. Until recently, residents were provided with one summative evaluation—an In-Training Evaluation Report (ITER)—at the end of each rotation. With the CBME reform, each Canadian specialty program is required to define explicit learning outcomes relevant for residents’ future practice and to regularly assess resident competencies.17 Some programs are now requesting that supervising ICU physicians complete workplace-based assessments for rotating ICU residents. Some view the CBME model as a chance to gather additional educational data while simultaneously optimizing opportunities for feedback.17,18 As different specialty programs select various learning outcomes and assessment frameworks for a given rotation, the feasibility of implementation in the ICU is unclear. The level of resident supervision in the ICU varies considerably based on the supervisor, time of the day, location, and workload. Acute care delivered outside of the ICU or outside of regular working hours often occurs without direct supervision.9 We hope that when present, supervising ICU physicians and senior trainees are able to directly observe and assess trainees’ clinical abilities. Nevertheless, a large body of literature has shown that direct observation and timely feedback continue to occur too infrequently in many clinical settings due to time constraints, lack of faculty development, and socio-cultural factors (e.g., fear to compromise relationships with learners).19,20 There is no reason to believe that ICU rotations offer better assessment opportunities than other clinical environments. In fact, ICU supervisors face unique challenges. Supervising ICU physicians are expected to provide assessments for trainees from multiple programs. They may disagree with or be unable to judge the relevance of certain competencies for specialty residents. Physicians’ limited cognitive capacity may force them to choose service over education when patient care is compromised. For example, it has been reported that ICU physicians make over 100 decisions daily during rounds, and few of these decisions relate to teaching or resident feedback.21 These represent just a few examples that force us to question the ability of ICU supervisors to provide rotating non-critical care residents with high-quality assessments that are accurate, fair, and meaningful.

Most physicians practicing in acute care settings will have to develop and maintain clinical abilities to assess, stabilize, and appropriately refer (for further care) the sickest of their patients. In theory, many of the skills required to do so are gained during residency through ICU rotations. In practice, we know very little about the educational experience of rotating specialty ICU residents. More information is urgently needed to establish a framework of competencies realistically achievable during ICU rotations as Canadian residency programs transition to CBME. We predict that many competencies expected to be learned by specialty residents and to be assessed in the ICU will not be accomplished. To understand the etiology of the problem and improve clinical learning, more data on ICU educational opportunities will be needed. The same argument could be made of many off-service and core rotations. We may realize that different approaches should be considered to train specialty residents, such as creating Rapid Response (Outreach) or procedural rotations, additional use of simulation for training and assessment, longitudinal integration of critical care training during residency, or recruitment and training of critical care physicians “assessment experts” (i.e., with non-clinical time dedicated to trainees’ assessment). Rigorous data, rather than personal opinions and traditions, should inform the choice of strategies used to train specialty residents in critical care. We therefore argue for a greater understanding of what it means to learn in today’s ICUs. Once we have established the opportunities for learning and assessment available during a typical ICU rotation, a realistic set of competencies can then be defined and assessed. Only then will we be able to confidently state that ICU rotations equip specialty residents with the appropriate tools to eventually care competently for their sickest of patients.