The novel coronavirus (SARS-CoV-2) has disrupted healthcare and placed additional strain on mental health systems around the world [1,2,3]. Hospital systems have had to act quickly to adapt to maintain services amid the difficulties of infection control and risks of exposure to frontline clinical staff. Emergency and intensive care services have been prioritized with resources and staff to treat patients with COVID-19, whereas outpatient clinics and elective procedures have been reduced or cancelled [4, 5]. Patients with severe mental illness have been recognized to be at risk due to systematic changes wrought by the pandemic [6].

One strategy for maintaining services is cross-training staff from other specialties and services. In a recent survey of United States psychiatrists, 42% reported cross-coverage by staff from different services or specialties at their workplace [7]. Academic psychiatrists are positioned to facilitate service cross-training and thereby maintain high quality and accessible mental health services. In this educational case report, we describe the rapid development and evaluation of a cross-training curriculum in our psychiatric emergency service (PES). Our experience provides lessons of success and challenges for future similar efforts.

Shifts in Patient Care on Our Service due to SARS-CoV-2

Denver Health is an integrated safety-net health system and a primary site for care of patients with COVID-19 in the Denver area. Part of Denver Health’s Level 1 Trauma Center, the PES is a 16-bed, autonomous locked unit that manages behavioral emergencies including for patients with COVID-19. The PES is staffed primarily by attending psychiatrists, physician assistants, and nurses with specific training in behavioral health. Most patients are seen by a nurse who completes an initial evaluation, gathers historical and collateral information, and consults with an attending psychiatrist.

In preparation to accept a surge of patients with COVID-19, our hospital made plans to re-allocate staff to address care needs. The PES’ nursing team prepared to provide stopgap staffing for medical services and anticipated absences due to illness. Meanwhile, the closure of Denver Health’s community clinics and a decrease in volume on the inpatient services freed behavioral health social workers from other teams to work in the PES. We sought to cross-train these social workers for service in the PES. Although social workers complete emergency assessments in many hospitals, they do not fill this role in our health system. Nevertheless, it was felt that their training and experience well-positioned them to perform emergency psychiatric assessments with the support of the attending psychiatrist.

Curriculum Development and Evaluation

We created a course to cross-train our behavioral health social workers from outpatient and inpatient settings to manage emergency psychiatric presentations in the PES. The course consisted of a 90-min lecture and case-based discussion followed by a 3-h shift shadowing in the PES. Due to the tight timeline, all presentation and evaluation materials were adapted from pre-existing materials that had been developed for instruction and orientation of residents, medical students and physician assistants working in the PES. Social work supervisors provided some initial suggestions for content, but no materials were piloted or reviewed with learners prior to implementation. The didactic portion was taught over live video conferencing, and those who missed the live lecture could watch a recorded version. The content included both logistical information around PES operations and patient flow as well as a clinical review of standards of care and best practices in emergency psychiatry [8]. We used case vignettes to illustrate typical patient presentations, summarize procedures for risk assessment and safety planning, review disposition options, and provide insight into clinical decision-making. The shadowing portion paired participants with a nurse evaluator to observe and participate in patient evaluations in the PES. They completed a checklist with PES staff reviewing the use of the medical record and procedures for personal protective equipment and telepsychiatry equipment. Participants also joined the attending psychiatrist during their portion of the assessment to see a patient encounter from start to finish and gain better understanding of treatment planning. Table 1 details the course objectives and content.

Table 1 Objectives and Content of Emergency Psychiatry Cross-Training Curriculum
Full size table

As the cross-training course was undertaken so quickly, the course evaluation utilized self-assessment instruments developed for a prior multidisciplinary training effort [9]. Using a secure web-based survey platform [10], participants were sent a pre-course survey to assess their comfort with a variety of clinical scenarios typically encountered in the PES. A post-survey was sent after the course to again assess comfort level. Comfort and preparedness were assessed using a five-point Likert scale (strongly agree, somewhat agree, neutral, somewhat disagree, strongly disagree). We hypothesized the training would increase social workers’ sense of preparedness to work in the PES. Analyses were conducted according to a published algorithm [11]. This evaluation activity was determined not to be human subjects research by our hospital quality improvement committee which is authorized by the local institutional review board.

Nineteen social workers volunteered for training, and 17 completed the course. One participant did not complete the course due to scheduling conflicts, and another stopped due to concerns about potential SARS-CoV-2 exposure in the PES. Of 17 participants, 16 (94%) completed the pre- and post-survey for analysis. All participants were social workers with master of social work or licensed clinical social work degrees. Most participants (63%) worked in inpatient psychiatry, evenly split between adult (50%) and child/adolescent (50%) populations. Most participants either had never worked in emergency or crisis settings (50%) or only had brief exposure during a clinical rotation in training (13%). Trainees were asked how similar they expected emergency psychiatry to be to their current clinical environment, on a scale of 0–100%: the mean response was 56% (SD ± 28%).

Curriculum Assessment Results

On the primary assessment question— “How prepared do you feel to practice in psychiatric emergency services”?”—thirteen respondents (81%) felt somewhat or very prepared before training, compared to 14 respondents (88%) after training. Two respondents felt less prepared after training. This outcome was not statistically significant by Wilcoxon signed-rank testing (p = 0.61). Figure 1 describes the change in reported comfort with different patient presentations and clinical tasks after the training; these differences were not statistically different. In further analysis of the data, we were intrigued that most respondents (63%) reported feeling less comfortable with at least one of the patient presentations or clinical tasks after completing the training. A similar number (69%) felt more comfortable with at least one presentation or task. How can we explain such apparently discrepant findings? To explore this phenomenon, we undertook two exploratory analyses.

Fig. 1
figure 1

Social workers' comfort managing patient presentations and clinical tasks after the cross-training course (n = 16)

Full size image

Our first question was whether our primary outcome question was valid. Although the question holds strong face validity, we assessed its convergent validity with the elements of emergency psychiatric practice described in Fig. 1 [12]. The 18 patient presentations and tasks were combined into a single cumulative score (from 18 for “very uncomfortable” on all items to 90 for “very comfortable”) and compared the score to our primary outcome question. Both pre- and post- scores were used. A Spearman’s correlation test revealed the cumulative comfort on those elements to strongly correlate with respondents’ answer on the primary outcome question (rs = 0.48, p < 0.01). Thus, we felt the primary outcome question reliably mirrored comfort with the underlying topic areas with which we were most concerned.

The second question was whether respondents with a certain degree of familiarity with emergency psychiatric care might respond differently to training. For example, the training might disproportionately benefit those with very little emergency practice exposure. In fact, the opposite was true: 75% (3/4) of those who reported feeling less prepared to work in the PES reported little to no experience working in this environment. We also considered whether expectations might play a role in how training was received. Respondents were stratified into those who felt emergency psychiatry was likely to be more (> 50%) or less (< = 50%) similar to their current position. Those who felt the PES would be less similar more often reported feeling more prepared (5/8, 63%) after the course, whereas those who felt the PES would be more similar rarely felt more prepared (2/8, 25%). These results were not statistically significant in signed-rank testing.

In the months following the training, the urgent need did not arise for participants to provide cross-coverage shifts in the PES. As a result, we did not have the opportunity to provide any additional supervision to participants, evaluate their work, or observe any impact on patient care. We sent a follow-up survey to participants to gather qualitative data regarding the training, asking the questions: “What helped you feel more comfortable practicing in the PES? What made you feel less comfortable? How can we better prepare clinicians from other services for working in the PES?” We received seven responses. Three participants were already familiar with PES processes and staff so they felt the training was an appropriate primer. Two respondents reported feeling prepared and valued the opportunity to shadow a nurse for an assessment. Two participants noted that having a single point person to orient all participants or a comprehensive written guide for intermittent staff members would be helpful.

Reflection on Reactions to the Training

There are several reasons that we may not have seen a difference on the primary outcome. The high rates of baseline comfort and low sample size made it difficult to detect an effect from a necessarily hurried educational intervention. Our evaluation instruments were re-purposed from previously published works and hastily applied, without pilot testing due to the immediacy of the crisis; thus, these instruments may have been insensitive to change for their purpose. We did not have the opportunity to objectively assess skills so reported comfort and confidence may not correlate with ability.

We theorize that both the change in clinical context and increased conscious awareness of deficits contributed to negative responses among social workers undergoing cross-training. While respondents are well-versed in assessing patients in their typical place of work, performing those tasks in a new environment is more challenging. This result is consistent with our understanding of learners’ sense of self-efficacy, which relies both on knowledge of skills and the context in which those skills are applied [13].With learning also comes awareness of skills deficits. Inability to recognize one’s own level of competence has been associated with inflated self-assessments. As competence improves, self-assessment becomes more accurate [14, 15].Through the course of training, some participants likely moved from a state of unconscious incompetence to conscious incompetence. This transition occurs at a point of learning when the learner becomes aware of skill deficits [16]. This increased awareness likely caused some to feel less comfortable with their skills than they felt prior to the training. This is not to imply respondents are incompetent at their jobs; rather, they are not yet competent in a new clinical environment in which they have never worked.

In addition, acute or complex patient presentations on shadowing shifts could have further contributed to respondents’ discomfort. The PES environment can be chaotic, and most patients are in an acute crisis. We were unable to control for the variability of patient presentations on shadowing shifts. Additionally, outside of a brief orientation guide, there was no standardization of the shadowing process with participants working with a variety of nurses and attendings. This lack of consistency was noted in the qualitative feedback and would be an opportunity for improvement through designating specific staff in the PES to spearhead the orientation process or at least providing a more detailed orientation guide to participants.

When viewed in the context of typical orientation procedures for a new clinical environment, it is understandable that participants would still feel uncomfortable after a brief shift shadowing in the PES. New nursing staff typically orients for several shifts before evaluating patients independently. Other learners such as medical, physician assistant and psychology students who rotate through the PES complete a twelve-hour on-site orientation shift prior to performing patient assessments. Despite our participants’ extensive background, the amount to learn in a shadowing shift may simply have been overwhelming.

Implications for Future Work in the COVID-19 Era

In the COVID-19 era, academic psychiatrists are leading rapid changes in staffing, maintaining safe and enriching educational programming, and adapting evidence-based practices to the exigencies of the coronavirus pandemic. This case reports the importance, feasibility, and challenges of applying the practices of medical education to emergent health systems change wrought by the pandemic. In this case, extant teaching and evaluation instruments appeared imperfectly applicable to new uses in multidisciplinary training or tele-learning. Cross-training clinical staff likely requires more facile teaching methods to engage learners from different disciplines and with different career backgrounds. Our plan was to repeat this evaluation after social workers’ first emergency psychiatry shift to assess for skills acquisition; however, we have been fortunate that this contingency plan has not been utilized. Our results suggest a critical need to be attentive to the ongoing training needs of staff who would be meeting critical demands in patient care even as they may feel unsuited to their responsibilities. There is no substitution for hands-on learning under the supervision of trained staff. If a similar need arises in the future, longer, more structured in-person shadowing would be recommended. Clear work expectations should be accompanied by written work protocols and a single point of contact for clinical and technical support. Academic psychiatrists’ skills in teaching and team leadership are apropos to helping teams meet the unprecedented mental health burden of this pandemic. Yet this experience must be applied thoughtfully and critically to be of greatest benefit for our colleagues and patients.