Designing and Evaluating a Virtual Patient Simulation—The Journey from Uniprofessional to Interprofessional Learning

Abstract

“Ready to Practice?”(R2P) is a virtual patient simulation designed for undergraduate medical and pharmacy students. After initial prototyping, R2P developed into a screen-based virtual patient (VP) simulation with an intuitive interface using photorealistic images of people and places with speech bubbles and decision menus. We describe the design of the VP, findings from student experiences with the software, and the potential of VPs for interprofessional learning. We used a mixed methods study to assess students’ perceptions of the VP as a learning tool. Qualitative data were gathered using semi-structured interviews and observations, and quantitative data through the Readiness for Interprofessional Learning Scale (RIPLS) and an evaluation questionnaire. Overall, participants showed significantly improved RIPLS scores after participation in the simulation (78.78 to 82.25, p < 0.0001), including in the Positive Professional Identify domain (p < 0.001). Students also showed significant improvement in RIPLS scores in the Teamwork and Collaboration domain when pharmacy and medical students were working together in interprofessional pairs (40.75 to 43.00, p < 0.006) but not when working alone (n.s.). Five themes emerged from interviews where participants identified specific interprofessional insights into each other’s roles and skills. Students found the VP engaging and valuable for their learning and their understanding of teamwork.

1. Introduction

A fast-changing healthcare environment has presented medical and health sciences education with a significant challenge [1]. Educational institutions have been tasked with producing work-ready graduates who can confidently enter into clinical roles [2]. Students are expected to be self-learners who demonstrate clinical competence, sound clinical decision-making skills, and effective communication, all the while keeping up to date with rapidly evolving medical science. Furthermore, there is increasing pressure for students to develop interprofessional communication skills and knowledge of other clinicians’ roles and abilities in order to practice safely and effectively in interprofessional healthcare teams [3,4].

To meet these learning requirements, students need to practice applying their clinical knowledge and skills in a variety of patient cases and medical conditions. However, clinical reasoning and how to work in an interprofessional team is often not explicitly addressed in the undergraduate curriculum. Furthermore, traditional teaching methods and clinical placements often do not adequately address this need due to a lack of available placement sites, poor student engagement and safety concerns in using real patients for such purposes [2]. The current ubiquity of technology-enhanced environments means that most contemporary learners are accustomed to learning using digital resources which support them as independent and self-directed learners, while allowing for a personalized learning approach [1,5].

To meet the learning needs of students and educational institutions, the use of simulation to imitate real patients and clinical cases has become widespread in medical education [1,6,7]. Several systematic reviews have unequivocally shown that technology-based simulation training improves outcomes across an array of clinical areas [6]. In interprofessional learning (IPL), simulations are effective in developing a wide range of skills in learners, including non-technical skills, such as communication, teamwork, leadership and insight into others’ skills and roles [7,8,9].

Anaesthesiologists were some of the early adopters of high-fidelity simulation education as it was seen to develop anaesthetic skills in time-compressed sequences that were difficult to recreate otherwise [9]. It has been shown that simulation experiences from an early stage can promote the development of situational awareness, multi-tasking and vigilance, which can fast-track the acquisition of expertise through teamwork and leadership [9]. Using high-fidelity mannequins, clinical simulation is known to be very effective in these regards [10], and is highly valued by students. However, its scalability in terms of its provision in other areas of learning, such as IPL, is heavily constrained by logistical and cost barriers. For example, coordinating timetables in order to get cohorts of students from different disciplines together in the same place and at the same time to participate in IPL mannequin-based simulation can be a major logistical hurdle.

Virtual Patients (VPs) do not have these limitations. VPs are a specific type of computer program that simulates real-life clinical scenarios where learners emulate the roles of healthcare providers to obtain a history, conduct a physical exam, and make diagnostic and therapeutic decisions in a safe space where deliberate practice is promoted, and where errors have no negative consequences on real patients [11,12]. VPs, like mannequins, are effective in addressing certain clinical skills, clinical decision making, teamwork and communication [11,12], but do not necessarily require co-location of students or even alteration of existing teaching timetables, since participation can occur at multiple points in the students’ free time (the VP can be “visited” as one would visit a real patient in a real hospital ward). Furthermore, VPs are a natural supplement and adjunct to enhance existing use of mannequin-based simulation in healthcare, and can be incorporated at several time points in the curricula to better prepare students for mannequin-based or other types of IPL opportunities. Although VPs can result in high initial production costs and can be time-consuming to develop, once developed, they are essentially free to offer as part of a curriculum, can be scaled inexpensively to larger cohorts of students, and can be available anywhere and anytime from a range of devices with internet access [12,13].

How to integrate VPs effectively into a curriculum requires careful consideration and planning. Implementing VPs into the first year of a health sciences programme may result in cognitive overload, where learners have not yet grasped core knowledge and are not ready to apply it in a simulated clinical encounter, or as part of an interprofessional team [11]. For such students, learning basic principles may better occur using other methods such as part-task models or role-playing patients [11]. Moreover, the optimal design of a VP remains an important area to address to achieve effective learning at various training levels [14]. Although it is recognized that VPs will never replace real patients entirely, it is imperative that for learning to occur, aspects of the technology should not distract from the realism of the experience or the engagement with the simulation.

In the following paper we describe the development and learning achieved with a screen-based VP simulation called “Ready to Practice?” (R2P). Intended for medical and pharmacy students, this VP was designed to test clinical decision-making skills in an emergency renal care situation. The simulation was piloted with pharmacy students in the first instance, and then offered as an interprofessional learning exercise for pharmacy and medical students, prior to its introduction as part of the curriculum in a third-year pharmacy course at the University of Auckland. We describe the design of the VP, the feedback provided by students who completed the simulation on their own and in interprofessional teams, and consider the future direction of VPs in the medical and health sciences curriculum.

Our objectives are the following:

• Review the design of the “Ready to Practice?” simulation with reference to current literature.
• Highlight the qualitative and quantitative findings from undergraduate students who completed the simulation either alone or in interprofessional pairs.

1.1. Background: Design of the R2P Simulation

R2P is a web-based simulation written in HTML5 using Javascript (Ecma International, Geneva, Switzerland) and libraries from the Monkey cross-platform scripting language (Blitz Research Ltd., Auckland, New Zealand). It is a screen-based patient-case simulation that enables students to experience a critical and challenging emergency situation in renal care.

1.1.1. Origin

The idea for R2P began in response to a growing need for more clinical placements in the BPharm degree at the University of Auckland. Prior to 2016, the degree provided very few experiential student placements in the undergraduate curriculum, which meant that students had little opportunity to put their clinical knowledge and skills to practice. The development of R2P was also consistent with recent policy directives from organizations such as the Australian Health Workforce Institute, who have highlighted the considerable need for, and value of, IPL using simulation, in order to achieve better health outcomes for patients [8].

In 2010, a Teaching Improvement Grant (University of Auckland) was secured to develop an interactive virtual kidney renal module containing core renal content in pharmacology, physiology and anatomy and clinical cases in a virtual online environment. The interactive renal project was seen as an opportunity for introducing interactive clinical cases that could address some of the gaps in the curriculum regarding experiential learning, and test student skills in communication and decision making in a safe and comfortable learning environment.

Second Life^® (SL), an online virtual world developed by Linden Lab (San Francisco, USA) where users interact with each other and the environment through avatars, was first selected as the simulation learning space. At that time, several medical projects at the University of Auckland had made use of SL medical facility that had been built for this purpose [15]. Although SL seemed like a worthwhile avenue to explore, the rising subscription fees made it challenging to continue using this platform and the bandwidth and processing power required for running SL were not optimal at the time. For our purposes, SL was seen as a rather technically complex platform, where moving and interacting in the environment can comprise a steep learning curve requiring self-directed or group training prior to any activity. In addition, a limited number of users can effectively participate at any one time, the time taken to run sessions is considerable, and activities are synchronous [16].

Due to these limitations, it was decided to develop a screen-based interface based on still photographic-quality images of real clinical scenes and people with speech bubbles with clickable menus options for interaction. This presented users with a simple and intuitive interface that did not require users to undergo a familiarization phase before the learning activity could begin. This was seen as something that would be more suitable in achieving our aims for a simulated clinical activity that would be self-directed, learner-driven and useful for independent study as well as a revision resource.

1.1.2. Script Development and Rationale for Case Content

The script for the R2P simulation was discussed through a series of meetings with representatives from the School of Pharmacy, School of Medicine and the Centre for Learning and Research in Higher Education (University of Auckland); a game developer; and a renal pharmacist and two renal consultants from Middlemore Hospital, the largest hospital in South Auckland, New Zealand.

From these initial discussions it was concluded that for a case to be relevant and contain essential learning for both medical and pharmacy students, it needed to incorporate elements of diagnosis and medication management. One case that met both these criteria focused on a drug interaction that could potentially lead to acute renal failure commonly known as the “triple whammy”. This is a potentially life-threatening drug combination resulting from a combination of an angiotensin-converting enzyme inhibitor/angiotensin receptor antagonist, diuretic and nonsteroidal anti-inflammatory drugs and is of particular concern in the elderly, patients with reduced renal function and those with co-morbidities, such as diabetes [17]. Unpublished data from Middlemore Hospital suggests that between January 2016 and 2017 over 100 patients had been exposed to the triple whammy drug interaction during their stay in hospital (personal communication with Deborah Bassett-Clarke, the Middlemore Hospital pharmacist).

Since the majority of patients at Middlemore Hospital with reduced renal function are type 2 diabetics of Māori or Pacific Island ethnicity, it seemed rational for the “patient” in the simulation to be representative of the latter group. Hence, the case focuses on a realistic case—Mr. Fa’alogo, a 58-year-old Pacific Island male with type 2 diabetes, high blood pressure, heart failure, chronic renal impairment and gout, who takes several medications for these conditions.

1.1.3. VP Design

R2P is a screen-based simulation which makes use of still photographic images of a patient actor and clinical staff, overlaid with speech bubbles, prompts and decision menus with a series of options that the participant can select from. These images are coupled with auditory sounds that are taken from the same clinical environment as the images (Middlemore Hospital, Auckland, New Zealand). This was to give participants enough realistic detail to allow them to engage with the VP without overloading them with unnecessary detail, and allows intuitive point-and-click interaction with the characters and objects in the simulation.

Screen-based cases, being the earliest forms of simulation, have a proven track record, and R2P falls within the second most common use of simulations, which is for patient care [18]. Patient care cases, such as R2P, also span across the procedural and situational simulation categories of Lyons [19]. Procedural simulations require participants to follow a correct sequence of steps to achieve the appropriate end goal while situational simulation approaches prompt participants to think critically and make relevant decisions with constant feedback to rethink and adapt.

The pace and interaction of R2P are learner driven and controlled, and a progress bar allows students to visualize how far they have progressed through the simulation exercise. Progress indicators are useful for users to track how much of a task has been completed. Although they are not representative of the time it takes to attend to a real-life emergency crisis, it may help to reduce anxiety by visually representing a goal with a beginning and an end [20]. Research also suggests that user satisfaction with a task is only likely to occur if the feedback communicates that the task is brief, or moving quickly, particularly in the beginning stages [21].

The R2P simulation is designed in three parts. Part I begins with a photograph outside of a hospital ward room. The student is given instructions on how to progress through the simulation with the use of a computer mouse graphic, indicating that to proceed the left mouse button must be clicked. Students are informed that they will be faced with a life-threatening emergency, and while the patient would not deteriorate as quickly in real life, for the purposes of the simulation they will need to avoid wasting time. After this instruction, the simulated nurse provides a medical handover by presenting a background and assessment of the patient, and asks the user for their recommendation.

The patient, Mr. Fa’alogo, has presented at hospital with acute kidney injury (AKI), and requires emergency treatment. The student is presented with Mr. Fa’alogo in a hospital bed in the hospital’s high dependency unit (Figure 1), and includes several options to obtain additional information to assist with the diagnosis of Mr Fa’alogo. These include a letter from the patient’s GP, patient notes, the readout of a cardiac monitor and interaction with the patient through predetermined questions and responses. The status of the patient is illustrated by a cardiac monitor, and the participant is warned—at three and six minutes into the simulation—if they are taking too long to reach a clinical care decision and that the patient is deteriorating. Responses of the nurse also change depending on the care choices made by the participant. Should the participant take longer than nine minutes to reach a diagnosis and make an appropriate treatment recommendation, a flat-lined ECG appears and the participant is informed that the patient has died. There is then an option to restart the simulation. A number of distractors—the nurse asking whether the student requires a neurology scan or further assessment—are also present to add realism, requiring the participant to assess the situation and determine the most important and relevant information.

Malone [22] believed that good educational simulation designs include challenge and curiosity that can be used to varying degrees to motivate and engage students in deep, meaningful learning. This challenge in the R2P simulation is represented by the uncertainty involved in achieving the defined goal of treating the patient under time-constrained circumstances. The provision of information (e.g., patient history, drug charts) from which the participant prioritizes and selects, creates a level of curiosity that invokes intrinsic motivation, often associated with deep learning. The death of the simulated patient was incorporated as a realistic consequence of a student not taking a certain action, i.e., giving the appropriate treatment within the time allowed. It has been suggested elsewhere that the death of the simulated patient not only improves the psychological fidelity of VP simulations, but may also enhance learning if conducted in a professional and ethical manner, and not used as student ‘punishment’ [23].

Part II allows students to reflect on their answers from Part I through interaction with a Senior Medical Officer, another character in the simulation. She asks a series of questions related to the diagnosis and treatment decisions made by the student in Part I followed by the display of a set of predetermined possible answers from which the participant is required to select the correct answer. An additional four questions, determined by the student’s previous responses, allow for open-text comments by the participant. These ask the student for their reasoning for making use of certain resources; requesting/not requesting additional information such as neurological scans; the relevance of back pain to the diagnosis; and key features that led to the clinical problem.

In this part, students are also requested to complete a drug chart for the patient by increasing, reducing or stopping the dose of the current medication that the patient has been charted (Figure 2), with an option to try again if incorrect. They are given feedback only if the drug dosage change is correct, and can progress to the next part if all dosages are correct. Further open-text response questions allow students to reflect on the most important medications involved in acute renal failure.

Part III is a Medical Handover Meeting, taking place in a simulated seminar room (Figure 3), where a series of multiple choice questions are presented to quiz students on the causes of AKI, how this would present in a patient, and the correct treatment thereof. The participant progresses through the simulation by selecting options from decision menus, identifying ECG changes in hyperkalaemia (Figure 4), and answering reflective questions as in Part II. Further technical detail on the design and development of the VP is available from the corresponding author.

Reflection is increasingly being recognized as an essential skill of competent healthcare professionals [24]. Activities to support reflection are progressively being incorporated into undergraduate medical education, and may support students to incorporate new learning into existing knowledge and skills [24]. Used as a learning strategy, and depending on the goal of the activity, VPs may offer a ‘safe’ space where students are not penalized for their reflections or incorrect actions.

Consistent with the work of Cela-Ranilla et al. [25], R2P was intended to (1) offer an alternative approach to learning patient care and management plans, (2) extend the learning in lectures and clinical practice and (3) provide an opportunity for interaction between potential team members, portrayed as characters in the simulation.

1.1.4. User Experiences and Feedback to the R2P Prototype

Nine students enrolled in third-year medicine and five from fourth-year pharmacy programmes were asked to test the R2P simulation, and to complete a series of open-text questions on their user experience at the completion of the simulation. These included questions on instructions for play, navigation, interface design, content (including the year of study R2P was most appropriate for), and level of feedback provided in the simulation.

Students saw an important navigation element was the option to return to a previous menu in the simulation where they had made a mistake, or if they wanted to review a previous screen. They liked the interface design, finding the graphics appealing and life-like. In terms of the content, students felt that warnings and consequences for wasting time, misdiagnosing and incorrect answers were important, and made the simulation more engaging. They suggested the inclusion of irrelevant ECGs and consequences for giving an incorrect treatment. They also proposed differential diagnosis pathways with an opportunity to misdiagnose. Only the medical students requested more information on the drug charts, and desired feedback on incorrect answers rather than only on correct ones.

Minor changes were made to the simulation following this feedback, including a requirement for students to communicate directly with the patient in order to progress through the simulation (previously it was possible to progress without engaging with the patient at all).

Medical students suggested that the simulation was mostly suitable for third years, but possibly also fourth years; pharmacy students believed it was mostly suited for fourth years.

1.1.5. R2P Pilot Study with Pharmacy Students

In 2014, a pilot study was designed to capture feedback from 19 fourth-year pharmacy students by examining the reflective questions in the simulation and by means of a quantitative evaluative survey [26]. Key findings from this study suggest that although students experienced an overall sense of immersion and intrinsic interest in the simulation, and were motivated by the challenge of caring for the simulated patient, a few students expressed difficulty in interpreting the diagnostic images. As pharmacy students are not required to interpret ECGs and other diagnostic images, this prompted the idea to investigate the use of R2P as an interprofessional simulation.

2. Materials and Methods

This study was conducted by five pharmacy student researchers (KF, SP, GT, CW and LW) as part of their final year honour’s dissertation under the supervision of NM and CW. Ethics approval for this research was granted by the University of Auckland Human Participants Ethics Committee (reference 017307).

2.1. Participant Selection

Pharmacy students in their fourth year of study and medical students in their third-year were invited to participate, based on the premise that third-year medical students were preclinical, and likely to be at a similar clinical skill level as fourth-year pharmacy students. Respective course coordinators advertised the study to students via an announcement on the university’s learning management system. Interested students were requested to contact a researcher by email and were provided with further information through a participant information sheet. Students who consented to take part in the study were randomized into one of the following groups: (A) Medical students who would complete the simulation alone; (B) pharmacy students who would complete the simulation alone; or (C) medical or pharmacy students who would complete the simulation in interprofessional pairs.

2.2. Study Design

We aimed to recruit forty students into the study—this number was chosen to allow the collection of sufficient quantitative data for the detection of significant changes in numeric scores, and sufficient qualitative data to allow the meaningful identification of themes [7]. Of the 40 students recruited, ten each from pharmacy and medicine, respectively, were to be allocated to complete the VP alone. A further ten students each from pharmacy and medicine were allocated to complete the VP in interprofessional pairs. Interprofessional learning in the simulator was assessed using a mixed methods approach comprising three distinct phases of data collection: (1) Readiness for Interprofessional Learning Scale (RIPLS), (2) structured observations and (3) a semi-structured debriefing interview.

2.2.1. RIPLS Questionnaire

Participants were requested to complete the RIPLS questionnaire prior to and following the simulation, to assess whether their attitudes towards IPL had changed. RIPLS is a widely used and validated tool that assesses the attitudes of healthcare students towards interprofessional engagement, and is comprised of 19 questions that are answered individually on a five-point Likert scale [27]. The overall score yields a total in the possible range of 19 to 95, with four sub-scales or domains: teamwork and collaboration, negative professional identity, positive professional identity and roles and responsibilities.

2.2.2. Structured Observations

An observation schedule (

Table A1. Observation schedule.

List the sequence in which the student sourced information	Computer [ ] GP letter [ ] Talked to patient [ ] ECG [ ]
Tick any sources of information that the student did not make use of	Computer [ ] GP letter [ ] Talked to patient [ ] ECG [ ]
Did the student/s have to restart the simulation?	Yes/No
Note the time it took for student/s to complete the simulation (patient “saved”)
Note how many times student/s reconfigured the patient’s drug regime
Was there was a specific medication that tripped student/s up while completing the medicine chart?
Note any student comments made when “talking” to the house officer
Note any student comments made during the ‘handover meeting’
Interprofessional groups:
Note how often discussions occurred between the students
Note any observations/student comments that occurred during discussions that led to a clinical decision being made
Note if there were specific sections of the simulation that the medicine or pharmacy student took charge of

) was developed to collect information on the sources of information used and the order in which participants undertook each investigation (e.g., opened the GP letter, talked to the patient, checked the computer and checked the ECG), the number of attempts it took to rescue the patient (i.e., completed Part I of the simulation within the required nine minutes), and the number of times it took to correctly adjust the patient’s medication regimen. In the group where pharmacy and medical students worked in interprofessional pairs, observation also included recording the verbal interaction and leadership behaviours during the simulation.

2.2.3. Semi-Structured Interview

A post-simulation interview schedule (

Table A2. Interview Schedule.

Participants who worked independently:

1. Which aspects of the simulation do you feel you performed well in?

2. Which aspects of the simulation did you find difficult/challenging?

3. Can you identify any points in the simulation where you think you may have wanted help?

4. From whom do you think you may have asked for help from?

5. How do you think that person/profession could have helped you solve the case?

6. Are you able to provide an example(s) of something you learnt as a result of this simulation?

7. How did the simulation aid your understanding of patient care?

8. In what aspects do you think simulations could play a role in undergraduate education?

Participants who worked in interprofessional pairs:

1. How did you feel working as part of a team when caring for the virtual patient?

2. Which aspects or parts of the simulation did you feel went well with your team member?

3. Which aspects or parts of the simulation did you feel didn’t go well with your team member?

4. To what extent do you feel teamwork aided your approach to solving the case?

5. Can you give examples of points during the simulation where a discussion with your teammate may have influenced your choice of action?

6. How did the simulation affect your prior understanding of the roles of a doctor and a pharmacist?

7. What were some skills/knowledge you learnt from working with the other professional in the simulation?

8. In what aspects do you think simulations could play a role in undergraduate education?

9. How do you feel the simulation will benefit your future interprofessional interactions?

) was designed for students who completed the simulation alone or as a pair. The separate schedules were designed to focus on the differences in individual and paired perceptions in their approach to solving the case. Questions were open-ended, asking about perceived performance, attitudes towards teamwork, understanding of professional roles, and the effectiveness of simulations for future interprofessional interactions. This method was chosen to provide an in-depth post-simulation exploration into the student’s perceptions and attitudes towards IPL beyond the quantitative and observational findings.

To ensure consistency in approach, two researchers were involved in each phase of data collection, and the following information was given to all participants before commencing the simulation:

“In this virtual simulation you are a health professional(s) aiding in the diagnosis and acute management of a patient. If you do not successfully manage the patient within 9 minutes, their health will deteriorate and the simulated patient will pass away. Should this happen, you will get a second attempt to complete the simulation.”

The same two researchers who were involved in the observations with paired students also conducted interviews with students post-simulation. These were conducted individually to allow for candid responses. Interviews were recorded and transcribed for analysis.

2.3. Data Analysis

Quantitative data were analysed with SPSS version 25 (IBM SPSS Statistics, Armonk, NY, USA). RIPLS scores were treated as interval data, negatively scored items were reversed for analysis, and analysis was conducted using paired t-tests. Sub-groups analyses were conducted after a significant overall effect was found, using Holm-Bonferroni correction to control for multiple comparisons. Qualitative data comprised transcripts of 40 interviews made by pharmacy student researchers—these underwent inductive thematic analysis using NVivo 11 (QSR International, Melbourne, Australia) [7]. Transcripts first underwent coding for instances of interprofessional insight and problem solving, consistent with interview questions in Table A2. Individual codes were then grouped on basis of similarity to form emergent themes, and any coding discrepancies were resolved through discussion between authors until consensus was reached. Coding and theme formation were then checked by the lead author (NM), and themes are reported with exemplary quotations.

Observations were analysed by counting the number of times students performed certain functions and cross-checking this with the quantitative and qualitative data.

3. Results

3.1. RIPLS Questionnaire

Overall students demonstrated a significant improvement in attitudes to IPL (RIPLS scores increasing from 78.78 to 82.25, p < 0.0001,

Table 1. Readiness for Interprofessional Learning Scale (RIPLS) comparisons.

	Pre-Course		Post-Course
Groups	Mean (SD)	Range	Mean (SD)	Range	p-Value
All students (n = 40)	78.78 (6.60)	62–88	82.25 (6.01)	61–91	<0.0001
Pharmacy alone	77.50 (7.74)	69–88	81.30 (9.06)	61–91	0.09
Pharmacy in pairs	79.60 (4.58)	72–86	81.50 (4.33)	74–86	0.02
Medicine alone	82.00 (5.10)	73–88	84.50 (4.84)	76–91	0.0002 *
Medicine in pairs	76.00 (7.70)	62–88	81.70 (4.99)	73–88	0.002 *
Pharmacy and medical students working alone in the VPS (n = 20) **
Domain 1—Teamwork and Collaboration (Items 1–9)
All students	40.85 (3.82)	32–45	42.7 (4.29)	27–45	0.014
Domain 2—Negative Professional Identity (Items 10–12)
All students	13.10 (1.55)	10–15	13.75 (1.62)	9–15	0.028
Domain 3—Positive Professional Identity (Items 13–16)
All students	17.65 (2.41)	12–20	18.85 (2.11)	12–20	0.001 *
Domain 4—Roles and Responsibilities (Items 17–19)
All students	8.15 (1.93)	4–11	7.60 (2.01)	4–11	0.102
Pharmacy and medical students working together in pairs in the VPS (n = 20) **
Domain 1—Teamwork and Collaboration (Items 1–9)
All students	40.75 (3.78)	32–45	43.00 (2.36)	38-45	0.006 *
Domain 2—Negative Professional Identity (Items 10–12)
All students	12.40 (1.73)	9–15	12.9 (2.51)	6–15	0.274
Domain 3—Positive Professional Identity (Items 13–16)
All students	17.15 (2.27)	13–20	18.50 (1.85)	14–20	0.0003 *
Domain 4—Roles and Responsibilities (Items 17–19)
All students	7.50 (2.04)	5–13	7.20 (2.19)	3-11	0.316

* Sub-group effects considered significant after Holm-Bonferroni correction; ** Total scores within domains vary due to each domain containing a different number of question items. Domain 2 is reverse scored.

). The increase in RIPLS scores was largely due to medical student participants, who showed significant increases in scores both when working alone with the VP and when working in interprofessional pairs (increases of 2.5 points, p = 0.0002, and 5.7 points, p = 0.002, respectively). Although pharmacy students showed an increase in scores of 1.9 points, equivalent to the p = 0.02 level, when working in pairs, this failed to reach significance after Holm-Bonferroni correction.

The RIPLS domain, Positive Professional Identity, was significantly increased in students working alone and in pairs (increases of 1.2 points, p = 0.001, and 1.35 points, p = 0.0003, respectively, Table 1). In addition, the domain Teamwork and Collaboration exhibited a significant increase in RIPLS scores in those students working together (an increase of 2.25 points, p = 0.006) but not when students worked alone.

3.2. Observations

Most (six of ten) individual medical students opted to communicate with the patient first, while only two of ten pharmacy students preferred to do so, preferring other sources of information for the initial interaction (

Table 2. Summary table of observations during the simulation exercise.

	Individual Pharmacy (n = 10)	Individual Medical (n = 10)	Interprofessional Teams (n = 20)
First investigation
Patient	2	6	5
Computer	1	2	3
GP letter	5	2	1
ECG	2	-	1
Diagnosis and treatment (Phase I)
1 attempt	6	3	1
2 attempts	4	7	7
DNC *	-	-	2
Completing the medicines chart
1 attempt	-	1	1
2 attempts	2	3	3
3 attempts	8	6	3
DNC *	-	-	3

* DNC—Did not complete. Students did not continue if they failed to rescue the patient after the second attempt, or did not complete the medicines chart after three attempts.

).

It was obvious from observations that the majority of medical and pharmacy students had difficulty interpreting the ECGs. Individual medical students struggled to complete the drug regimen on their own; while many were familiar with the names of the medicines, they had minimal understanding of their indications or dosing. All pharmacy students recognized the triple whammy drug interaction as the cause of the patient’s AKI, and those in paired simulations explained the significance of this interaction to their medical counterpart.

During observation, students often reported a lack of clinical confidence. Paired participants would discuss their thoughts with their peer before committing to their chosen plan of action. Where students stated that they lacked confidence in making a particular clinical decision, they felt more confident in being in agreement with their counterpart. In the cases where the patient died, most pairs critiqued their initial approach and discussed how their problems could be rectified before the second simulation attempt. In all but two of the paired simulations, the pharmacy student took the role of being the one to directly interact with the VP.

3.3. Interview Results

Five main themes emerged from the interview transcripts: (1) Confidence in clinical decision making, (2) teamwork—shared decision making, (3) communicating thought processes, (4) appreciation of roles and responsibilities and (5) attitudes to the simulation and IPL.

3.3.1. Confidence in Clinical Decision Making

Alone and in pairs, the majority of the medical and pharmacy students acknowledged that the pharmacy students had a better knowledge of the medicines and the diagnosis of AKI, whereas the medical students were acknowledged as better at taking a patient history. Pharmacy students voiced uncertainty in how to prioritise patient information, and from observation only two selected to “talk with the patient” as the first investigation (Table 1). In contrast, most medical students mentioned the importance of speaking with the patient first to gather clinical information.

“Often when talking to patient first they will give the best idea of how the patient is doing…It’s important to ask patient everything because they have the greatest idea of where they are with each of their conditions.

(Medical Student 8—(MS8))

On first encountering the simulated patient, a number of medical students believed the patient’s renal failure was chronic. They had not known of the drug interaction, and hence the majority subsequently struggled with the diagnosis, treatment and medication chart. Most had heard of the drug names previously but had little knowledge about giving drugs in specific instances, or in combinations.

“Drugs, I cannot do drugs because I was in a rush and I tend to forget everything, especially things that you don’t focus on when learning. For example, in medicine we don’t focus a lot on very specific [drugs], for example what we should not give if [AKI] happens.”

(MS1)

Most paired medical students admitted that they would not have had the confidence to complete the scenario on their own, and discussion with their teammate aided their own decision making, especially when the pharmacy student vocalized their opinions, which helped them to link aspects that they had learnt in class.

“I wanted to do it by myself, but I also wanted to consult someone else. I found it helpful having another person there and didn’t realise what the triple whammy was, didn’t know it was a cause of AKI. It was interesting seeing the pharmacy student’s perspective of how they wanted to undertake the investigations”

(Interprofessionally Paired Medical Student 6—(IPMS6))

“I felt relief not knowing everything and that it was okay, because I had someone else that could help to fill in the gaps in my knowledge.”

(IPMS10)

Although some pharmacy students felt they had a better knowledge of the case and believed they would have performed better on their own, many disclosed that they did not have the confidence to voice their opinions to their teammate. This often occurred due to uncertainty about the medical student’s knowledge and experience.

“If they wanted to do something different to what I thought, I went along with it because I assumed they would have more clinical experience than me.”

(IPPS8)

In some instances, pharmacy students explained that given that they knew more about the case than their medical teammate, they felt they were dominating the task, and hence would have preferred to complete the simulation alone.

3.3.2. Teamwork—Shared Decision Making

The majority of students saw teamwork as an opportunity to share decision making and responsibility. Most students working alone acknowledged that it would have been beneficial to have a student from another health discipline present during the simulation. Pharmacy students sought help from a medical student to interpret the diagnostic tests, and a peer to consult on the medication chart, and medical students desired help from a pharmacy student regarding the medicines. Some students from both groups thought a nurse would also be a helpful team member.

A number of students recognized the importance of teamwork to “ensure optimum health provision for the patient” (PS3), and an opportunity to see how others think and to bounce ideas off one another.

“I think it’s important to get other people’s views even if it is different from yours, just to see how they are thinking.”

(MS9)

“I would explain my thinking and they’d explain theirs and we would discuss ideas back and forth kind-of thing. I’d try see if any parts of their arguments are flawed and any part of mine is flawed and try figure out who’s right. The nurse and pharmacist would know their stuff and it would have made life easier for me. The first part, even if the pharmacist didn’t have input, just like listening to what I’m saying, what the patient is saying and sort of getting their own feel for it.”

(MS3)

A few students believed that working with others could (or did) save time as they could (or did) share ideas and clinical knowledge.

“It would have taken maybe half the time. I would have bounced ideas of them and got assurance if we were doing the right thing and they might confirm something that I didn’t know… Having two people the whole time would be better because it’s more accurate and a greater resource of information.”

(MS4)

“Teamwork helped a lot, there was some knowledge I was familiar with. I can see that using other health professionals would help me to solve the case quicker and is more effective”

(IPMS8)

However, from observations students working in teams often took longer to reach a shared clinical decision as they were often conscious of making the ‘right’ decision, which in some instances meant that they ran out of time debating an appropriate treatment and had to restart the simulation. Even so, this did not detract from the opportunity to share the burden of clinical decision making.

“I’d much rather have completed that as a team than alone. I think it was great to be able to bounce our knowledge off one another even if it did take us longer to treat the patient which led to their death the first time around.”

(IPPS7)

“Nice not to have full responsibility on yourself, good to have someone else with a lot of knowledge about the drugs, it brings up a lot of ideas”

(IPMS9)

3.3.3. Communicating thought Processes

To complete the scenario in teams, students felt “obliged to communicate and to express thought processes when making decisions” (IPMS2). Most students said that they were able to use their partner as a resource and that it helped to communicate their ideas with them.

“We communicated very well together and were both open to hearing what each other had to say before we made a decision…”

(IPMSI)

From observation, one team stood out from the rest in communicating their clinical decision-making processes. Both students displayed good knowledge and discussed the reflective questions in depth before answering. This helped them to realize where and why they made mistakes during the scenario.

“I think the whole thing went well because of our communication…The medical student was really knowledgeable and vocal about their opinions which made me more confident in my own actions and opinions.”

(IPPS7)

The medical student in the team understood that he needed to explain what he was thinking rather than keep his opinions to himself; something most other teams had difficulty with.

“I think that at the start we were both trying to be very polite, so that took a lot of time and probably the reason why we ended up killing the patient. It would have been more efficient had someone been assertive if they knew the knowledge.”

(IPMS10)

“Being vocal about what you’re thinking because the other person isn’t going to read your mind and if they don’t know what’s going on but you do then you are putting the patient at risk by not saying anything.”

(IPPS7)

“[I realized] how important it is to explain to others my thought process and keep them involved with what I’m thinking- especially in an emergency scenario like this one where every minute counts. If you know what’s going on, say it and don’t be afraid just because they are a doctor and you think they should know best.”

(IPPS9)

3.3.4. Appreciation of Roles and Responsibilities

Several students claimed that their opinions of pharmacists and doctors were unaffected by the VP because they had not formed pre-defined roles or expectations of either profession prior to the simulation. However, some individual medical students communicated their understanding of what their role and that of others in the healthcare team would be.

“You might be choosing the drugs and suggesting them, but at the end of day it’s the pharmacist that looks at it and goes this will work, this won’t, [and] these will interact. My role would probably be like maybe taking more of the leader and talking to the patient and talk to the nurse and bringing it all together, all their inputs.”

(MS5)

Others were less inclined to specify a specific role, or did not know what their role would be, nor what strengths others could bring to the team as they hadn’t worked with them previously.

“I’m not sure what pharmacy students know and how much they know. I’m not sure what parts are what their strengths but maybe picking out the drugs and which drugs are causing the problems.”

(MS6)

“If I was doing it with a pharmacy student I think, I can see what they know and see what they can contribute. But doing it by myself it’s hard to see what you are supposed to know heaps about and what you don’t have to know so much about or just have a general idea about.”

(MS8)

In pairs, many medical students realized that the pharmacists’ role is broader than a mere knowledge of medicines.

“I found out a little bit more about what pharmacists do, and that they don’t just dispense medicines, but actually have quite a broad knowledge of the pathology of different diseases and other skills.”

(IPMS1)

“Before this, I was not aware of what pharmacists do. I thought that they just dispense medicines. It shows that pharmacists have a much bigger role; they can work in hospitals, community pharmacies and as part of a big team. The pharmacist made me think twice about what I was saying and that it is important to identify a differential diagnosis, because there may be multiple options as cause of the underlying problems.”

(IPMS8)

Some students expressed that understanding their role or that of their colleague allowed them to “respect their place in healthcare” (IPMS4).

“I always saw doctors as the authority figure but this simulation was beneficial because it made me realise that I have a lot to bring to the team too.”

(IPPS7)

Most pharmacy students’ perceptions of the doctor’s role did not change as they felt the medical students were still inexperienced, and that the simulation was too short to give a deeper insight into each other’s final roles.

3.3.5. Attitudes to Simulation and IPL

The majority of students thought that the simulation was useful in undergraduate studies as it tested their ability to think actively, apply their knowledge under time pressure in a low stakes environment, and gave students a chance to work collaboratively.

“[The simulation] made me realise what I had done wrong and confirm the ones that I thought I had an idea about but wasn’t certain. We can make mistakes and it’s not necessary life and death…It integrates things quite nicely. If everyone had access to it, it would help build teamwork, it’s a good knowledge base to use.”

(MS4)

“This simulation will allow the increase of connections between pharmacy, medical and nursing students. It will allow students at an undergrad level to know how to work together and carry these skills over into their professional lives.”

(PS3)

Most students recognized that the simulation was a good way to encourage them to keep up to date with their reading, train them in clinical decision making and reinforce their learning, and would allow them to feel more comfortable working in an interprofessional team.

“It would just make you more comfortable working with people and you would know when to ask people things. I think you would have a better idea of going into the hospitals, when you step back, when you step forward, when someone else comes in, what they know, what you know”.

(MS5)

“This would be great after we complete our modules to review our knowledge of the medications involved as well. This would definitely help with the progress tests and make me feel more comfortable about going into the hospital environment next year. I feel much more confident about future interprofessional interactions.”

(IPMS7)

Some students thought it was a great opportunity to “share knowledge before entering practice” (IPP3) and expose students to other professionals which they “feel isolated from” (IPMS4).

“I could definitely see simulations being used in undergraduate education because we don’t really have any interactions between the other professions other than Maori Health week and Quality and Safety [Two existing interprofessional learning initiatives in our University] which are not really hands on like this…I now feel more prepared for real world interactions and think that I would be more comfortable interacting with other health professionals.”

(IPM10)

“…cases like this are very beneficial because it’s something you’re never going to forget working in an interprofessional team.”

(IPPS10)

4. Discussion

Findings from this study have demonstrated that the “Ready to Practice?” VP can lead to a significant improvement in attitudes towards interprofessional learning, and important insights into the skills, knowledge and roles of other team members. Comparisons between student groups showed the greatest change in pre–post scores occurred in medical students who worked in interprofessional pairs. Only interprofessionally paired students exhibited a significantly greater post-score in the domain Teamwork and Collaboration, while Positive Professional Identity was significantly improved in both interprofessionally paired and unpaired student groups. The questions in the Positive Professional Identity domain consider shared learning as an opportunity to improve communication, work on small group projects, share lectures, and clarify the nature of a patient’s problems. For students who worked alone on the simulation, it provided an opportunity to communicate with the simulated nurse and patient, and gave insights into how they could work together to problem solve. Students reported they felt more ready for future practice following the simulation. Participants also recounted powerful interprofessional insights in terms of the usefulness of the involvement of another profession in diagnosing and treating the simulated patient, and the relevance of these insights and experiences in terms of their future work with real patients.

From the literature we know that teamwork involves the effective communication and combined action of a group, which helps students understand the contributions of teams in building a safer workplace, and can result in better attitudes towards IPL [28]. In healthcare, this is considered to be fundamentally important in reducing the number of medical errors and improving patient safety [29,30]. While care provided by interprofessional teams may be of superior quality to that provided by a single profession [31,32,33], challenges may arise where conflicting ideas are present due to differing professional values and beliefs [34,35]. In this study, most students acknowledged the benefits of working with a student from another health discipline, but some teams found communicating their thought processes to one another was challenging due to barriers in knowledge and confidence. While pharmacy students had a good knowledge of the scenario, almost half of the pharmacy students working in a team did not have the confidence to voice their opinion. Many assumed their medical peer knew more than they themselves did, which impacted on the communication in the team and, therefore, the patient outcome, at least in their first attempt to rescue the patient. In comparison, many paired medical students expressed that they would not have confidence in completing the simulation on their own and found that by working with a pharmacy student they performed better than if they were to do the task alone. Still, the majority of students realized the importance of teamwork in a pressured environment and found it useful to have another student present to share ideas.

Medical students who completed the simulation alone and in pairs exhibited increased positive attitudes towards teamwork following the study, while pharmacy students did not. Increased positive attitudes towards teamwork following simulation are not only aided by an appreciation for other professions, but also by an enhanced perception of individual professional identity within a team [36,37], and by the realization that the limitations of any profession, at times, necessitate assistance from others [33]. Hence, such realizations may have been more profound for medical students during the VP, as considerable evidence suggests they typically have a more individualistic approach to patient care and responsibility than other allied professions [38].

It was evident in this interprofessional study that students had different levels of knowledge and, therefore, handled diagnostic and treatment decisions differently. In teams, students spent time justifying their clinical decisions to one another, which sometimes increased the time for decision making. Paired participants were subsequently more likely to exceed the time allowed in Part I of the simulation (Table 2). A contributing factor may have been the historical notion of hierarchy between medicine and other health professions [39,40,41], which can pose a barrier to shared decision making and have the potential to adversely impact the quality of care [42]. However, in this study paired students found the collaborative approach beneficial, regardless of the simulated patient outcome. There was also a marked acknowledgement of the knowledge their professional peer shared, which individual participants were not able to experience and benefit from. These results are clearly evidenced in the qualitative analysis of interviews and the significantly increased RIPLS scores in the Teamwork and Collaboration domain for paired participants (Table 1).

For pharmacy students the large variation in responses (range 61–91) meant that their increased RIPLS scores did not reach significance post-simulation. A likely explanation for this is that, apart from a few diagnostic tests which were not essential for progression through the simulation, the simulation could be completed by a Year-4 pharmacy student on their own. Pharmacy students had a good knowledge of the medicines and the interaction which was causing the AKI in the patient, could interpret laboratory values and expressed confidence in completing the medication chart. Working on their own, pharmacy students said they would have appreciated help from a medical student in interpreting the electrocardiogram; however, many medical students admitted they had not yet developed the skills to do so. In teams some pharmacy students felt they were dominating the task, and felt that they would have progressed faster had they completed the simulation on their own. In contrast, medical students were less confident in their pharmacology knowledge, did not understand what had caused the acute renal failure in the patient and had not understood the drug interaction or the drug chart. As a result, the majority of medical students working alone rescued the patient on their second attempt, compared with the majority of pharmacy students who completed this on their first attempt. All medical students could see the advantages of working with a pharmacist who they thought could help them with the medicines and their side-effects. Most students wanted someone to bounce ideas off of, to listen to what was being said and to get a feel for the situation, even though they were not always sure what the role of the other health professional would be in the team. Teams relied more on the pharmacy colleague to help diagnose the condition and select appropriate medicines for the chart in Part II of the simulation.

Effective interprofessional collaboration requires mutual understanding of the respective roles and responsibilities between health professionals [43,44]. It is, therefore, crucial for students to be able to identify their place in practice amongst their future healthcare colleagues [44,45]. In this study, some students claimed that their opinion of other healthcare profession roles was not influenced by the simulation, possibly because the VP was too short to affect their understanding of discipline-specific roles and responsibilities, or the students had not formed pre-defined ideas of their roles. In teams, most medical students expressed that they had not known much about the role of pharmacists beyond their drug knowledge, but as a result of the simulation could see opportunities for collaboration as part of a bigger team, allowing them to respect the pharmacists’ place in healthcare. In turn, pharmacy students could appreciate what they themselves could bring to the team.

The students in teams valued what they had learned from their peer, and a few students felt that earlier integration of IPL initiatives into their education could aid their rapport with their health student counterparts whom they felt isolated from. Collaborative learning in a simulated environment enables participants to appreciate the complementary skills and knowledge of their counterpart before stereotypes are formed [46,47,48]. Poorly timed integration, however, may reinforce stereotypical assumptions (both positive and negative) of one another, which may create resistance to future interprofessional exchanges [47,49].

A limitation of the study was that the current version of the “Ready to Practice?” simulation was designed for a single user, which meant that only one student could operate the computer at a time. For students who worked in pairs, this meant they were required to establish a relationship very quickly in order to complete the time-pressured scenarios. Furthermore, an error in the simulation meant that students understood the medication chart was to be completed for a patient “on discharge” rather than in the acute situation, which affected their performance. It is not clear whether these aspects had any influencing effects on student attitudes to IPL. The RIPLS scale is a measure of attitudes and perceptions and not a measure of behaviour itself [27], and may not reflect the realities of practice in a clinical environment upon graduation. Methodological strengths of our study offset some of these limitations. Our study design allowed random allocation to three independent experimental groups, and paired pre-post testing for our quantitative findings. Holm-Bonferroni corrections were used to reduce the risk of false positives, giving us greater confidence in our results. In addition, the use of mixed methods, incorporating observational data and interviews, strengthened our study design and allowed a deeper and complementary understanding of our findings.

From the individualistic nature and often poor display of attitudes towards IPL by medical students in other studies [49,50,51], it can be argued that the VP was effective in facilitating interprofessional insight and shows promise as an IPL tool. Improved teamwork has been shown to result from better attitudes towards IPL, as participants feel they understand the contributions of other team members and this facilitates communication and the building of a safer environment [28]. The findings imply that the collaborative aspects of IPL programs may play a role in improving participant attitudes towards IPL. However, as the RIPLS questionnaire was completed pre- and post-simulation, the findings may only reflect the immediate or short-term effects of the VP, and more research will be needed to confirm that the increased confidence about working in interprofessional teams reported in our participants is sustained into the real workplace.

5. Conclusions

The findings from this study provide support for using VP to improve attitudes towards IPL, and to promote interprofessional teamwork and collaboration in undergraduate medical and pharmacy students. Overall, participants showed a significant improvement in RIPLS scores after working on the VP. In addition, participants working in pairs showed significant improvements in the Teamwork and Collaboration RIPLS domain, while those working alone did not.

The VP enabled students to communicate their ideas, clarify their own role in patient care, gain insight into others’ knowledge and skills, and to see collaboration as an opportunity to share decision making and responsibility. Furthermore, the VP promoted IPL interactions that were seen by students to be beneficial in preparing them for interprofessional teamwork in a clinical setting. Currently, cost, logistical and scheduling constraints hinder undergraduate implementation of IPL initiatives in healthcare using mannequin-based simulation. Our results demonstrate that VP offers an effective and scalable solution to these difficulties at the undergraduate level.

5.1. Concluding Remarks and Integration of VPs into the Curriculum

Recent research in neurobiology recommends that education should involve active involvement, engagement, visualization and repetition [52]. By forming a connection with the learning material through more active engagement, longer lasting memories can be created for the learner [53]. Over the last twenty years, serious games and simulations have become more readily available for students in the classroom and have been shown to be capable of increasing student’s achievements, motivation and concentration [53]. Typically designed so as to complement conventional teaching rather than to replace it [53], VPs—like virtual worlds—create a space that allow for more complex interactions between the users, letting them think through problems and put their “knowledge into action” [54]. Simulation can offer considerable amounts of practice with feedback, while exposing students to realistic situations that may be impracticable or rare in real-world settings [13].

It is also fair to say that simulations “are not a silver bullet for education”, in part because they may be unfavourable for reflective learners who prefer traditional methods of teaching, but also because simulations alone will likely not change the attitudes and values towards learning for everyone [2,55]. Little research is available on how best to embed VPs in a curriculum; however, a blended learning approach is thought necessary to account for the different preferences and learning needs of students [2]. Furthermore, with the widespread use of e-portfolios in healthcare education, this may be an effective resource, where students can collect and connect content, record their learning experiences, reflect on feedback and document VP-related activities [12].

5.2. Future of R2P

Although R2P simulation is a web-based software tool, the current version of our VP requires participants to use a desktop computer in order to interact with the patient, Mr Fa’alogo. This restriction also meant that the interprofessional pairs of medical and pharmacy students in our study had to both use the same computer at the same time in order to complete the VP. We are currently planning to remove the need for co-location and synchronous operation of the VP by revising the software in order to allow the allocation of specific virtual teams of participants to the care of Mr Fa’alogo—such teams could comprise a medical student, a pharmacy student and, perhaps, a nursing student. When any member of a particular team login to the VP they will be presented with the version of Mr Fa’alogo who has undergone their care decisions, and team members may log in at the same time or at different times. Team members will be able to log in from any computer or device, potentially including mobile devices, thus, making the VP analogous to a physical place such as a hospital ward where any of the care team may visit at any time. Our plan is also to allow team members to communicate through the VP, for example, in order to allow the medical student to “page” the pharmacy student when they require assistance with prescriptions or drug chart changes. Removing the need for co-location and synchronous operation will add additional flexibility from a timetabling point of view, thus, facilitating the integration of the R2P into an already busy undergraduate curriculum. The creation of virtual teams who see different versions of the same patient also provides powerful scalability in the sense that essentially any number of teams can be interacting with the VP (and their respective version of the patient) at any given time. We also plan to develop additional simulation scenarios. These could be further care episodes involving Mr Fa’alogo, a number of years on from the first simulation, allowing the same care team to see the longer term consequences of their care (which could be either positive or negative), thus, providing potentially powerful longitudinal insights into care which would not be possible with a real patient. Additional scenarios involving further simulated patients are also planned in order to address other key aspects of learning for undergraduate students.