Introduction

Free open access medical education (FOAMed), accessible via computers and mobile devices were becoming increasingly preferred to traditional resources by trainees, even before the COVID-19 pandemic [1,2,3]. Digital medical education platforms offer exciting and innovative modalities to augment the quality and variety of didactic resources available to trainees [4]. E-learning provides the advantages of individualized, self-directed, independent, and on-demand education, and has been recognized as an efficient tool for opportunistic learning among residents [5]. In certain interactive forms, e-resources may additionally be more engaging and enjoyable than traditional formats [6], while allowing for greater global outreach and more rapid dissemination of peer-reviewed medical knowledge. The COVID-19 pandemic reshaped the delivery of medical education [7], necessitating new modes of online instruction that support asynchronous learning [8], while retaining the interactive nature of in-person learning.

Gastroenterology (GI) and hepatology are visual fields that require specialists to be proficient in recognizing pathology through endoscopic and radiographic information. The ability to apply fundamental knowledge of pathophysiology to clinical contexts and to generate appropriate differentials and management plans are critical clinical reasoning skills that trainees are expected to develop. Traditionally, these skills were imparted by high-volume patient care and instructor-led sessions; however, COVID-19-related distancing practices and limitations on elective procedures significantly reduced such opportunities for in-person training [9]. Thus far, very few online resources have offered case-based visual learning combining endoscopic, radiographic, and histologic images with hypothetical clinical scenarios to simulate real-life patient encounters.

The COVID-19 pandemic created a surge in online learners and a demand for innovative, interactive online learning tools across all specialty areas, including GI/hepatology that would support clinical reasoning skill development [10]. To address this need, we created GISIM, a free, mobile-optimized, case-based GI/hepatology educational resource designed for medical students, internal medicine (IM) residents, and GI/hepatology fellows.

Aims

(1) To describe the creation of a mobile-optimized, GI/hepatology educational resource for medical trainees, and (2) to report on trainee feedback on completing and authoring GISIM cases.

Methods

Our website, www.GiSIM.com, was created on WordPress and modeled after NephSIM (www.nephsim.com), an innovative e-learning platform created by our nephrology collaborators that features a case-based approach to teaching key nephrology topics [11].

Instructional Design

Each of GISIM’s clinical scenarios or “cases” focuses on a unique GI/hepatology-related chief complaint. Cases are designed to teach pathophysiology and disease management to trainees, while supporting active learning and the development of critical thinking and effective problem-solving skills through an interactive, case-based design [12, 13]. Hypothetical patient complaints vary in acuity level, introducing trainees to a range of potential scenarios encountered in both ambulatory and acute care settings. Cases aim to engage users through an interactive question-based format, challenging users’ process of selecting diagnostic workup and treatment options, their understanding of treatment side effects and disease course, and their approach to patient counseling, follow-up, and disease monitoring. To build a learner-oriented, user-driven resource, IM residents and GI/hepatology fellows are involved in authoring cases and selecting case topics. Key Accreditation Council for Graduate Medical Education (ACGME) Core Competencies addressed within GISIM’s content include patient care skills, as well as multiple domains of medical knowledge and practice-based learning. Cases are designed to supplement clinical training and to help trainees acquire the skills and knowledge needed to reach specific performance milestones toward mastery within the ACGME Core Competencies [14].

Content Development

Individual cases evolve sequentially, starting with history and examination details for a hypothetical patient encounter, leading to laboratory and imaging findings, endoscopy, and pathology results, and eventually ending with a final diagnosis (Fig. 1). Cases incorporate labeled histopathologic, radiologic, and endoscopic images depicting common findings, along with up-to-date diagnostic and treatment algorithms (Fig. 2). A case summary page concluding each case journey describes how the data unfolded to reach the final diagnosis; summarizes key takeaway points; and highlights references, guidelines, and resources for additional learning. Single-answer multiple choice questions are embedded throughout the case, prompting users to develop differential diagnoses and select next best steps in assessment and treatment. Real-time iterative feedback is provided for multiple-choice responses including a rationale for both correct and incorrect selections (Fig. 3).

Fig. 1
figure 1

Schematic illustration of a sample case

Full size image
Fig. 2
figure 2

Sampling of infographics, radiologic, histologic, and endoscopic images from GISIM to enhance trainee learning

Full size image
Fig. 3
figure 3

Sample case demonstrating sequential nature, with interactivity and instant feedback on choices made with detailed reasoning

Full size image

Cases were drafted on a voluntary basis by the Icahn School of Medicine at Mount Sinai GI/hepatology faculty, fellows, and IM residents in collaboration with faculty from the Departments of Pathology and Radiology. A volunteer team of three GI/hepatology faculty members with active experience as medical educators assisted with case development and review. Case topics were proposed by case authors and were selected in conjunction with an overseeing GI/hepatology faculty member. Bloom’s taxonomy model [15] (Fig. 4) was then used to establish case objectives. Cases were drafted independently by case authors using a standardized case template created by GISIM faculty members. Drafts were subsequently reviewed by an overseeing GI/hepatology faculty member for content accuracy and case complexity with the goal of promoting higher-order thinking and advancing cognitive learning beyond knowledge recall to analysis, evaluation, and application [13]. Endoscopic and radiographic images and histologic slides were provided by case authors or collaborating faculty members. Once finalized, cases were uploaded to GISIM’s website by a team of two volunteer IM residents.

Fig. 4
figure 4

Learning objectives for the website using Bloom’s taxonomy model

Full size image

Content Distribution

GISIM (www.Gi-SIM.com) was launched in February 2021 with four cases. Website information was disseminated to IM residents, GI/hepatology fellows and attendings across the Mount Sinai health system via institutional email subscriber lists, Twitter (@GISIM_website), and to the GI/Hepatology divisional webpage. New cases were uploaded to GISIM on a near-monthly basis and a Twitter alert was posted to notify social media followers. A total of ten cases were available on GISIM by February 2022 when this manuscript was compiled.

Content Evaluation

WordPress analytics were used to track GISIM website visitor and viewership numbers. Website user and case author surveys were developed on Google Forms. Website users accessed the user survey through a link embedded into the summary page of each case. User surveys evaluated users’ demographic information and GISIM experience with questions on website usability, content quality, and difficulty, as well as perceived educational value of cases, corresponding to Level 1 on Kirkpatrick’s evaluation model [16]. Users were also asked about their likelihood of completing additional GISIM cases and recommending the resource to peers. The user survey included 14 questions (six closed-ended multiple-choice questions, seven Likert-scale questions with four responses, and one open-ended question for general feedback). A separate survey was sent to case authors via email. The author survey included 22 questions (five closed-ended multiple-choice questions, 12 Likert-scale questions with four responses, four five-point scale questions, and one open-ended question for general feedback), evaluating case authors’ demographics and GISIM experience, as well as their perceived educational value of case authorship. Surveys were developed by GISIM team members (two IM residents and three GI/hepatology faculty). Survey participation was voluntary and all survey questions were optional.

Google Forms’ integrated analytics software was used to anonymously aggregate and analyze survey responses. Case completion rate was calculated as a ratio of case introduction page views to case summary page views. Website user survey response rate was calculated by dividing the total number of completed user surveys by the total number of completed cases (approximated by the number of case summary page views).

Results

GISIM website/Twitter analytics and survey responses were collected from February 1 2021, when GISIM was launched, until February 28 2022, when data was analyzed for this manuscript.

Website Usage and Twitter Analytics

During the analysis period, GISIM had 12,184 website views and 2003 unique visitors from 76 countries. Views were primarily from users in the USA (64%), followed by India (8%) and Canada (3%). Case completion rate was 45%. GISIM’s Twitter account had 119 followers and nearly 3000 views.

Survey Results

Website User Surveys

Sixty-one user surveys were collected and 629 cases were completed, corresponding to a survey response rate of approximately 10%. First-time SIM-series users contributed to 80% of the submissions. Respondent training level was varied (Table 1). GI/hepatology fellows submitted the majority of responses (38%), followed by IM residents (26%), attendings (21%), others (8%), and medical students (7%).

Table 1 Composition of user survey respondents based on training level
Full size table

Most users completed cases in < 5 min (32%) or 5–10 min (53%), without a clear correlation between completion time and training level. Case length was reported as “just right” by 92% of users, suggesting overall satisfaction with the level of detail and time required for completion. Users’ evaluation of GISIM’s website and cases are shown in Fig. 5. All users found the website easy to use. Ninety percent of users agreed that cases were interactive. Sixty-seven percent of users agreed that cases enhanced their confidence and 86% agreed that cases improved their understanding of selected topics. Eighty-three percent of users reported that they would use the resource again and 87% agreed with the statement that they would recommend it to their colleagues. A subgroup analysis of nonattending users showed largely similar trends in results (Fig. 6).

Fig. 5
figure 5

Feedback on website from user surveys

Full size image
Fig. 6
figure 6

Feedback on website from user surveys, excluding attending responses

Full size image

Case Author Surveys

Author Demographics

Nine author surveys were collected. Surveys were completed by two attendings, two fellows, and five residents.

Evaluation

Primary motivators for author participation included an interest in medical education and a desire to contribute to GISIM as a learning resource. Authors primarily selected topics that they wanted to learn more about (78%) and felt would be useful to others (100%). After writing cases, authors uniformly indicated feeling “comfortable” obtaining history/examination, ordering/interpreting diagnostics, and following guideline-directed management within chosen topic areas, representing an improvement from prior to case authorship (Table 2). All authors intended to incorporate GISIM into future teaching sessions with trainees, and the majority planned to volunteer to write additional cases and intended to recommend the authorship opportunity to their peers (Table 3). Most authors felt the experience provided significant learning (75%) and had high utility (100%).

Table 2 Feedback on case development experience as solicited by author surveys
Full size table
Table 3 Authors’ plans for future use and involvement
Full size table

Discussion

The unique strains on trainee education created by the COVID-19 pandemic have highlighted the importance of effective virtual learning platforms [17, 18]. While a variety of self-assessment modules, video and PowerPoint presentation libraries, question banks, and weekly webinars are offered by GI/hepatology societies for trainees [17], the majority of these resources support passive learning and focus heavily on knowledge recall, with limited opportunities for application-based learning. Clinical reasoning, data synthesis, and evidence-based practice are emphasized as important competencies by the ACGME [14]; however, imparting these skills has traditionally required intensive, in-person, preceptor-led approaches. GISIM uses an interactive case-based format, which enables integration of multifaceted and interdisciplinary curricular concepts to support critical thinking skill development and to enhance retention of basic science knowledge [19]. Survey data demonstrates that GISIM modules strengthen trainees’ clinical confidence and decision-making, suggesting GISIM to be a potentially valuable e-learning resource for both case authors and website users.

GISIM serves a global audience and provides asynchronous, on-demand distance GI/hepatology learning. Survey data shows that GISIM subjectively improves users’ understanding of covered topics and provides a perceived enhancement in medical knowledge. Author surveys illustrate a reciprocally beneficial experience for case writers as well, with authors overwhelmingly reporting that writing cases enhanced clinical confidence across the continuum of patient care from obtaining a history and physical to developing a management plan for the particular topic. While the significance of authors’ enhanced clinical confidence is difficult to quantify, their notably positive feedback about the utility and learning provided by the authorship experience as well as their intention to volunteer for additional authorship opportunities indicate that the experience was at least subjectively beneficial for this group. Additionally, our observation that the majority of authors selected topics they wished to learn more about, suggests that the process of case authorship itself can promote independent and self-directed, self-guided learning.

A unique aspect of GISIM is its foundation in peer-to-peer or peer-assisted (PAL) learning, an approach in which students take on the role of tutor and trainee interchangeably. GISIM case authors and website users are primarily residents and fellows, with similar average training levels between groups. With regard to medical education, PAL has been shown to improve teaching skills, reinforce prior knowledge, and enhance academic performance; and has been shown to be a particularly effective tool to support independent learning [20]. In the context of GISIM, the PAL approach enables learners to focus on topics that have been identified as high yield and training level appropriate by their peers authoring the cases.

GISIM’s mobile-optimized platform offers a portable resource for residents and fellows to incorporate into impromptu bedside teaching sessions, enhancing the untapped potential of trainees as front-line educators [21]. GISIM modules may additionally be used to prepare for in-person didactic sessions, to review specific content areas after morning reports, or to augment gaps in clinical exposure during rotations. To further promote on-the-go learning and optimize student engagement [22], cases were kept short and concise with the majority taking less than 10 min to complete. GISIM’s Twitter page provides an open line of communication with current and prospective followers and is key to expanding the website’s reach and user base. Our Twitter page has received feedback through tweets and direct messages from national and international trainees stating that they found the website useful and shared/retweeted a link with their colleagues. Faculty from outside institutions have also provided feedback through Twitter and email that they plan to incorporate GISIM into teaching sessions. Both trainees and instructors have offered to contribute to GISIM and several have written cases for the website, demonstrating the importance of social media in the resource’s continued growth.

Several of our study’s limitations arise from our method and mode of data collection. Relying on self-reported survey data instead of objective knowledge assessments could compromise internal validity of the study. Furthermore, given that survey completion was optional, selection bias could have skewed survey completion towards more positively impacted users. Our relatively low user response rates (10%) present another limitation of our study given the potential impact on generalizability; however, it must be pointed out that optimum response rates are a matter of ongoing debate, with some researchers suggesting rates of 5–10% still provide reliable results for sample sizes of at least 500 [23]. A few respondents evaluated multiple cases in a single survey, creating the potential for recall bias, due to delays between case completion and assessment. GISIM usage and viewership was also possibly overestimated by website host analytics.

User experience is critically important to the success of online learning resources, and was a central focus of this study. While this study only provides a Kirkpatrick level 1 assessment of our learning tool by evaluating user satisfaction, additional Kirkpatrick levels will be evaluated in subsequent phases of our study. In our next phase, we will introduce GISIM cases into medical student workshops and IM residents’ outpatient didactic sessions and will use pre- and posttest surveys to objectively measure knowledge acquisition (Kirkpatrick level 2). To guide future case development, we will add a question to our online user survey inquiring about how the resource is being used (i.e., bedside teaching tool, on-the-go tutorial, or traditional computer-based learning).

In summary, we have developed a novel GI/hepatology educational resource that supports on-the-go learning and caters to learners at multiple levels of training. Availability of this case-based resource on an open-access website could enable independent, self-paced learning, and clinical reasoning development both during and after the pandemic.