Introduction

Over the past decade, the importance of collecting patient-reported outcomes (PROs) alongside objective data regarding cancer status has gained widespread recognition amongst clinical oncologists and interdisciplinary cancer teams [1, 2]. PROs provide a measure of the value of investigations and therapies to the individual patient, assisting with decision-making in the acute phase of cancer management [3,4,5]. PROs also indicate the burden of symptoms, both physical and psychosocial, felt by the cancer survivor [6,7,8]. The Edmonton Symptom Assessment Scale (ESAS) is a validated self-reporting tool of symptom severity for nine common symptoms of advanced cancer that accurately captures PROs including a variety of physical symptoms (pain, tiredness, nausea, drowsiness, appetite, wellbeing, and shortness of breath) and psychological distress (depression and anxiety), with the option of adding a tenth patient-specific symptom [9,10,11]. There is good evidence that PROs captured through ESAS completion more accurately depict symptom burden than do interrogation and documentation by a clinician [12, 13]. In view of this, completion of the ESAS by all patients attending cancer clinics has been mandated in many jurisdictions, including the Province of Ontario, Canada [14, 15].

Prior to March 15, 2020, ambulatory patients seen at Princess Margaret Hospital, Toronto, completed the ESAS at each clinic visit. Compliance exceeded 85% at our institution in the immediate pre-COVID era. Patients used touchpads provided by the clinic to enter their scores for each symptom category, using an 11-point scale from 0 (no symptom) to 10 (worst possible symptom) [16]. Patients who had any difficulty navigating the platform, for any reason, were assisted in person by clinic volunteers. This direct electronic patient entry then prompted a real-time review by clinicians. Scores that indicated clinically significant symptoms (≥2 for depression; ≥4 for all other symptoms) triggered follow-up by the primary oncology team and referral as per Cancer Care Ontario practice guidelines to the appropriate health care providers, for example, a psychiatrist specializing in the care of oncology patients [17]. Similar systems have been implemented across cancer clinics in Canada and the US [18,19,20,21].

The advent of the COVID-19 pandemic dramatically altered the care of cancer patients worldwide. Early reports implicating visits to the cancer center in the transmission of the SARS-Cov2 virus to vulnerable cancer patients [22, 23] triggered severe measures to curtail in-person clinic visits. This resulted in the widespread adoption of virtual assessments, both for initial consultations and for follow-up visits. For patients who did attend ambulatory clinics in person, many restrictions were immediately implemented. In particular, at our institution touchpads were removed from clinics. Collection of PROs via ESAS completion, so recently championed and supported, was abandoned. While this precaution was at the time understandable given the concern for patient safety, we wondered what impact these physical safety measures might have on our ability to detect significant psychological distress in cancer patients who were already facing delays in investigation and treatment, as well as alterations in routine surveillance protocols. In an attempt to fill this gap, we introduced virtual ESAS completion, but in an initial cohort of patients found that implementation was not successful. After demonstrating near-universal completion of hard-copy ESAS by patients visiting a clinic in person, and identifying barriers to receipt and completion of virtual ESAS forms, we developed a personal virtual mentoring program and showed that this significantly improved completion rates. However, for many patients, barriers remain with respect to handling electronic documents. We also here report the experience of severe levels of psychosocial distress in ~50% of ambulatory cancer clinic patients amidst the COVID-19 pandemic.

Methods

The primary objective was to evaluate the rate of ESAS form completion through virtual means. Secondary objectives were to determine if associations existed between patient characteristics and virtual ESAS questionnaire completion, and to identify barriers to the virtual collection of PROs. This study was approved by the Princess Margaret and Mount Sinai Hospital REBs.

ESAS questionnaire

The ESAS [9] is a validated, brief, practical, and comprehensive self-reporting tool of symptom severity for nine common symptoms of advanced cancer (pain, tiredness, nausea, depression, anxiety, drowsiness, appetite, wellbeing, and shortness of breath), with the option of adding a tenth patient-specific symptom; higher scores represent worse symptom intensity [24]. A fillable PDF version using the Cancer Care Ontario (CCO) form (cancercareontario.ca), with clickable buttons associated with the numerical scale values for each symptom, was generated for virtual dissemination. Only page 1 of the form was used for the present study. Overall assessment of symptom severity was generated by adding individual symptom scores to calculate the ESAS total symptom burden score. The total symptom burden score and individual symptom scores were classified as absent (0), mild (1–3), moderate (4–6), and severe (7–10) using defined cutoffs from the literature, with clinically significant scores defined as scores of >30 (total symptom score), ≥4 (individual symptom scores), and ≥2 (depression symptom score) [25,26,27,28].

Study population

Consecutive patients attending a virtual or in-person appointment at a mixed cancer site surgical oncology clinic were approached for inclusion in this study, beginning in May 2020. Our cancer center had by then undergone a major transition to virtual clinics due to the COVID-19 pandemic. Patients had to be ≥18 years and able to provide informed consent. Patients were enrolled into three sequential cohorts according to the platform used for ESAS completion.

For the first cohort (virtual), patients were contacted by phone shortly after their appointment, whether it was virtual or in-person. Their permission to email the fillable PDF ESAS form was requested, and they were asked to return the completed form electronically. Patients who could not be reached with an initial phone call were called up to 2 more times. The ESAS form was emailed to the address provided by the patient. If the form was not completed and returned within one week, an email reminder was sent.

For the second cohort (hard-copy), patients who attended in-person appointments were asked to complete a hard-copy ESAS form in clinic. The completed form was collected at the time of the appointment.

For the final cohort (virtual-mentored), a research team member provided remote support to facilitate ESAS completion. Patients were approached at the time of virtual or in-person clinic visits, and those who agreed to participate provided an email address and phone number at which they could be contacted. On the same day, they were sent an email with a clickable PDF attachment of the ESAS survey and a note that explained that they would receive a follow-up call to discuss their experience with the survey. Patients were called within 3 days of sending the survey. If they had successfully completed and returned it, the caller explored their experience in completing the survey and took the opportunity to address any elevated scores on the ESAS form. The caller also asked all patients if they encountered any barriers to completion and to describe these. Any symptom concerns that were not already being actively addressed by health care professionals were brought to the attention of the most responsible treating physician, and this prompted a follow-up visit. Two patients were referred to psychosocial oncology for unaddressed anxiety/depression. If they had not completed the survey, the caller reminded them to complete it and verified the contact email address. Up to 2 additional reminder calls were made.

All patients in the virtual-mentored cohort were asked about barriers to electronic completion and return of the clickable PDF. Patients who stated that they did not receive it were asked to check spam or junk mailboxes, and the survey was resent. Messages were not left on an answering machine/voice mail or with a third party.

A total of 6 patients (4 in the virtual and 2 in the virtual-mentored cohorts) required their responses to be transcribed by a research team member over the phone, as they were unable to understand how to complete the PDF and submit it. In the virtual-mentored cohort, 10 out of 58 patients who completed the ESAS questionnaire did so by means of a hard-copy form that they obtained in clinic at their request. The 16 patients who did not comply with the conditions stipulated for their assigned cohort were analyzed within that assigned cohort on an intention-to-treat basis. We also performed sensitivity analyses that eliminated these patients from their assigned cohorts.

Statistics

Summary statistics were used to describe the demographic and clinical characteristics of the study population. Statistical significance was assessed by Student’s t-tests for continuous variables and Chi-squared (Fisher’s exact) test for categorical variables using Prism software (GraphPad Software, La Jolla, CA). A p-value <0.05 was considered significant. Error bars reflect median and IQR.

Results

Description of study participants

A total of 174 patient visits to a complex surgical oncology ambulatory clinic between May 27 and July 29, 2020 were included in the study (Table 1). The median patient age was 62 (range 19–90) and 53% were female. The primary tumor site was gastrointestinal in 31%, soft tissue in 48%, and “other” in 21%. Recruitment into the three cohorts (virtual, hard-copy, and virtual-mentored) was sequential based on clinic date, with cohort size adapted to the observed ESAS completion rate as the study proceeded (see Methods). Age, sex, and tumor site did not differ between the three cohorts (Table 1). We further characterized patients according to the phase of their cancer journey at the time of the study: 41% of patient visits were within a phase of active investigation or treatment, 50% were in surveillance following treatment, and 9% were receiving symptom-directed care for cancer that was not considered curable (palliative). There was a significant difference in the distribution of management phase between the cohorts. Patients in the hard-copy cohort were more likely to be undergoing active investigation/treatment (54%) or receiving palliative measures (20%) and less likely to be in surveillance (26%) than those in either the virtual or virtual-mentored cohorts (p < 0.05, χ2).

Table 1 Patient clinical and disease characteristics
Full size table

ESAS completion rates

Of the 174 ambulatory visits, 127 culminated in ESAS completion by the patient; three patients formally declined to complete the ESAS form (Fig. 1). At the start of the study, a research team member contacted the patient by phone shortly after their virtual or in-person visit with the physician. The ESAS form was then emailed to an address provided by the patient. For the first cohort (virtual), 46 patients were approached over a three-week interval, 15 of whom did not respond after up to 3 phone calls and 3 emails. One patient declined to complete the form (Supplemental Table 1). The ESAS form was emailed to the remaining 30 patients, together with instructions to complete and return the form electronically. While 20 patients did complete the survey, 4 of these requested to complete it via phone transcription with a member of the research team, as they stated that they had difficulty handling the PDF. The overall completion rate for this virtual cohort, analyzed by “intention to treat”, was 44%, and ~60% of the non-completions reflected an upfront lack of response from the patient. Postulating that the low completion rate might be related to the virtual format and remote relationship between study participants and the research team, we reverted to the hard-copy format initially employed when ESAS forms were first rolled out at our institution in 2007. Over the next 3 weeks, 50 patients who visited the clinic in person were asked to complete a printed ESAS form (hard-copy cohort). A total of 49 (98%) agreed and did so; one patient agreed to complete the form, but deferred to the end of the appointment and did not return it.

Fig. 1
figure 1

Diagram depicting study work flow and patient response over the study period

Full size image

Given the notable discrepancy in completion rate between the two initial cohorts, we speculated that significant barriers to patient handling of the virtual ESAS form might have existed. We, therefore, created an intervention, offering patients “virtual mentoring” to assist with the practical aspects of questionnaire completion and return. For this virtual-mentored cohort, 78 patients who attended the clinic in-person or virtually were approached over a 4-week interval. 2 formally declined to complete the form, and 18 of the 76 who had originally agreed to complete the form did not ultimately return a completed ESAS, 2 returning a blank form, and 16 not returning a form despite the verbal offer of mentorship. Of the 58 patients (74%) who successfully completed the form, two requested to do so via phone transcription, dictating their responses to the research team member, and ten requested to complete a hard-copy form in the clinic (Fig. 1). If we eliminate these 10 patients from analysis, the re-calculated completion rate for the virtual-mentored cohort is 71% (48/68).

We queried patient- and disease-related variables that might be associated with ESAS completion. Analysis of all patients in the three cohorts grouped together showed that completion was not predicted by age, sex, or primary tumor site (Table 2); this remained the case when patients who did not comply with the platform assigned to their cohort (n = 16) were eliminated from the analysis. Patients who did not complete were less likely to be under active investigation/treatment than those who did (28% vs. 46%, p = 0.04, χ2). Analysis of the subgroup of patients included in the virtual and virtual-mentored cohorts (n = 124) also showed that questionnaire completion was not predicted by age, sex, or primary tumor site, nor was it predicted by management phase (Table 3). Again, sensitivity analyses with the 16 non-complying patients excluded yielded a similar result.

Table 2 Patient characteristics by ESAS completion for all cohorts
Full size table
Table 3 Patient characteristics by ESAS completion for virtual and virtual-mentored cohorts
Full size table

ESAS patient reported data

Of the 127 ESAS forms completed by patients in all three cohorts, 117 (92%) reported at least one symptom score of ≥1, indicating the presence of a symptom. The overall responses for each symptom scale are shown in Fig. 2A, categorized by symptom severity. The symptom with the highest prevalence of clinically significant severity was depression (42% had a score of ≥2), followed by wellbeing (35% ≥4), tiredness (31% ≥4), and anxiety (27% ≥4). The ESAS total symptom burden score (>30: moderate-to-severe vs. ≤30: absent-to-mild), as another measure of clinically significant distress, showed that 20% of respondents (26/127) had a moderate-to-severe total symptom burden (Suppl Table 2).

Fig. 2
figure 2

Distribution of ESAS symptom scores in study patients. a Proportion of patients with ESAS scores in the following severity categories: unknown, no symptoms (0), mild-to-moderate symptoms (1–3), moderate-to-severe symptoms (4–6), and severe symptoms (7–10), by symptom type, in the total cohort of 127 patient encounters that resulted in ESAS questionnaire completion. Depression score categories (right) were as follows: unknown, no symptoms (0), mild symptoms (1), severe symptoms (2–10) [27]. b–e Distribution of patient scores for depression (b), well-being (c), tiredness (d), and anxiety (e), displayed according to assigned platform for ESAS completion. Each patient is represented by a dot, asterisk or triangle. A clinically significant level of distress was defined a priori as score ≥2 for depression, and ≥4 for the other symptoms, and is indicated by the horizontal dashed line. There was no difference in the degree of distress reported by patients in the three ESAS platform groups, p = NS. Error bars: median and IQR

Full size image

There was no difference in the ESAS total symptom burden score reported between the three cohorts (>30 in 20%, 22%, and 19% of virtual, hard-copy, and virtual-mentored cohort respondents, respectively). A sensitivity analysis was performed by crossing the 10 patients in the virtual-mentored cohort who actually completed hard-copy forms over to the hard-copy cohort, and this showed similar results (>30 in 20%, 20%, and 21% of virtual, hard-copy, and virtual-mentored cohort respondents, respectively). Similarly, there were no significant differences in individual symptom scores reported for each cohort, as illustrated by the distribution of scores for depression, wellbeing, tiredness, and anxiety (Fig. 2B–E) and the remaining individual symptoms (Supplemental Figure). High total symptom burden was not predicted by age, sex, or management phase, although patients with high total symptom burden were more likely to have a primary GI tumor (50 vs. 28% of those with low symptom burden, p = 0.02, χ2) (Suppl Table 2).

In nine patients (2 virtual; 1 hard-copy; 6 virtual-mentored), a review of symptom severity scores by a member of the research team prompted follow-up conversations to validate and clarify the reported symptoms. Six patients were having significant, poorly controlled physical symptoms, and with patient consent, this was directly communicated to the most responsible physician by a member of the research team. Three patients who had high anxiety and depression scores were questioned further: one had an established relationship with a psychiatrist, and two were offered referral to psychosocial support services at the cancer center. In particular, a review of the ESAS forms triggered an intervention in 4 of 6 virtual-mentored patients with high scores.

Barriers to virtual completion of patient reported outcome questionnaire

Of 106 visits that yielded an agreement by the patient to virtually complete and return an ESAS form, 28 did not culminate in completion (10/30 and 18/76 in the virtual and virtual-mentored cohorts, respectively). Some patients in the virtual cohort stated that they were unable to open, complete or save the PDF, and others reported generalized “technology-phobia” or email handling issues. Of the subgroup of patients in the virtual-mentored cohort who completed the form entirely virtually (46), the majority (n = 32) felt it was a straightforward process and had no difficulty with completing it independently, while six individuals noted time constraints or email issues as a barrier to prompt completion; the few individuals who had challenges with handling the PDF (n = 8) were able to overcome them with the help of a research team mentor. In the group of individuals assigned to the virtual-mentored cohort who requested and completed a hard-copy form during their clinic visit, the main explanation they provided was that this would be a more straightforward and efficient process for them, with 3 of 10 noting that they did not have an email address.

Discussion

This study demonstrates the existence of significant barriers to the virtual collection of PROs in ~40% of patients attending a complex surgical oncology clinic. Explicitly stated and inferred barriers were partially addressed by the provision of personalized remote mentoring to facilitate successful electronic handling of the fillable form by patients. The latter required a considerable investment of time outside of the clinic, as a patient care team member needed to follow up with each patient by phone after the clinic appointment to support ESAS completion by addressing their individual barriers. The resources required would be justified if virtual-mentored ESAS completion revealed a high incidence of actionable symptom burden, which according to the results described here, it indeed did.

Importantly, ESAS forms completed during the first wave of the pandemic showed a high rate of severe psychosocial distress amongst ambulatory patients being treated/followed through the cancer clinic. Anxiety regarding treatment delays and depression related to social isolation have been cited as COVID-related sources of psychological distress that persist even now [29,30,31]. The capacity to not only identify significant symptoms but also support the patient experience by virtual means is challenging to build, particularly with resources diverted to pandemic-related processes and care. Nevertheless, this capacity is critical to optimal oncologic management of the whole patient. A body of research first disseminated in the palliative care literature clearly demonstrates that during a clinical encounter in the ambulatory clinic, treating clinicians commonly overlook symptoms that are impinging significantly on a patient’s quality-of-life (QoL) [6, 32, 33]. Prospective studies have shown that patient satisfaction and QoL are objectively improved by routine PRO capture in cancer centers [5, 34]. In the early 2000s, the evidence-based movement to measure, review, and target PROs gained sufficient strength that this practice had become standard in major cancer centers prior to the COVID-19 pandemic.

Within the province of Ontario, Canada, routine administration of the ESAS questionnaire was implemented in ambulatory clinics at all regional cancer centers in 2007 [11]. Implementation expanded to include many of the partner community hospital sites, and ESAS data were integrated into many of the local EMRs. In 2015, the Ontario Cancer Plan IV specifically stressed the importance of PROs. Real-time electronic capture of PROs at each clinic visit was effected by directing the patients to a computer workstation or tablet following registration. The output was then given to the clinical team to use within or after the clinical encounter with the physician. Both provider-facing and patient-facing symptom management guides for each symptom found in ESAS were used as guidance for the clinical care team as to how to respond to elevated symptom scores (www.cancercareontario.ca/en/symptom-management).

Even in the pre-pandemic era, various barriers to ESAS/PRO completion were described. These include insufficient time, challenges with language fluency/literacy, misinterpretation of ESAS terms, difficulties in rating current symptom level, and technology issues including access/proficiency [35,36,37]. As in many cancer centers pre-pandemic, we provided patients with on-site personal support from clinic staff and/or volunteers, to enhance PRO completion.

Sweeping changes were quickly implemented within North American ambulatory cancer clinics as the first reports of high rates of Sars-Cov2 transmission and serious illness in cancer patients emerged from centers in China and Italy [22, 38, 39]. At our center, as at many others, routine ESAS completion was abandoned for over 18 months. The high rates of severe depression and anxiety revealed by the ESAS administration we conducted here, whether virtual or hard-copy, confirm the value of this tool and highlight the importance of continuing to collect PROs during turbulent times. Furthermore, though we had conjectured that high symptom burdens would be seen largely in the perioperative period and in patients undergoing other active investigation/treatment, our results indicate that neither age, gender, diagnosis, or treatment stage were predictive of symptom burden. Again, the value of PROs was demonstrated and must be championed as a priority.

While the remote mentoring program we implemented as part of this study did result in improved ESAS completion rates, barriers remained. Within the framework of this study, it was not possible to ascertain what barriers, if any, were encountered by non-responding patients. Health literacy and/or language proficiency may have played a role [35,36,37, 40], but these variables were not captured in our study. We acknowledge that we may have missed PROs on the most vulnerable of our cancer patients amidst the COVID pandemic [41,42,43]. Other barriers to virtual care cited by patients and providers have included poor reliability of caregiver assessments, provider burden, and lack of understanding regarding the timeframe of assessments [35, 37].

Virtual mentored ESAS form completion allowed patients to have an opportunity to report on symptoms after their clinic visit. For many patients, this was a brief interaction, and the form was easy to complete. A unique component of this format was that it allowed for a second interaction after the clinic visit and therefore some unaddressed or persistent symptoms or concerns could be brought to the attention of the care team, allowing for immediate action to be taken. The organization and conduct of the virtual mentoring interaction could be time-intensive. All patients required at a minimum one phone call, but some had several reminder phone calls. Although most calls were only 1–2 min in length, for those with questions or concerns, these calls took up to 10 min. In this study, a physician was completing the calls and was able to answer questions, however, this is a resource that is not always readily available.

The high completion rate observed with in-person hard-copy ESAS questionnaires suggests that time and accessibility were factors; time spent waiting for the physician in the clinic was readily directed toward form completion. The simplicity and efficiency of this now obsolete platform were also cited by patients who were assigned to be virtually mentored but instead requested and completed hard-copy forms. We additionally note the significant minority of patients (6/124) who did not wish to participate in virtual form completion, regardless of the availability of mentoring, but were willing to be guided through the form and convey their responses verbally to a team member. This indicated a degree of “technology-phobia” that did not vary with age or sex. While physicians, nurses, and hospital administrators may feel comfortable with electronic platforms [44], it is incumbent upon us to recognize that not all of our patients do. This recognition should inform planning for the future of virtual care, currently ongoing at the College of Physicians and Surgeons of Ontario and other health care agencies. Further qualitative research could enhance our understanding of what limits virtual completion, facilitating improvement. Our study demonstrates that if we want to mandate PRO completion despite virtual care, some patients will require mentoring, meaning that additional resources are required.

Conclusions

We found significant challenges in the remote electronic completion of ESAS PROs by patients during the COVID-19 pandemic, with improved completion rates achieved by providing virtual mentoring. The severe degree of psychological distress reported by ~50% of respondents demonstrates the need for ongoing regular collection of these data. Increased resources and innovative strategies are required to improve compliance with virtual PRO completion in order to maintain high quality oncology patient care.