Skip to main content

Advertisement

SpringerLink
Log in

Usability, acceptability, and implementation strategies for the Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm: a Delphi study

  • Original Article
  • Published:
Supportive Care in Cancer Aims and scope Submit manuscript

Abstract

Introduction

Oncology guidelines recommend participation in cancer rehabilitation or exercise services (CR/ES) to optimize survivorship. Yet, connecting the right survivor, with the right CR/ES, at the right time remains a challenge. The Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm was developed to enhance CR/ES clinical decision-making and facilitate access to CR/ES. We used Delphi methodology to evaluate usability, acceptability, and determine pragmatic implementation priorities.

Methods

Participants completed three online questionnaires including (1) simulated case vignettes, (2) 4-item acceptability questionnaire (0–5 pts), and (3) series of items to rank algorithm implementation priorities (potential users, platforms, strategies). To evaluate usability, we used Chi-squared test to compare frequency of accurate pre-exercise medical clearance and CR/ES triage recommendations for case vignettes when using EXCEEDS vs. without. We calculated mean acceptability and inter-rater agreement overall and in 4 domains. We used the Eisenhower Prioritization Method to evaluate implementation priorities.

Results

Participants (N = 133) mostly represented the fields of rehabilitation (69%), oncology (25%), or exercise science (17%). When using EXCEEDS (vs. without), their recommendations were more likely to be guideline concordant for medical clearance (83.4% vs. 66.5%, X2 = 26.61, p < .0001) and CR/ES triage (60.9% vs. 51.1%, X2 = 73.79, p < .0001). Mean acceptability was M = 3.90 ± 0.47; inter-rater agreement was high for 3 of 4 domains. Implementation priorities include 1 potential user group, 2 platform types, and 9 implementation strategies.

Conclusion

This study demonstrates the EXCEEDS algorithm can be a pragmatic and acceptable clinical decision support tool for CR/ES recommendations. Future research is needed to evaluate algorithm usability and acceptability in real-world clinical pathways.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of data and coding

Data and coding available upon request.

References

  1. Mctiernan A, Friedenreich CM, Katzmarzyk PT et al (2019) Physical activity in cancer prevention and survival: a systematic review. Med Sci Sports Exerc 51:1252–1261

    Article  Google Scholar 

  2. Campbell KL, Winters-Stone KM, Wiskemann J, May AM, Schwartz AL, Courneya KS, Zucker DS, Matthews CE, Ligibel JA, Gerber LH, Morris GS, Patel AV, Hue TF, Perna FM, Schmitz KH (2019) Exercise guidelines for cancer survivors: consensus statement from international multidisciplinary roundtable. Med Sci Sports Exerc 51(11):2375–2390. https://doi.org/10.1249/MSS.0000000000002116

    Article  PubMed  PubMed Central  Google Scholar 

  3. Avancini A, Pala V, Trestini I et al (2020) Exercise levels and preferences in cancer patients: a cross-sectional study. Int J Environ Res Public Health 17:5351. https://doi.org/10.3390/ijerph17155351

    Article  PubMed Central  Google Scholar 

  4. Covington K, Marshall T, Campbell G et al (2021) Development of the Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm. Support Care Cancer 29:6469–6480. https://doi.org/10.1007/s00520-021-06208-7

    Article  PubMed  Google Scholar 

  5. Sleight AG, Gerber LH, Marshall TF et al (2022) A systematic review of functional outcomes in cancer rehabilitation research. Arch Phys Med Rehabil. https://doi.org/10.1016/J.APMR.2022.01.142

    Article  PubMed  Google Scholar 

  6. Patel AV, Friedenreich CM, Moore SC et al (2019) American College of Sports Medicine roundtable report on physical activity, sedentary behavior, and cancer prevention and control. Med Sci Sports Exerc 51:2391–2402. https://doi.org/10.1249/MSS.0000000000002117

    Article  PubMed  PubMed Central  Google Scholar 

  7. Faro JM, Mattocks KM, Mourao D et al (2021) (2021) Experiences and perceptions of referrals to a community-based physical activity program for cancer survivors: a qualitative exploration. BMC Heal Serv Res 211(21):1–10. https://doi.org/10.1186/S12913-021-06365-9

    Article  Google Scholar 

  8. Hardcastle SJ, Maxwell-Smith C, Kamarova S et al (2018) Factors influencing non-participation in an exercise program and attitudes towards physical activity amongst cancer survivors. Support Care Cancer 26:1289–1295. https://doi.org/10.1007/s00520-017-3952-9

    Article  PubMed  Google Scholar 

  9. Stout NL, Brown JC, Schwartz AL et al (2020) An exercise oncology clinical pathway: screening and referral for personalized interventions. Cancer 126:2750–2758. https://doi.org/10.1002/cncr.32860

    Article  PubMed  Google Scholar 

  10. Mayer DK, Alfano CM (2019) Personalized risk-stratified cancer follow-up care: its potential for healthier survivors, happier clinicians, and lower costs. JNCI J Natl Cancer Inst 111:442–448. https://doi.org/10.1093/jnci/djy232

    Article  PubMed  Google Scholar 

  11. Schmitz KH, Campbell AM, Stuiver MM et al (2019) Exercise is medicine in oncology: engaging clinicians to help patients move through cancer. CA Cancer J Clin 69:468–484. https://doi.org/10.3322/caac.21579

    Article  PubMed  PubMed Central  Google Scholar 

  12. Coletta AM, Campbell A, Morris GS, Schmitz KH (2020) Synergy between licensed rehabilitation professionals and clinical exercise physiologists: optimizing patient care for cancer rehabilitation. Semin Oncol Nurs 36:150975. https://doi.org/10.1016/j.soncn.2019.150975

  13. Stout NL, Santa Mina D, Lyons KD, et al (2020) A systematic review of rehabilitation and exercise recommendations in oncology guidelines. CA Cancer J Clin caac.21639. https://doi.org/10.3322/caac.21639

  14. Shallwani SM, King J, Thomas R et al (2019) Methodological quality of clinical practice guidelines with physical activity recommendations for people diagnosed with cancer: a systematic critical appraisal using the AGREE II tool. PLoS ONE 14:e0214846. https://doi.org/10.1371/JOURNAL.PONE.0214846

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Nadler M, Bainbridge D, Tomasone J et al (2017) Oncology care provider perspectives on exercise promotion in people with cancer: an examination of knowledge, practices, barriers, and facilitators. Support Care Cancer 25:2297–2304. https://doi.org/10.1007/s00520-017-3640-9

    Article  PubMed  Google Scholar 

  16. Pergolotti M, Deal AM, Lavery J et al (2015) The prevalence of potentially modifiable functional deficits and the subsequent use of occupational and physical therapy by older adults with cancer. J Geriatr Oncol 6:194–201. https://doi.org/10.1016/j.jgo.2015.01.004

    Article  PubMed  PubMed Central  Google Scholar 

  17. Brick R, Bender C, Skidmore E (2021) Impact of cancer and cancer-related treatments on participation restrictions. Br J Occup Ther 84:222–229. https://doi.org/10.1177/0308022620923858

    Article  Google Scholar 

  18. Alfano CM, Jefford M, Maher J et al (2019) Building personalized cancer follow-up care pathways in the United States: lessons learned from implementation in England, Northern Ireland, and Australia. Am Soc Clin Oncol Educ B 39:625–639. https://doi.org/10.1200/edbk_238267

    Article  Google Scholar 

  19. Osheroff JA, Teich JM, Middleton B et al (2007) A roadmap for national action on clinical decision support. J Am Med Informatics Assoc 14:141–145. https://doi.org/10.1197/jamia.M2334

    Article  Google Scholar 

  20. Sittig DF, Wright A, Osheroff JA et al (2008) Grand challenges in clinical decision support. J Biomed Inform 41:387–392. https://doi.org/10.1016/j.jbi.2007.09.003

    Article  PubMed  Google Scholar 

  21. Pawloski PA, Brooks GA, Nielsen ME, Olson-Bullis BA (2019) A systematic review of clinical decision support systems for clinical oncology practice. JNCCN J Natl Compr Cancer Netw 17:331–338

    Article  Google Scholar 

  22. Ligibel JA, Bohlke K, May AM, et al (2022) Exercise, diet, and weight management during cancer treatment: ASCO guideline. 101200/JCO2200687.

  23. (2018) Using Prioritization Matrices to Inform UX Decisions. https://www.nngroup.com/articles/prioritization-matrices/. Accessed 14 Oct 2020

  24. Powell BJ, Waltz TJ, Chinman MJ et al (2015) A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 10:21. https://doi.org/10.1186/s13012-015-0209-1

    Article  PubMed  PubMed Central  Google Scholar 

  25. Damschroder LJ, Aron DC, Keith RE et al (2009) Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 4:50. https://doi.org/10.1186/1748-5908-4-50

    Article  PubMed  PubMed Central  Google Scholar 

  26. Knoerl R, Mazzola E, Hong F et al (2021) Exploring the impact of a decision support algorithm to improve clinicians’ chemotherapy-induced peripheral neuropathy assessment and management practices: a two-phase, longitudinal study. BMC Cancer 21:236. https://doi.org/10.1186/s12885-021-07965-8

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Khairat S, Marc D, Crosby W, Al Sanousi A (2018) Reasons for physicians not adopting clinical decision support systems: critical analysis. J Med Internet Res 20. https://doi.org/10.2196/medinform.8912

  28. Birko S, Dove ES, Özdemir V, Dalal K (2015) Evaluation of nine consensus indices in delphi foresight research and their dependency on Delphi survey characteristics: a simulation study and debate on delphi design and interpretation. PLoS ONE 10:1–14. https://doi.org/10.1371/journal.pone.0135162

    Article  CAS  Google Scholar 

  29. Hsu CC, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assessment, Res Eval 12:1–8. https://doi.org/10.4135/9781412961288.n107

    Article  Google Scholar 

  30. Trevelyan EG, Robinson N (2015) Delphi methodology in health research: how to do it? Eur J Integr Med 7:423–428. https://doi.org/10.1016/j.eujim.2015.07.002

    Article  Google Scholar 

  31. Chang AM, Gardner GE, Duffield C, Ramis M-A (2010) A Delphi study to validate an advanced practice nursing tool. J Adv Nurs 66:2320–2330. https://doi.org/10.1111/j.1365-2648.2010.05367.x

    Article  PubMed  Google Scholar 

  32. Hagen NA, Stiles C, Nekolaichuk C et al (2008) The Alberta breakthrough pain assessment tool for cancer patients: a validation study using a Delphi process and patient think-aloud interviews. J Pain Symptom Manage 35:136–152. https://doi.org/10.1016/j.jpainsymman.2007.03.016

    Article  PubMed  Google Scholar 

  33. Boyer L, Pepin J, Dubois S et al (2020) Adaptation and validation of a nursing competencies framework for clinical practice on a continuum of care from childhood to adulthood: a Delphi study. Nurse Educ Today 93:104530. https://doi.org/10.1016/j.nedt.2020.104530

    Article  PubMed  Google Scholar 

  34. Sinha IP, Smyth RL, Williamson PR (2011) Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 8:e1000393. https://doi.org/10.1371/journal.pmed.1000393

    Article  PubMed  PubMed Central  Google Scholar 

  35. Qualtrics (2020) Qualtrics XM

  36. Weiner BJ, Lewis CC, Stanick C et al (2017) Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 12:108. https://doi.org/10.1186/s13012-017-0635-3

    Article  PubMed  PubMed Central  Google Scholar 

  37. Schmitz KH (2011) Exercise for secondary prevention of breast cancer: moving from evidence to changing clinical practice. Cancer Prev Res 4:476–480. https://doi.org/10.1158/1940-6207.capr-11-0097

    Article  Google Scholar 

  38. Mazo C, Kearns C, Mooney C, Gallagher WM (2020) Clinical decision support systems in breast cancer: a systematic review. Cancers (Basel) 12

  39. Hardcastle SJ, Kane R, Chivers P et al (2018) Knowledge, attitudes, and practice of oncologists and oncology health care providers in promoting physical activity to cancer survivors: an international survey. Support Care Cancer 26:3711–3719. https://doi.org/10.1007/s00520-018-4230-1

    Article  PubMed  Google Scholar 

  40. Yang DD, Hausien O, Aqeel M et al (2017) Physical activity levels and barriers to exercise referral among patients with cancer. Patient Educ Couns 100:1402–1407. https://doi.org/10.1016/j.pec.2017.01.019

    Article  PubMed  Google Scholar 

  41. Jyothi NS, Parkavi A (2016) A study on task management system. In: International Conference on Research Advances in Integrated Navigation Systems (RAINS) 2016, pp 1–6. https://doi.org/10.1109/RAINS.2016.7764421

  42. Bratterud H, Burgess M, Fasy BT, et al (2020) The Sung diagram: revitalizing the Eisenhower matrix. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, pp 498–502

  43. Kleynen M, Braun SM, Bleijlevens MH et al (2014) Using a Delphi technique to seek consensus regarding definitions, descriptions and classification of terms related to implicit and explicit forms of motor learning. PLoS ONE 9. https://doi.org/10.1371/journal.pone.0100227

  44. Vogel C, Zwolinsky S, Griffiths C et al (2019) A Delphi study to build consensus on the definition and use of big data in obesity research. Int J Obes 43:2573–2586. https://doi.org/10.1038/s41366-018-0313-9

    Article  Google Scholar 

  45. Fong AJ, Sabiston CM, Nadler MB, Sussman J, Langley H, Holden R, Stokes-Noonan M, Tomasone JR (2021) Development of an evidence-informed recommendation guide to facilitate physical activity counseling between oncology care providers and patients in Canada. Transl Behav Med 11(4):930–940. https://doi.org/10.1093/tbm/ibaa127

  46. Gonzalez BD (2018) Promise of mobile health technology to reduce disparities in patients with cancer and survivors. JCO Clin Cancer Informatics 1–9. https://doi.org/10.1200/CCI.17.00141

  47. Dicker AP, Jim HSL (2018) Intersection of digital health and oncology. JCO Clin cancer informatics 2:1–4. https://doi.org/10.1200/CCI.18.00070

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank additional members of the American Congress of Rehabilitation Medicine’s Cancer Rehabilitation Networking Group for their contributions and support. We would also like to thank Dr. Catherine Alfano for her role in algorithm development, and the following colleagues for their help with recruitment: Dr. Michelle Nadler, Dr. Terence Pugh, Dr. Michael Stubblefield, and Dr. Keith Thraen-Borowski.

Funding

Dr. Williams is supported in part by the National Cancer Institute of the National Institutes of Health (K08CA234225). The authors declare no additional funds, grants or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. ReVital Cancer Rehabilitation, Select Medical, Mechanicsburg, PA, USA

    Kelley C. Wood, Mackenzi Pergolotti & Tiffany D. Kendig

  2. Department of Occupational Therapy, Colorado State University, Fort Collins, CO, USA

    Kelley C. Wood, Mackenzi Pergolotti & Anita Bundy

  3. Ivy Rehab Network, White Plains, NY, USA

    Tim Marshall

  4. Department of Health and Exercise Science, Colorado State University, Fort Collins, USA

    Heather J. Leach

  5. Department of Community and Behavioral Health, Colorado School of Public Health, Aurora, USA

    Heather J. Leach

  6. Department of Statistics, Colorado State University, Fort Collins, CO, USA

    Julia L. Sharp

  7. Duquesne University School of Nursing, UPMC Hillman Cancer Center at UPMC Magee Women’s Hospital, Pittsburgh, USA

    Grace Campbell

  8. Department of Obstetrics, Gynecology, and Reproductive Science (Adjunct), University of Pittsburgh School of Medicine, Pittsburgh, USA

    Grace Campbell

  9. Department of Medicine, Division of Hematology/Oncology, University of Alabama at Birmingham, Birmingham, AL, USA

    Grant R. Williams

  10. Department of Palliative, Rehabilitation & Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Jack B. Fu

  11. Edson College of Nursing and Health Innovation, Arizona State University, Phoenix, AZ, USA

    Nancy Howe

Authors
  1. Kelley C. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mackenzi Pergolotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim Marshall
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heather J. Leach
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julia L. Sharp
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Grace Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Grant R. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jack B. Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tiffany D. Kendig
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Nancy Howe
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Anita Bundy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Dr. Wood led all aspects of the study as PhD dissertation. Drs. Pergolotti, Sharp, Leach, and Bundy served as dissertation committee members and reviewed and advised the study at each stage. The remaining authors participated in algorithm development or assisted with recruitment and aspects of study design.

Corresponding author

Correspondence to Kelley C. Wood.

Ethics declarations

Ethics approval

The Colorado State University Institutional Review Board has declared the study as exempt from review (protocol #: 20-10145H).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

N/A

Competing interests

Authors, Wood, Pergolotti, and Kendig receive salaries from Select Medical. The remaining authors have no relevant financial or non-financial interested to disclose. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wood, K.C., Pergolotti, M., Marshall, T. et al. Usability, acceptability, and implementation strategies for the Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm: a Delphi study. Support Care Cancer 30, 7407–7418 (2022). https://doi.org/10.1007/s00520-022-07164-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00520-022-07164-6

Keywords

Search

Navigation