Skip to main content

Advertisement

SpringerLink
Log in

A Not So Perfect Score: Factors Associated with the Rate of Straight Line Scoring in Oncology Training Programs

  • Published:
Journal of Cancer Education Aims and scope Submit manuscript

Abstract

Straight line scoring (SLS), defined as trainee assessments with the same score for all evaluation items, is statistically improbable and potentially indicates inaccurate assessment. Factors contributing to higher SLS rates are unknown, and knowledge of SLS prevalence within oncologic training is lacking. SLS frequency was measured for evaluations from all Accreditation Council for Graduate Medical Education (ACGME)-accredited programs at a single cancer care institution between 2014 and 2018. SLS prevalence was estimated using hierarchical linear models (HLM) that considered characteristics of evaluator, trainee, and evaluation potentially related to SLS. Results were compared with national SLS rates. Six thousand one hundred sixty evaluations were included from 476 evaluators. Overall prevalence of SLS was 12.1% (95% CI 4.5–28.8). Residents (vs fellows) were less likely to have SLS evaluations (OR 0.5, 95% CI 0.4–0.8), though for all trainees increasing training year corresponded with increasing SLS frequency (OR 1.5, 95% CI 1.3–1.7). SLS was more common in procedural specialties compared with medical specialties (OR 2.1, 95% CI 1.1–3.8). Formative evaluations had lower SLS rates (OR 0.6, 95% CI 0.5–0.9) than summative evaluations, while milestone-based evaluations had higher rates than those that were not milestone-based (OR 1.5, 95% CI 1.03–2.2). Features of evaluators, such as subspecialty within oncology, and of trainees, such as seniority or trainee type, were related to SLS. Summative intent and milestone-based evaluations were more likely to be straight line scored. Specific evaluation scenarios at higher risk of SLS should be further examined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

National data used within the current manuscript is publicly available, and institutional data can be made upon specific relevant request.

References

  1. Nasca TJ, Philibert I, Brigham T, Flynn TC (2012) The next GME accreditation system — rationale and benefits. N Engl J Med 366(11):1051–1056

    Article  CAS  Google Scholar 

  2. Hauer KE, Vandergrift J, Hess B, Lipner RS, Holmboe ES, Hood S, Iobst W, Hamstra SJ, McDonald FS (2016) Correlations between ratings on the resident annual evaluation summary and the internal medicine milestones and association with ABIM certification examination scores among US internal medicine residents, 2013-2014. JAMA 316(21):2253–2262

    Article  Google Scholar 

  3. Beeson MS, Holmboe ES, Korte RC, Nasca TJ, Brigham T, Russ CM, Whitley CT, Reisdorff EJ (2015) Initial validity analysis of the emergency medicine milestones. Acad Emerg Med 22(7):838–844

    Article  Google Scholar 

  4. Aagaard E, Kane GC, Conforti L, Hood S, Caverzagie KJ, Smith C, Chick DA, Holmboe ES, Iobst WF (2013) Early feedback on the use of the internal medicine reporting milestones in assessment of resident performance. J Grad Med Educ 5(3):433–438

    Article  Google Scholar 

  5. Peabody MR, O’Neill TR, Peterson LE (2016) Examining the functioning and reliability of the family medicine milestones. J Grad Med Educ 9(1):46–53

    Article  Google Scholar 

  6. Turner TL, Bhavaraju VL, Luciw-Dubas UA, Hicks PJ, Multerer S, Osta A, McDonnell J, Poynter S, Schumacher DJ, Tenney-Soeiro R, Waggoner-Fountain L, Schwartz A, the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network–National Board of Medical Examiners Pediatrics Milestones Assessment Group (2017) Validity evidence from ratings of pediatric interns and subinterns on a subset of pediatric milestones. Acad Med 92(6):809–819

    Article  Google Scholar 

  7. Heath JK, Dine CJ (2019) ACGME milestones within subspecialty training programs: one institution’s experience. J Grad Med Educ 11(1):53–59

    Article  Google Scholar 

  8. Beeson MS, Hamstra SJ, Barton MA, Yamazaki K, Counselman FL, Shayne PH, Holmboe ES, Muelleman RL, Reisdorff EJ (2017) Straight line scoring by clinical competency committees using emergency medicine milestones. J Grad Med Educ 9(6):716–720

    Article  Google Scholar 

  9. Hamstra SJ, Edgar L, Sangha S (2018) Milestones National Report 2018, Milestones Resources National Report. [Online]. Available: https://www.acgme.org/What-We-Do/Accreditation/Milestones/Resources. Accessed 22 Oct 2018

  10. Chemtob CM, Tanaka P, Keil M, Macario A (2018) Analysis of milestone-based end-of-rotation evaluations for ten residents completing a three-year anesthesiology residency. Cureus 10(8):e3200

  11. Sebesta EM, Cooper KL, Badalato GM (2019) Program director perceptions of usefulness of the accreditation Council for Graduate Medical Education Milestones System for urology resident evaluation. Urology 124:28–32

    Article  Google Scholar 

  12. Drolet BC, Marwaha JS, Wasey A, Pallant A (2017) Program director perceptions of the general surgery milestones project. J Surg Educ 74(5):769–772

    Article  Google Scholar 

  13. Hariton E, Bortoletto P, Barnes KL, Kaimal AJ, Stagg AR (2018) Resident and program director’s perceptions of milestone-based feedback in obstetrics and gynecology. J Med Educ Curric Dev 5:238212051877479

    Article  Google Scholar 

  14. Conforti LN, Yaghmour NA, Hamstra SJ, Holmboe ES, Kennedy B, Liu JJ, Waldo H, Selden NR (2018) The effect and use of milestones in the assessment of neurological surgery residents and residency programs. J Surg Educ 75(1):147–155

    Article  Google Scholar 

  15. Carter WA (2014) Milestone myths and misperceptions. J Grad Med Educ 6(1):18–20

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Gynecologic Oncology and Reproductive Medicine, The University of Texas MD Anderson Cancer Center, 1155 Pressler St, CPB 6.3258, Houston, TX, USA

    Emily Hinchcliff, Michaela Onstad, Aaron Shafer, Michael Frumovitz & Amir Jazaeri

  2. Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Jillian Gunther & Annette Eakes Ponnie

  3. Department of Surgical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Brian Bednarski

  4. Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Diana Urbauer

  5. Department of Clinical Education, Department of Gynecologic Oncology and Reproductive Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Diane C. Bodurka

Authors
  1. Emily Hinchcliff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jillian Gunther
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Annette Eakes Ponnie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Brian Bednarski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michaela Onstad
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aaron Shafer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael Frumovitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Amir Jazaeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Diana Urbauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Diane C. Bodurka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emily Hinchcliff.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interests.

Ethics Approval

This research received appropriate Institutional Review Board (IRB) approval.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was presented as an oral presentation at the Texas Educator’s Academies Collaborative for Health Professionals Southeast Educational Symposium (TEACH-S) in May 2019.

Appendix

Appendix

Table 3 Specialty category classification of institutional ACGME accredited programs
Full size table
Table 4 Specialty category classification of national programs
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hinchcliff, E., Gunther, J., Ponnie, A.E. et al. A Not So Perfect Score: Factors Associated with the Rate of Straight Line Scoring in Oncology Training Programs. J Canc Educ 37, 615–620 (2022). https://doi.org/10.1007/s13187-020-01855-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13187-020-01855-6

Keywords

Search

Navigation