Surgical Residency Recruitment—Opportunities for Improvement

doi:10.1016/j.jsurg.2016.09.005

Journal of Surgical Education

Volume 73, Issue 6, November–December 2016, Pages e104-e110

https://doi.org/10.1016/j.jsurg.2016.09.005 Get rights and content

Introduction

The Association of Program Directors in Surgery convened a panel during Surgical Education Week 2016 to discuss the current state of the general surgery residency application process and to review alternative ways to evaluate the suitability of each applicant to a residency program.

Methods/Results

Over 40,000 applicants registered for the National Resident Matching Program’s 2016 Main Residency Match. General Surgery had 2345 applicants for 1241 categorical postgraduate year (PGY)-1 positions, and 1239 of those positions were filled when the matching algorithm was processed. Program Directors reported that only 33% of applications received an in-depth review, and 62% were rejected with minimal review. Eventually (after all applications had been reviewed), only 13% of applicants were invited to interview.

Conclusions

There are several opportunities for improvement within the current application process. These included standardized letter of recommendation and personal statements, refinement of the interview process, and recalibration of the Medical Student Performance Evaluation.

Introduction

The Association of Program Directors in Surgery (APDS) convened a panel during Surgical Education Week 2016 to discuss the current state of the general surgery residency application process and to review alternative ways to evaluate the suitability of each applicant to a residency program. The panel explored possible implementation of a secondary or supplemental surgical application within the Electronic Residency Application Service (ERAS), standardized letters of recommendation (SLOR), revision of the personal statement and Medical School Performance Evaluation (MSPE), and alternative methods of residency interviews. This article summarizes the panel discussion and highlights options for improving the application process.

Section snippets

Current State of the Residency Application Process

Launched in 1996 to streamline the residency application process for all participants, ERAS currently supports approximately 33,000 program users, 56,000 applicants, 250,000 authors of letters of recommendation (LOR), and 600 medical school users. ERAS does not provide document verification services and all matching activities are entirely separate. More than 40,000 applicants registered for the National Resident Matching Program’s (NRMP) 2016 Main Residency Match. General surgery had 2345

Medical Student Performance Evaluation

The MSPE was designed to be an objective assessment of a student’s performance throughout medical school and not a letter of recommendation.⁸ In 1989, the Association of American Medical Colleges (AAMC) proposed guidelines for standardizing those letters, including “comparative performance data.”⁹ In 2002, the AAMC went further by recommending that the MSPE include the following 6 distinct sections: identifying information, unique characteristics, academic history, academic progress, summary

Letters of Recommendation

An applicant may submit up to 4 LOR as part of the application to residency. Each LOR is intended to establish the relationship between the writer and the applicant, review the applicant’s performance, and compare the applicant to peers.14, 15 Multiple prior studies have cast doubt on the validity and usage of narrative LOR because they suffer from high interreader variability,16, 17 intentional selection bias (on the part of the applicant), and conflict of interest (on the part of the writer).

Personal Statement

In its current format, the personal statement provides limited insight into the motivation and interest an applicant may have for a program. However, it is the only place in the application that offers a free text area for the applicant to explain any challenges or unique characteristics. The value of the personal statement for applicant selection by programs is not consistent. A total of 72% of PDs cite the personal statement as a factor in selecting applicants to interview, and 51% of PDs

The Interview

Interviews for surgery residencies are still largely conducted in person, although electronic interviews are increasing in prevalence.²⁶ NRMP survey data suggest that applicants’ personal interactions weigh most heavily in determining where an applicant is placed on a program’s rank list³ (Table 5).

Recruitment involves screening, interviewing, and ranking residency applicants. Of the 3, the interview is consistently cited as the most important aspect in resident selection.²⁷ Although it is

Other Considerations

The Accreditation Council for Graduate Medical Education (ACGME) program requirements for general surgery mandate is a 65% first-time pass rate on both the qualifying and certifying examinations for the American Board of Surgery (ABS).³² Some institutions also track programs’ in-training examination scores, and programs’ ABS examination results are publically reportable and are being used in ranking methodologies.³³ Multiple studies have shown that USMLE scores correlate with the American Board

Conclusion

The number of medical schools within the United States continues to steadily grow, placing pressure on fourth-year students to secure residency training positions. This competition, in turn, has led medical students to submit an increased number of ERAS applications.

Wholesale changes to the MSPE are likely to be a slow and deliberate process because many different stakeholders must agree about how to balance medical schools’ need to successfully place their students with their responsibility to

Acknowledgments

The authors are grateful to Mona Signer, MPH, for her thorough editing of the manuscript.

References (33)

B.T. Jarman et al.
Factors and influences that determine the choices of surgery residency applicants
J Surg Educ
(2015)
J.B. Naidich et al.
The meaning of excellence
Acad Radiol
(2007)
B.A. White et al.
Is the evaluation of the personal statement a reliable component of the general surgery residency application?
J Surg Educ
(2012)
G. Makdisi et al.
How we select our residents—a survey of selection criteria in general surgery residents
J Surg Educ
(2011)
ERAS 2015–2016 data. Available at: 〈www.aamc.org/download/362126/data/generalsurgerycategorical.pdf〉; Accessed...
National Resident Matching Program, Data Release and Research Committee: Results of the 2015 NRMP Applicant Survey by...
National Resident Matching Program, Data Release and Research Committee: Results of the 2014 NRMP Program Director...
N.M. Benson et al.
Going “Fourth” from medical school: fourth-year medical students’ perspectives on the fourth year of medical school
Acad Med
(2015)
E. Aagaard et al.
The residency application process—burden and consequences
N Engl J Med
(2016)
National Association for College Admission Counseling State of College Admission Report. Available at:...

C.S. Kiefer et al.

The “good” dean′s letter

Acad Med

(2010)

S.L. Korn et al.

The dean’s letter summary statement: valuing scholarship, leadership, and academic achievement

Acad Med

(2001)

J.A. Shea et al.

Medical student performance evaluations in 2005: an improvement over the former dean’s letter?

Acad Med

(2008)

L.I. Leiden et al.

National survey of writers of dean’s letters for residency applications

J Med Educ

(1986)

E.K. Alexander et al.

Variation and imprecision of clerkship grading in U.S. medical schools

Acad Med

(2012)

A.J. Kimple et al.

Standardized letters of recommendation and successful match into otolaryngology

Laryngoscope

(2016)

Cited by (20)

Exploring Characteristics of Academic General Surgery Residency Applicants: A Group Concept-Mapping Approach
2022, Journal of Surgical Education
Citation Excerpt :
Due to the volume of applicants to US surgery training programs, utilizing objective data to screen applications for interview offers may be more practical than applying holistic review principles. In 2020, there were 4,178 applicants to US general surgery training programs,27 and only a fraction of these were invited to interview.7 To expand the pool of applicants selected for an interview, one solution has been to use USMLE scores lower than previously established thresholds.8
Holistic review, which emphasizes qualitative attributes over objective measures, has been proposed as a method for selecting candidates for surgical residency in order to improve diversity in graduate medical education, and, ultimately, the field of surgery. This study seeks to articulate desirable traits of applicants as a first-step in standardizing the holistic review process.
Using Group Concept Mapping, a web-based mixed-methods participatory research methodology, residency selection committee members were asked to 1) list desirable characteristics of applicants, 2) group these into categories, 3) rate their importance to academic/clinical success on a 5-point Likert scale (1 = not at all important, 5 = extremely important), and 4) rate the degree to which each characteristic is feasible to assess on a 3-point Likert scale (1 = not at all feasible, 3 = very feasible). Grouped characteristics submitted to hierarchical cluster analysis depicted committee's consensus about desirable qualities/criteria for applicants. Bivariate scatter-plots and pattern-matching graphics demonstrated which of these criteria were most important and reliably assessed.
A single academic general surgery residency training program in Western Pennsylvania.
Members of the selection committee for the UPMC General Surgery Residency program who had participated in at least 1 prior cycle of applicant selection.
Desirable characteristics of highly qualified applicants into an academic general surgery residency were clustered into domains of 1) scholarly work and research, 2) grades/formal assessments, 3) program fit, 4) behavioral assets, and 5) aspiration. Behavioral assets, which was felt to be the most important to clinical and academic success were considered to be the least feasible to reliably assess. Within this domain, initiative, being self-motivated, intellectual curiosity, work ethic, communication skills, maturity and self-awareness, and thoughtfulness were viewed as most frequently reliably assessed from the application and interview process.
High quality applicants possess several behavioral assets that faculty deem are important to academic and clinical success. Adapting validated metrics for assessing these assets, may provide a solution for addressing subjectivity and other challenges scrutinized by critics of holistic review.
The Virtual Recruitment Onion: Peeling Back the Layers of the Interview Season During the COVID-era
2022, Journal of Surgical Education
Citation Excerpt :
While literature reports overall user satisfaction with recently conducted virtual fellowship interviews for both fellowship interviewers and applicants,4–7 it is still too early to determine if virtual can replace in-person interviews. Even prior to the changes necessitated by the COVID-19 pandemic, questions had been raised regarding the residency recruitment process.13–16 Specifically, virtual interviews using teleconferencing and video-conferencing had been explored to mitigate the significant cost and time constraints on both residency programs and applicants with overall positive responses from applicants.17–21
We sought to characterize General Surgery residency program directors’ (PDs) baseline perspective on how the COVID-19 mandated changes to the recruitment and interview processes impacted how the PDs evaluated and recruited the applicants.
An anonymous cross-sectional questionnaire survey.
A large, mid-western academic general surgery residency program.
47 PDs of Accreditation Council for Graduate Medical Education (ACGME) accredited General Surgery residency programs.
During the virtual-only interviews during the COVID-19 pandemic-era 2020-21 General Surgery residency application cycle, PDs shifted their focus to virtual outreach efforts and bolstered social media presences to recruit strong applicants. Also, our study found statistically significant changes to the increased value of letters of recommendation (LORs) for the PDs when assessing an applicant's commitment to surgery. These findings suggest that the necessity of adapting to the virtual-only interview format significantly altered how the PDs recruited and evaluated applicants for the General Surgery residency match.
A complete replacement of the in-person interviews with virtual-only interviews may be challenging unless buy-in exists from key stakeholders in the surgical community. Our study highlights the PDs’ hesitation in assessing candidates’ commitment to surgery from virtual interviews alone. Incorporating virtual interviews as a part of the screening process for applicants may serve as an avenue to maximize the benefits of the virtual interview format. Furthermore, COVID-19 pandemic has normalized the growing social media presence of residency programs, adding to the changing landscape of recruiting and interviewing applicants for General Surgery residency match.
Virtual interviews – Utilizing technological affordances as a predictor of applicant confidence
2021, American Journal of Surgery
In the midst of a pandemic, residency interviews transitioned to a virtual format for the first time. Little is known about the effect this will have on the match process. The study aim is to evaluate resident application processes and perceived outcomes.
An electronic survey was distributed to 142 colon and rectal surgery residency applicants (95% of total).
A total of 77 applicants responded to the survey (54% response rate). Applicants reported high levels of satisfaction with virtual interviews but less comfort. Utilizing the mute button and using notes in a different way from face-to-face interviews were significantly associated with applicant confidence that they ranked the right program highest. A majority of applicants (73%) would recommend virtual interviews next year even if COVID-19 is not a factor.
While applicants appear generally satisfied with virtual interviews, they also reported less comfort. Applicant confidence was predicted by utilizing the unique technological affordances offered by the virtual platform.
General Surgery Residency Match: Time for More than a Virtual Change
2021, Journal of Surgical Education
Program Director Opinions of Virtual Interviews: Whatever Makes my Partners Happy
2021, Journal of Surgical Education
Citation Excerpt :
First, programs are only able to interview a small selection of applicants. In the 2016 general surgery national residency match, only 13% of applicants to a program were invited to interview.8 With such limited capacity for interviewing, it is plausible that some truly well-fitting trainee-program pairs have missed out.
To assess the processes and outcomes of 2021 colon and rectal surgery match season: one of the first National Resident Matching Program (NRMP) match to conduct uniformly virtual interviews for all programs and candidates due to the Covid-19 pandemic. Since this if the first-year interviews were held entirely virtual for a (NRMP) match season, we sought to determine: (1) How did program directors (PDs) in this year's fellowship conduct their virtual interviews? (2) Were any of these conduct decisions associated with the PD satisfaction with the resulting match? (3) What is the PDs opinion of how interviews should occur next year if COVID-19 is not a factor?
The authors sent an anonymous survey to the PDs of all programs participating in the 2021 colon and rectal surgery residency match directly following match day 2020.
Forty-one colon and rectal residency PDs (70% response rate) responded to the survey (78% Male) representing a range of experience (M = 7.61, SD = 5.66, years as PD at current institution), and program type (77.5% Academic, 7.5% Independent Academic Medical Center, 15% Nonacademic).
While programs utilized several different platforms, conducted various forms of training for their faculty, and provided applicants with different types of information, interview day(s) across the specialty are reported to have proceeded smoothly. PDs as a whole were very satisfied with their match results this year (M = 4.65, SD = .66), and this satisfaction was not impacted by virtual interview decisions or processes. However, only 55% of PDs agree or strongly agree that next year's interviews should be virtual regardless of COVID-19, a judgement solely influenced by the opinion of other program faculty on virtual interviews, regardless of satisfaction with match or comfort with technology.
While PDs report high satisfaction with virtual interview processes and outcomes, there is less agreement that colon and rectal surgery residency interviews should move to a solely virtual platform.
An Early Interview Day with Feedback to Prepare General Surgery Applicants for Residency Interviews
2021, Journal of Surgical Research
Citation Excerpt :
Interviewers are specifically instructed to score applicants just as any other interview day; however, they are aware that this is the first interview for the applicants and thus may make unconscious adjustments toward a more forgiving score. Additionally, general surgery residency interview practices and question content vary by program.2,3,14 Thus, it is unlikely that the IDWF could be comprehensive in preparing applicants for every potential interview question.
Success in the residency interview is important in obtaining a position in general surgery residency. Preparing applicants for interviews is imperative yet can be resource intensive. The primary objective of our study was to investigate whether an early interview day with feedback (IDWF) provides interview preparation to internal applicants to general surgery residency without negatively impacting their rank list position at our program.
Internal applicants to the general surgery residency program at a single academic institution were invited to an early interview day after which they received individualized feedback and attended a workshop on interview techniques. Applicants were anonymously surveyed after The Match to measure their experiences with the IDWF. Two years of post-intervention program rank lists were compared to those from the five years pre-intervention to assess for change in rank list position of internal applicants. Participants included the 16 of 20 internal applicants who completed the survey. De-identified rank order data of internal applicants between 2014 and 2020 included 48 applicants.
All applicants believed the IDWF should be continued, and over half reported improved confidence and made changes from feedback. Rank list analysis demonstrated no statistically significant change in the proportion of internal applicants who ranked in the top 40 nor the average rank position of internal applicants.
An early interview day with feedback provides interview preparation in a resource-efficient manner without harming rank list position. The IDWF may be generalizable to other institutions to provide interview preparation to general surgery residency applicants.

View all citing articles on Scopus

View full text

2016 APDS SPRING MEETINGSurgical Residency Recruitment—Opportunities for Improvement

Introduction

Methods/Results

Conclusions

Introduction

Section snippets

Current State of the Residency Application Process

Medical Student Performance Evaluation

Letters of Recommendation

Personal Statement

The Interview

Other Considerations

Conclusion

Acknowledgments

J Surg Educ

Acad Radiol

J Surg Educ

J Surg Educ

Going “Fourth” from medical school: fourth-year medical students’ perspectives on the fourth year of medical school

Acad Med

The residency application process—burden and consequences

N Engl J Med

The “good” dean′s letter

Acad Med

The dean’s letter summary statement: valuing scholarship, leadership, and academic achievement

Acad Med

Medical student performance evaluations in 2005: an improvement over the former dean’s letter?

Acad Med

National survey of writers of dean’s letters for residency applications

J Med Educ

Variation and imprecision of clerkship grading in U.S. medical schools

Acad Med

Standardized letters of recommendation and successful match into otolaryngology

Laryngoscope

2016 APDS SPRING MEETING
Surgical Residency Recruitment—Opportunities for Improvement