Comparing resident measurements to attending surgeon self-perceptions of surgical educators

doi:10.1016/S0002-9610(02)01421-6

The American Journal of Surgery

Volume 185, Issue 4, April 2003, Pages 323-327

https://doi.org/10.1016/S0002-9610(02)01421-6 Get rights and content

Abstract

Objective

The purpose of this study was to evaluate the initiation and utility of evaluating attending surgeons as educators by resident trainees. Additionally, we were interested in comparing resident measurements to attending self-perceptions.

Methods

A written evaluation form, (utilizing five-point ordinal scale assignments) queried respondents regarding the performance of surgical attendings in the operating room, and other clinical settings. A similar form was distributed to the faculty members, which they used to evaluate themselves. Mean scores were determined, as were comparisons between self-perception and resident assessments. Differences in scores with p values less than 0.05 were considered statistically significant.

Results

Thirty-six residents evaluated 23 attendings. Mean assignments by residents of performance in the operating room, other clinical settings, and overall scores for all faculty members as a group were 4.22 ± 0.04, 4.11 ± 0.03, and 4.16 ± 0.03, respectively, with a score of five, generally corresponding to a most favorable rating. When overall scores were analyzed, 10 attendings received scores that differed significantly from those received by their peers, with half of subjects above, and the other half being below the 95% confidence interval. Eighteen (78%) of attendings completed the self-evaluation forms, and of these, 11 (61%) had self-perceptions that differed significantly from overall scores as reported by the residents.

Conclusions

Our evaluation process delineated significant differences among attending faculty members and identified individual strengths and weaknesses. Many educators’ self-perceptions differed significantly from resident assessments, and attendings who did not evaluate themselves scored lower than their peers.

Section snippets

Methods

Surgical residents were asked to anonymously evaluate all the surgical attending physicians at the University Hospital using a questionnaire form designed through a consensus process utilized by a small ad-hoc committee of residents, medical students, and surgical attendings. The forms consisted of initial questions evaluating demographic information of the resident, and then progressed into two subsequent sections involving 10 questions each. The first section dealt with questions as they

Results

All 23 surgical attendings were evaluated at the University Hospital by 36 surgical residents. The average postgraduate year (PGY) level of the residents were 3.0 ± 0.3 years. The distribution of residents is shown in Table 2.

The overall means of the surgical attendings for each question is illustrated in Table 3. The range of scores were 3.53 ± 0.05 to 4.48 ± 0.04. The lowest observed scores were on questions 9 and 19, which in both sections evaluated how well attendings gave feedback to

Comments

Surgical attending physicians have a substantial role in educating medical students and residents. There have been (and remain) a great number of incredible educators and role models in the surgical fields. However, there does not seem to be any standardized, validated systems available to provide useful feedback or opportunities for improvement to surgical educators. Furthermore, it remains unknown whether written performance evaluations of surgical educators have any effect on future

References (15)

A.V. Blue et al.
Surgical teaching quality makes a difference
Am J Surg
(1999)
A.J. Seely et al.
Do surgical residents rated as better teachers perform better on in-training examinations?
Am J Surg
(1999)
C.H. Griffith et al.
Relationships of how well attending physicians teach to their students’ performances and residency choices
Acad Med
(1997)
D. Johnson et al.
Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit
Crit Care Med
(1998)
H. Sanfey
General surgery training crisis in America
Br J Surg
(2002)
C.H. Organ
The generation gap in modern surgery
Arch Surg
(2002)
A.A. Meyer et al.
The generation gapperspectives of a program director
Arch Surg
(2002)

There are more references available in the full text version of this article.

Cited by (52)

The Importance of Multiorgan Procurement in the Improvement of Residents’ Open Surgical Skills
2024, Journal of Surgical Research
The multiorgan procurement (MOP) represents a chance for the general surgery resident to learn the fundamental steps of open abdominal surgery. The objective of this study was to evaluate the impact of MOP on the residents’ open surgical skills.
Residents' surgical skills were assessed during a 6-month transplant rotation (October 2020-March 2021) using a modified Objective Structured Assessment of Technical Skills with the global rating scale. The surgeries were self-assessed by residents and tutors based on 9 specific steps (SS) and 4 general skills (GS). Each item was rated from 1 (poor) to 5 (excellent) with a maximum score of 45 points for SS and 20 for GS. A crossed-effects linear regression analysis was performed both to evaluate any associations between GS/SS scores and some prespecified covariates, and to study differences in the assessments performed by residents and tutors.
Residents actively participated in a total of 59 procurements. In general, there were no significant differences in SS/GS mean scorings between residents (n = 15) and tutors (n = 5). There was a significantly positive association between mean GS/SS scorings and the number of donor surgeries performed (at least 5). Comparing the evaluations of the tutors with the residents, this significance was retained only when scorings were assigned by the tutors.
MOP was shown to improve basic open surgical skills among residents. Awareness of the utility of a clinical rotation in transplant surgery should be raised also on an institutional level.
Teaching Before, During, and After a Surgical Resuscitation
2024, Surgical Clinics of North America
The reliability of resident self-evaluation of operative performance
2021, American Journal of Surgery
Self-assessment is critical to professional self-regulation yet many trainees may not reliably self-evaluate. We examine the gap between resident and faculty perceptions of trainee operative performance and contributing factors.
Surgery resident and faculty evaluations of trainee performance were collected from 14 academic institutions using smartphone-based performance assessments. Differences in resident/faculty ratings evaluating the same procedure were analyzed using descriptive statistics and Bayesian mixed models.
Of 7382 evaluations, 46% trainees and faculty performance ratings were discrepant (r = 0.47), with 80% residents rating themselves lower than faculty in those cases. This gap existed regardless of case complexity and widened as trainees gained experience. Trainees who overrated themselves had the lowest mean performance scores from faculty.
Half of residents perceived their performance differently from faculty, and this difference widened for senior residents. Future focus should be to provide opportunity for trainees to improve skills to reliably assess themselves before graduation.
Correlation between students' and trainers’ evaluations while learning delegated surgical procedures: A prospective cohort study
2019, International Journal of Surgery
The delegation of procedures within the medical competence to the nurses can increase the effectiveness of the healthcare provided. The objectives of the study are (1) to assess the quality of training courses for delegated surgical procedures through implementation for graduate scrub nursing (“students”) (2) and to evaluate the correlation between the evaluation of this training carried out by students and the self-assessment conducted by the faculty (“trainers”).
We set up a 49-h training for five groups of 10 students from July 2016 to July 2017 in our tertiary academic hospital. The course consisted mostly in simulations based on the “Zwisch” model and focused on acquiring the control of the gesture as well as on the development of critical reasoning. An evaluation of the training by the students but also a self-assessment of trainers were prospectively collected using the SFDP26 questionnaire.
52 active scrub nursing students and 21 trainers were included. 96% of students and 86% of trainers evaluated the training from “good” to “very good”. Progress was observed for 41 (79%) of the students and 18 (86%) of the trainers, and 98% of students felt able to put their new skills into clinical practice after training. There was no difference between the total scores of students and teachers (p = 0.153). A statistically significant difference between the evaluations produced by the students and the self-evaluations produced by the trainers was observed for 8 of the 26 items of assessment. In case of inadequacy, the trainers’ scores were always lower than those of the students.
Training in performing delegated surgical procedures by mixed cognitive and motor gestures learning, based on the development of critical thinking and simulations seems to be effective, with a significant improvement in students’ knowledge and skills. Expectations of students and trainers are well correlated.
Identifying Priorities for Faculty Development in General Surgery Using the Delphi Consensus Method
2018, Journal of Surgical Education
Citation Excerpt :
Faculty, conversely, perceived their teaching effectiveness to be good or very good over 75% of the time.8 In general, faculty overestimate the quantity and quality of teaching efforts and there seems to be a discrepancy between resident and faculty perceptions of teaching effectiveness.8, 9 Another reason for these disconnects is that in unstructured educational environments, learners inconsistently recognize educational experiences.10
Faculty teaching skills are critical for effective surgical education, however, which skills are most important to be taught in a faculty development program have not been well defined. The objective of this study was to identify priorities for faculty development as perceived by surgical educators.
We used a modified Delphi methodology to assess faculty perceptions of the value of faculty development activities, best learning modalities, as well as barriers and priorities for faculty development. An expert panel developed the initial survey and distributed it to the membership of the Association of Program Directors in Surgery. Responses were reviewed by the expert panel and condensed to 3 key questions that were redistributed to the survey participants for final ranking.
Seven experts reviewed responses to 8 questions by 110 participants. 35 participants determined the final ranking responses to 3 key questions.
The top three priorities for faculty development were: 1) Resident assessment/evaluation and feedback 2) Coaching for faculty teaching, and 3) Improving intraoperative teaching skills. The top 3 learning modalities were: 1) Coaching 2) Interactive small group sessions, and 3) Video-based education. Barriers to implementing faculty development included time limitations, clinical workload, faculty interest, and financial support.
Faculty development programs should focus on resident assessment methods, intraoperative and general faculty teaching skills using a combination of coaching, small group didactic and video-based education. Concerted efforts to recognize and financially reward the value of teaching and faculty development is required to support these endeavors and improve the learning environment for both residents and faculty.
Perioperative self-reflection among surgical residents
2017, American Journal of Surgery
Citation Excerpt :
Self-reflection is a powerful educational tool. Anthropologists have suggested that there may be some validity to the concept of a common “surgical personality”28–30 and that this personality may not be attracted to the concept of self-reflection.12 Just as we have allowed the new dictates of professionalism to supersede some of the older misbehavior conceded to traditional surgical personalities, so it is now imperative that all of us learn that self-reflection is possible, beneficial, and mandatory if we are to develop and progress as surgeons.
We studied prevalence and predictors of meaningful self-reflection among surgical residents and with prompting/structured interventions, sought to improve/sustain resident skills.
Residents from six programs recorded 1032 narrative self-reflective comments (120 residents), using a web-based platform. If residents identified something learned or to be improved, self-reflection was deemed meaningful. Independent variables PGY level, resident/surgeon gender, study site/Phase1: July2014-August2015 vs. Phase2: September2015-September2016) were analyzed.
Meaningful self-reflection was documented in 40.6% (419/1032) of entries. PGY5's meaningfully self-reflected less than PGY1-4's, 26.1% vs. 49.6% (p = 0.002). In multivariate analysis, resident narratives during Phase 2 were 4.7 times more likely to engage in meaningful self-reflection compared to Phase1 entries (p < 0.001). Iterative changes during Phase2 showed a 236% increase in meaningful self-reflection, compared to Phase1.
Surgical residents uncommonly practice meaningful self-reflection, even when prompted, and PGY5/chief residents reflect less than more junior residents. Substantial/sustained improvements in resident self-reflection can occur with both training and interventions.

View all citing articles on Scopus

View full text

Association for surgical educationComparing resident measurements to attending surgeon self-perceptions of surgical educators

Abstract

Objective

Methods

Results

Conclusions

Section snippets

Methods

Results

Comments

Am J Surg

Am J Surg

Relationships of how well attending physicians teach to their students’ performances and residency choices

Acad Med

Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit

Crit Care Med

General surgery training crisis in America

Br J Surg

The generation gap in modern surgery

Arch Surg

The generation gapperspectives of a program director

Arch Surg

Association for surgical education
Comparing resident measurements to attending surgeon self-perceptions of surgical educators