Do Clinical Clerks Provide Candidates with Adequate Formative Assessment during Objective Structured Clinical Examinations?

Abstract

Context: Various research studies haveexamined the question of whether expert ornon-expert raters, faculty or students,evaluators or standardized patients, give morereliable and valid summative assessments ofperformance on Objective Structured ClinicalExaminations (OSCEs). Less studied has beenthe question of whether or not non-facultyraters can provide formative feedback thatallows students to take advantage of theeducational opportunity that OSCEs provide. This question is becoming increasinglyimportant, however, as the strain on facultyresources increases.Methods: A questionnaire was developed toassess the quality of feedback that medicalexaminers provide during OSCEs. It was pilottested for reliability using video recordingsof OSCE performances. The questionnaires werethen used to evaluate the feedback given duringan actual OSCE in which clinical clerks,residents, and faculty were used as examinerson two randomly selected test stations.Results: The inter-rater reliabilityof the 19-item feedback questionnaire was 0.69during the pilot test. The internalconsistency was found to be 0.90 during pilottesting and 0.95 in the real OSCE. Using thisform, the feedback ratings assigned to clinicalclerks were significantly greater than thoseassigned to faculty evaluators. Furthermore,performance on the same OSCE stations eightmonths later was not impaired by having beenevaluated by student examiners.Discussion: While evidence of markinflation within the clinical clerk examinersshould be addressed with examiner training, thecurrent results suggest that clerks are capableof giving adequate formative feedback to morejunior colleagues.