Teacher judgments as measures of children's cognitive ability: A multilevel analysis☆
Introduction
Being able to assess students' ability correctly is one of teachers' crucial professional skills (Eaves et al., 1994, Ready and Wright, 2011). A recent meta-analysis of 75 studies on teacher judgment accuracy (Südkamp, Kaiser, & Möller, 2012) highlights five reasons why this is so: (1) teachers' judgments determine instructional decisions—what to teach, and how to teach it; (2) they are an important source of information in special education placement decisions; (3) they influence what teachers expect of their students; (4) they affect students' academic careers and success in life through grades, even over time periods as long as 40 years (Fischbach, Baudson, Preckel, Martin, & Brunner, 2013); and (5), mediated by grades, they contribute to student academic self-concept which, in turn, affects achievement (e.g., Marsh & Martin, 2011). Overall, teacher judgments (TJs) are relatively accurate: Correlations between TJs of student intelligence and IQ range between r = .45 and .80 (DeYoung, 2009) which is well aligned with the mean correlation of r = .63 between TJs and students' academic achievement reported by Südkamp et al. (2012). However, not all teachers are equally good at assessing student cognitive ability, and some students are assessed more inaccurately than others. In their meta-analysis, Südkamp et al. (2012) found a substantial proportion of the variation of TJs to be unrelated to actual student performance. (This proportion of variance amounts to approximatively 1 − r2 = 1 − .632 = .60 on average.) Teacher, student, and assessment characteristics thus influence TJs, which—although quite accurate on average—are still far from being objective, reliable, and valid (Schrader, 2009).
To date, most studies on TJs have focused on achievement (mostly in terms of grades, but also standardized achievement tests or curriculum-based measures) rather than potential (as approximated by intelligence tests teachers usually cannot readily access). This comes as no surprise, considering that teachers' judgment always depends on some manifestation of underlying ability. However, the younger children are, the more important it is to take their cognitive potential into consideration. Investment theory posits that fluid intelligence affects the acquisition of crystallized intelligence via learning (Schweizer & Koch, 2001). Therefore, the younger children are, the fewer chances they have had to unfold their potential yet; this is even more true for disadvantaged children. Especially in early years, fluid intelligence can be considered a valid indicator of children's overall cognitive ability (Baudson & Preckel, 2013) which, in turn, is the single best predictor of later academic and professional success (Neisser et al., 1996).
Section snippets
Factors that impact teacher judgments (TJs)
In the following, we outline findings on predictors of TJs at both student level and teacher/classroom level and introduce one new possible predictor. The subsequent hypotheses derived from our literature review reflect that despite almost half a century of research on TJs and their accuracy, the influence of many variables is still inconclusive (Jussim & Harber, 2005).
Level 1 sample
Teachers were asked to rate the cognitive ability of students they taught as classroom teachers. Overall, TJs of cognitive ability were available for 1864 primary school children from grades 1 to 3. Because mean TJs were comparable between grades, we collapsed data into one sample. We dropped twelve students from the sample because we could not ascertain the match between child and parent data (see below). Furthermore, we excluded all classrooms for which fewer than 10 TJs were available from
Overall relationships between TJs and other variables
We identified a zero-order correlation of r = .58 (p < .001) between TJs of student cognitive ability and actual student intelligence, which is in line with prior findings and indicates that teachers are quite capable to rank their students' intelligence accurately, yet far from perfect. However, this was by far the highest correlation; possible multicollinearity issues could therefore be excluded. Intercorrelations of all predictor variables are indicated in Table 2.
Intra-class correlations
We found an ICC = .031 for TJs.
Discussion
Teachers' ability to identify student cognitive potential is crucial to creating learning contexts that develop intellect and achievement. However, TJs of students' cognitive ability are based on more than actual student ability alone. Furthermore, teachers seem to differ in their aptitude to assess student cognitive ability. The aims of our study were to contribute to our understanding of what impacts these TJs and what explains their variability. We examined established as well as
References (80)
- et al.
Environmental moderators of genetic influence on verbal and nonverbal abilities in early childhood
Intelligence
(2005) - et al.
Do teacher judgments predict life outcomes?
Learning and Individual Differences
(2013) - et al.
Sex differences in mental abilities: g masks the dimensions on which they lie
Intelligence
(2007) - et al.
Self-fulfilling prophecies, perceptual biases, and accuracy at the individual and group levels
Journal of Experimental Social Psychology
(1998) - et al.
Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS
European Economic Review
(2006) - et al.
Early teacher perceptions and later student achievement
Journal of Educational Psychology
(1999) Some correlates of teachers' expectancy bias
American Educational Research Journal
(1985)- et al.
Auf einer anderen Schule oder bei einer anderen Lehrkraft hätte es für's Gymnasium gereicht!” Eine Untersuchung zur Bedeutung von Schulen und Lehrkräften für die Übertrittsempfehlung
Zeitschrift für Pädagogische Psychologie
(2011) - et al.
“Ethnic choosing”: Minority ethnic students, social class and higher educational choice
Race Ethnicity and Education
(2002) Parental involvement: Low socioeconomic status (SES) and ethnic minority parents' struggle for recognition and identity
Development and validation of the German Test for (Highly) Intelligent Kids — T(H)INK
European Journal of Psychological Assessment
Schulumwelten – Institutionelle Bedingungen des Lehrens und Lernens
Convergence of stranger ratings of personality and intelligence with self-ratings, partner ratings, and measured intelligence
Journal of Personality and Social Psychology
Exposure and affect: Overview and meta-analysis of research, 1968–1987
Psychological Bulletin
Measuring family socioeconomic status: An illustration using data from PIRLS 2006
Assessing pupils' intelligence through self, parental, and teacher estimates
Educational Psychology
Applied multiple regression/correlation analysis for the behavioral sciences
A comparison of inclusive and restrictive strategies in modern missing data procedures
Psychological Methods
Sustainability of teacher expectation bias effects on long-term student performance
Journal of Educational Psychology
Teachers' judgments of students' academic functioning: A comparison of actual and predicted performances
School Psychology Quarterly
Intelligence and personality
Long-term effects of parents' education on children's educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations
Merrill-Palmer Quarterly
Self-discipline gives the edge: Gender in self-discipline, grades, and achievement test scores
Journal of Educational Psychology
Using teacher judgment and IQ to estimate reading and mathematics achievement in a remedial reading programme
Psychology in the Schools
Soziale Herkunft und Kompetenzerwerb: Vergleiche zwischen PISA 2000, 2003 und 2006
Class size and student achievement
Psychological Science in the Public Interest
Bevölkerung und Erwerbstätigkeit. Bevölkerung mit Migrationshintergrund – Ergebnisse des Mikrozensus 2009
Reference groups and self-appraisals of academic ability and performance
Social Psychology Quarterly
Teachers' attributions and beliefs about girls, boys, and mathematics
Educational Studies in Mathematics
How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability
Psychological Bulletin
Mathematische Kompetenz von Schülerinnen und Schülern mit Zuwanderungshintergrund
Does nationality matter? The impact of stereotypical expectations on student teachers' judgments
Social Psychology of Education
Adding missing-data relevant variables to FIML-based structural equation models
Structural Equation Modeling: A Multidisciplinary Journal
Missing data analysis: Making it work in the real world
Annual Review of Psychology
Das “Drama” der hochbegabten Underachiever – “gewöhnliche” oder “außergewöhnliche” Underachiever?
Zeitschrift für Pädagogische Psychologie
Sex differences in mental test scores, variability, and numbers of high-scoring individuals
Science
The longitudinal relations of teacher expectations to achievement in the early school years
Journal of Educational Psychology
Cited by (17)
The Domain-Specificity of Elementary School Teachers’ Judgment Accuracy
2023, Contemporary Educational PsychologyWhen classmates influence teacher judgment accuracy of students’ cognitive ability: Studying frame-of-reference effects in primary school
2022, Contemporary Educational PsychologyCitation Excerpt :In this project, an intelligence test for primary school students (grades one to four) was developed (THINK 1–4; Baudson, Wollschläger, & Preckel, 2016). It should be noted that Baudson, Fischbach, and Preckel (2016) also used data from Project THINK for their analyses on frame-of-reference effects. However, whereas they used the pilot sample, we used the norm sample; the data from the two studies do not overlap.
What is the basis of teacher judgment of student cognitive abilities and academic achievement and what affects its accuracy?
2022, Contemporary Educational PsychologyContent-specificity of teachers’ judgment accuracy regarding students’ academic achievement
2021, Teaching and Teacher EducationA review on the accuracy of teacher judgments
2021, Educational Research ReviewStudent characteristics affecting the recognition of high cognitive ability by teachers and peers
2020, Learning and Individual Differences
- ☆
Tanja Gabriele Baudson, Department of Psychology, Giftedness Research and Education, University of Trier, Germany; Antoine Fischbach, Faculty LSHASE, ECCS Research Unit, University of Luxembourg, Luxembourg; Franzis Preckel, Department of Psychology, Giftedness Research and Education, University of Trier, Germany.