Elsevier

Learning and Individual Differences

Volume 52, December 2016, Pages 148-156
Learning and Individual Differences

Teacher judgments as measures of children's cognitive ability: A multilevel analysis

https://doi.org/10.1016/j.lindif.2014.06.001Get rights and content

Highlights

  • Predictors of teachers' judgments (TJ) of student cognitive ability are examined.

  • Predictors are examined at individual and classroom/teacher level.

  • The Big-fish-little-pond model is successfully applied to TJ.

  • Individual/class-average student ability are positively/negatively related to TJs.

  • Class size, student sex and language background have no effect on teacher judgments.

Abstract

Teachers' ability to identify student cognitive potential is crucial to creating learning contexts that develop intellect and achievement. The younger students are, the more important is a focus on potential rather than achievement. Teacher judgments (TJs) as measures of intelligence are particularly important where objective IQ tests are not standard. Although most studies on TJs have been conducted within classrooms, few have accounted for the nested data structure. We predicted TJs of student's cognitive ability through both established and under-researched factors pertaining to student, teacher, and classroom using multilevel analysis. Student intelligence was the strongest predictor at both individual (positive effect) and class-average level (negative effect), followed by parent level of education. Better-known students received higher TJs. Student sex and linguistic background had no effect. Teachers were comparably able to rank their students. Results are discussed with a focus on the quality of the “measuring instrument” teacher.

Introduction

Being able to assess students' ability correctly is one of teachers' crucial professional skills (Eaves et al., 1994, Ready and Wright, 2011). A recent meta-analysis of 75 studies on teacher judgment accuracy (Südkamp, Kaiser, & Möller, 2012) highlights five reasons why this is so: (1) teachers' judgments determine instructional decisions—what to teach, and how to teach it; (2) they are an important source of information in special education placement decisions; (3) they influence what teachers expect of their students; (4) they affect students' academic careers and success in life through grades, even over time periods as long as 40 years (Fischbach, Baudson, Preckel, Martin, & Brunner, 2013); and (5), mediated by grades, they contribute to student academic self-concept which, in turn, affects achievement (e.g., Marsh & Martin, 2011). Overall, teacher judgments (TJs) are relatively accurate: Correlations between TJs of student intelligence and IQ range between r = .45 and .80 (DeYoung, 2009) which is well aligned with the mean correlation of r = .63 between TJs and students' academic achievement reported by Südkamp et al. (2012). However, not all teachers are equally good at assessing student cognitive ability, and some students are assessed more inaccurately than others. In their meta-analysis, Südkamp et al. (2012) found a substantial proportion of the variation of TJs to be unrelated to actual student performance. (This proportion of variance amounts to approximatively 1  r2 = 1  .632 = .60 on average.) Teacher, student, and assessment characteristics thus influence TJs, which—although quite accurate on average—are still far from being objective, reliable, and valid (Schrader, 2009).

To date, most studies on TJs have focused on achievement (mostly in terms of grades, but also standardized achievement tests or curriculum-based measures) rather than potential (as approximated by intelligence tests teachers usually cannot readily access). This comes as no surprise, considering that teachers' judgment always depends on some manifestation of underlying ability. However, the younger children are, the more important it is to take their cognitive potential into consideration. Investment theory posits that fluid intelligence affects the acquisition of crystallized intelligence via learning (Schweizer & Koch, 2001). Therefore, the younger children are, the fewer chances they have had to unfold their potential yet; this is even more true for disadvantaged children. Especially in early years, fluid intelligence can be considered a valid indicator of children's overall cognitive ability (Baudson & Preckel, 2013) which, in turn, is the single best predictor of later academic and professional success (Neisser et al., 1996).

Section snippets

Factors that impact teacher judgments (TJs)

In the following, we outline findings on predictors of TJs at both student level and teacher/classroom level and introduce one new possible predictor. The subsequent hypotheses derived from our literature review reflect that despite almost half a century of research on TJs and their accuracy, the influence of many variables is still inconclusive (Jussim & Harber, 2005).

Level 1 sample

Teachers were asked to rate the cognitive ability of students they taught as classroom teachers. Overall, TJs of cognitive ability were available for 1864 primary school children from grades 1 to 3. Because mean TJs were comparable between grades, we collapsed data into one sample. We dropped twelve students from the sample because we could not ascertain the match between child and parent data (see below). Furthermore, we excluded all classrooms for which fewer than 10 TJs were available from

Overall relationships between TJs and other variables

We identified a zero-order correlation of r = .58 (p < .001) between TJs of student cognitive ability and actual student intelligence, which is in line with prior findings and indicates that teachers are quite capable to rank their students' intelligence accurately, yet far from perfect. However, this was by far the highest correlation; possible multicollinearity issues could therefore be excluded. Intercorrelations of all predictor variables are indicated in Table 2.

Intra-class correlations

We found an ICC = .031 for TJs.

Discussion

Teachers' ability to identify student cognitive potential is crucial to creating learning contexts that develop intellect and achievement. However, TJs of students' cognitive ability are based on more than actual student ability alone. Furthermore, teachers seem to differ in their aptitude to assess student cognitive ability. The aims of our study were to contribute to our understanding of what impacts these TJs and what explains their variability. We examined established as well as

References (80)

  • T.G. Baudson et al.

    Development and validation of the German Test for (Highly) Intelligent Kids — T(H)INK

    European Journal of Psychological Assessment

    (2013)
  • J. Baumert et al.

    Schulumwelten – Institutionelle Bedingungen des Lehrens und Lernens

  • P. Borkenau et al.

    Convergence of stranger ratings of personality and intelligence with self-ratings, partner ratings, and measured intelligence

    Journal of Personality and Social Psychology

    (1993)
  • R.F. Bornstein

    Exposure and affect: Overview and meta-analysis of research, 1968–1987

    Psychological Bulletin

    (1989)
  • D.H. Caro et al.

    Measuring family socioeconomic status: An illustration using data from PIRLS 2006

    (2012)
  • T. Chamorro-Premuzic et al.

    Assessing pupils' intelligence through self, parental, and teacher estimates

    Educational Psychology

    (2009)
  • J. Cohen et al.

    Applied multiple regression/correlation analysis for the behavioral sciences

    (2003)
  • L.M. Collins et al.

    A comparison of inclusive and restrictive strategies in modern missing data procedures

    Psychological Methods

    (2001)
  • H. de Boer et al.

    Sustainability of teacher expectation bias effects on long-term student performance

    Journal of Educational Psychology

    (2010)
  • M.K. Demaray et al.

    Teachers' judgments of students' academic functioning: A comparison of actual and predicted performances

    School Psychology Quarterly

    (1998)
  • C.G. DeYoung

    Intelligence and personality

  • E.F. Dubow et al.

    Long-term effects of parents' education on children's educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations

    Merrill-Palmer Quarterly

    (2009)
  • A. Duckworth et al.

    Self-discipline gives the edge: Gender in self-discipline, grades, and achievement test scores

    Journal of Educational Psychology

    (2006)
  • R. Eaves et al.

    Using teacher judgment and IQ to estimate reading and mathematics achievement in a remedial reading programme

    Psychology in the Schools

    (1994)
  • T. Ehmke et al.

    Soziale Herkunft und Kompetenzerwerb: Vergleiche zwischen PISA 2000, 2003 und 2006

  • R.G. Ehrenberg et al.

    Class size and student achievement

    Psychological Science in the Public Interest

    (2001)
  • Federal Statistical Office

    Bevölkerung und Erwerbstätigkeit. Bevölkerung mit Migrationshintergrund – Ergebnisse des Mikrozensus 2009

    (2010)
  • R.B. Felson et al.

    Reference groups and self-appraisals of academic ability and performance

    Social Psychology Quarterly

    (1986)
  • E. Fennema et al.

    Teachers' attributions and beliefs about girls, boys, and mathematics

    Educational Studies in Mathematics

    (1990)
  • P.A. Freund et al.

    How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability

    Psychological Bulletin

    (2012)
  • M. Gebhardt et al.

    Mathematische Kompetenz von Schülerinnen und Schülern mit Zuwanderungshintergrund

  • S. Glock et al.

    Does nationality matter? The impact of stereotypical expectations on student teachers' judgments

    Social Psychology of Education

    (2013)
  • J.W. Graham

    Adding missing-data relevant variables to FIML-based structural equation models

    Structural Equation Modeling: A Multidisciplinary Journal

    (2003)
  • J.W. Graham

    Missing data analysis: Making it work in the real world

    Annual Review of Psychology

    (2009)
  • P. Hanses et al.

    Das “Drama” der hochbegabten Underachiever – “gewöhnliche” oder “außergewöhnliche” Underachiever?

    Zeitschrift für Pädagogische Psychologie

    (1998)
  • L.V. Hedges et al.

    Sex differences in mental test scores, variability, and numbers of high-scoring individuals

    Science

    (1995)
  • J.B. Hinnant et al.

    The longitudinal relations of teacher expectations to achievement in the early school years

    Journal of Educational Psychology

    (2009)
  • Cited by (17)

    • When classmates influence teacher judgment accuracy of students’ cognitive ability: Studying frame-of-reference effects in primary school

      2022, Contemporary Educational Psychology
      Citation Excerpt :

      In this project, an intelligence test for primary school students (grades one to four) was developed (THINK 1–4; Baudson, Wollschläger, & Preckel, 2016). It should be noted that Baudson, Fischbach, and Preckel (2016) also used data from Project THINK for their analyses on frame-of-reference effects. However, whereas they used the pilot sample, we used the norm sample; the data from the two studies do not overlap.

    • A review on the accuracy of teacher judgments

      2021, Educational Research Review
    View all citing articles on Scopus

    Tanja Gabriele Baudson, Department of Psychology, Giftedness Research and Education, University of Trier, Germany; Antoine Fischbach, Faculty LSHASE, ECCS Research Unit, University of Luxembourg, Luxembourg; Franzis Preckel, Department of Psychology, Giftedness Research and Education, University of Trier, Germany.

    View full text