Are teachers accurate in predicting their students’ performance on high stakes’ exams? The case of Russia

https://doi.org/10.1016/j.ijedudev.2015.04.007Get rights and content

Highlights

  • Students in Russia take high stakes secondary school exit exams (USE)

  • Teachers have considerable information about student achievement.

  • Teachers who report good class relations over predict their class’ USE results.

  • Teachers who report poor class relations under predict USE results.

  • Even with good information about students, teachers may not be accurate.

Abstract

The paper focuses on how accurate teachers may or may not be in gauging their class’academic abilities. We use a sample of classrooms in three Russian regions to identify sources of mathematics and Russian teachers’ inaccuracies in predicting their high school classes’ scores on Russian and mathematics high stakes college entrance tests (the Unified State Exam, or USE). We test the hypothesis that teachers’ perceptions of their relationship with their classes are good predictors of such inaccuracies. This is important because teachers often focus on their relationship with the class as an end in itself or as a means to engaging students. Good teacher–student relations may indeed result in more students’ learning, but perhaps not nearly as much as teachers’ believe. We find that both Russian and mathematics teachers make inaccurate predictions of their class’ high stakes examination results based on how they perceive their relationship with their class. Teachers who believe they have a very good relationship with the class significantly overestimate their class’ performance on the USE, and those who perceive a poor relationship, underestimate their class’ performance, although this underestimate is generally not statistically significant.

Introduction

Much of the literature on effective teaching emphasizes the ideal of teachers as reflective professionals capable of individualized approaches to student learning (for example, Cohen, 1993, Darling Hammond, 1996). An implicit assumption underpinning this ideal is that teachers are accurate, fair judges of their students’ abilities, and that they can (and should) individualize broad curricular guidelines to fit each student’s capacity and learning style.

Two strands of research have questioned this assumption. One strand, going back to the 1960s, argues that teachers may not be neutral observers of students’ abilities, that teachers’ expectations may vary among students, and that teachers’ expectations (positive or negative) can affect students’ performance (Rosenthal and Jacobson, 1968). More recently, this research has turned to finding students' (and teachers’) characteristics that may affect teacher expectations in particular subjects, and, hence, their students’ performance (Brophy, 1983, Rosenthal and Rubin, 1978, Rosenthal, 1994). A number of studies find teacher gender bias—teachers viewing boys as having greater math and science skills and girls as having greater literary skills (Qing, 1999, Ready and Wright, 2011, Riegle-Crumb and Humphries, 2012, Shepardson and Pizzini, 1992), but others find no evidence of teacher gender bias (Dusek and Joseph, 1983, Madon et al., 1998). Similarly, many studies have found teacher race/ethnic bias (Ready and Wright, 2011, Rubie-Davies et al., 2006, Tenenbaum and Ruck, 2007), and social class bias (Auwarter and Aruguete, 2008, Ready and Wright, 2011). This may also relate to how teachers view students in different academic tracks (Kelly and Carbonaro, 2012, Oakes, 1985, Page, 1987, Tach and Farkas, 2006). Teacher gender and ethnicity have also been shown to play a role in affecting the performance of students’ of particular gender and ethnicity (Dee, 2005, McKown and Weinstein, 2008, Ready and Wright, 2011, Van den Bergh et al., 2010). Some of these studies estimate causal effects and show that teachers’ subjective judgment—consciously or unconsciously—can and does affect students’ academic outcomes.

Although much less studied, the second strand of research argues that teachers’ expectations for students’ performance compared to actual results may differ not because of conscious or unconscious “biases,” but because of what Jussim and Harber (2005) called “predictive validity without self-fulfilling influence.” Ferguson (2003) argued that both teacher inaccuracy and bias regarding student performance are deviations from a ‘true” benchmark, namely how much students have actually learned and their performance on measures of their learning. Teachers may be poor predictors of student performance because teachers may misestimate how much certain teaching practices or classroom conditions positively or negatively affect student learning and test performance. Teachers’ expectations based on these views of practices and classroom conditions can be inaccurate but not necessarily biased if the difference of predicted and actual class average test scores does not vary systematically according to classroom (students’) characteristics.

In this paper, we focus on this second strand of “predictive validity,” namely how accurate teachers may or may not be in gauging their class’ academic abilities. We use a sample of classrooms in three Russian regions to identify sources of mathematics and Russian teachers’ inaccuracies in predicting their high school classes’ scores on Russian and mathematics high stakes college entrance tests (the Unified State Exam, or USE). We test the hypothesis that teachers’ perceptions of their relationship with their classes are good predictors of such inaccuracies. This is important because teachers often focus on their relationship with the class as an end in itself or as a means to engaging students. Good teacher–student relations may indeed result in more students’ learning, but perhaps not nearly as much as teachers’ believe.

Teachers should be able to predict how their class will score on either the mathematics or Russian section of the USE, since in the 11th grade of Russian schools great emphasis is placed on preparing for this examination, including homework assignments and practice tests. Teachers in Russia usually teach the same students for two years in either mathematics or Russian. We control for the characteristics of a class that could “bias” teachers’ expectations of students’ performance. We also control for two variables that should help teachers make better predictions of their class’ performance on the USE: (a) students’ grades in the first semester of the 11th grade (information about students’ previous performance) and (b) teachers’ years of teaching experience (accumulated expertise).

We find that both Russian and mathematics teachers make inaccurate predictions of their class’ high stakes examination results based on how they perceive their relationship with their class. Teachers who believe they have a very good relationship with the class significantly overestimate their class’ performance on the USE, and those who perceive a poor relationship, underestimate their class’ performance, although this underestimate is generally not statistically significant. Teachers’ view of their relationship with the students in their class is not significantly related to the gender composition of their class in either math or Russian, but Russian teachers are significantly more likely to report a very good relation with their class when the cultural capital (Bourdieu and Passeron, 1976, Bourdieu, 1986)1 of their students is higher, even controlling for students’ academic performance. This may suggest some “cultural capital bias” on the part of Russian teachers.

The paper is structured as follows: in the next section, we describe our study’s Russian secondary education context; Section 3 describes our methodology and data; Section 4, the results, and the final section discusses the results and concludes.

Section snippets

The advantages of studying the Russian case

Russian high school is a particularly interesting context in which to study teachers’ accuracy in assessing how well their students have learned an academic subject. High school teachers generally know their students well in Russia. Almost all students have the same mathematics teacher and the same Russian language teacher for both high school grades (10–11th). In many instances, they even have the same teacher for each subject since the 5th grade. Thus 11th grade teachers have known their

Survey timing and sampling

We use data from a sociological survey conducted in May 2010 in three Russia regions: Pskovskaya and Yaroslavskaya oblasts and Krasnoyarsky krai. These regions were selected because they provide significantly different demographic, social and economic contexts for high school education. Yaroslavskaya oblast is a small region in the center of Russia, rather average in regional ratings of social and economic development. Pskovskaya oblast is located in the northwest part of the country. It is

Estimation strategy

To test our hypothesis that teachers’ view of their relationship with the class influences the accuracy of their predictions of student performance, we employ a variant of a structural equation model (SEM). The underlying argument of the model is that teachers’ view of their relationship to the class is fashioned by student and teacher characteristics (see Eq. (1)), and that both teachers’ predictions of class average performance and actual class average performance on the USE are influenced by

Estimating teachers’ views of their relationship with their class

Teachers’ views of their relationship with their class could depend in part on class performance in the subject taught, in part on teachers' experience and teaching skill (as measured by their “category”), and, if the teacher is biased in her perceptions of students with certain “inherent” characteristics, on the characteristics of the students in the class. We test this model by estimating an ordinal probit regression. The ordinal values of the probit regression are the three possible views

Discussion

We set out to show that teachers may not be accurate in their assessment of students’ academic abilities. Our results using a sample in three Russian regions suggest that even under conditions of teachers having extensive information about how well students should do on an important high stakes test of their abilities, teachers who feel particularly good about their relations with their students greatly overestimate their class’ actual performance in mathematics.

Such mis-estimation also

Acknowledgement

The authors gratefully acknowledge financial assistance from the Basic Research Program of the National Research University Higher School of Economics.

References (29)

  • C. McKown et al.

    Teacher expectations, classroom context, and the achievement gap

    J. School Psychol.

    (2008)
  • L.M. Tach et al.

    Learning-related behaviors, cognitive skills, and ability grouping when schooling begins

    Social Sci. Res.

    (2006)
  • A.E. Auwarter et al.

    Effects of student gender and socioeconomic status on teacher perceptions

    J. Educ. Res.

    (2008)
  • J.H. Bishop

    The effect of national standards and curriculum-based exams on achievement

    Am. Econ. Rev.

    (1997)
  • P. Bourdieu

    The forms of capital

    Handbook of Theory and Research for the Sociology of Education

    (1986)
  • P. Bourdieu et al.

    Reproduction in Education, Society and Culture

    (1976)
  • J.E. Brophy

    Research on the self-fulfilling prophecy and teacher expectations

    J. Educ. Psychol.

    (1983)
  • Carnoy, M., Rothstein, R. (2013). What do international tests really show about U.S. student performance? Retrieved...
  • Cohen, D.K. (Ed.), 1993. Teaching for Understanding: Challenges for Policy and Practice JosseyBass, San...
  • L. Darling Hammond

    The right to learn and the advancement of teaching: research, policy, and practice for democratic education

    Educ. Res.

    (1996)
  • T.S. Dee

    A teacher like me: does race, ethnicity, or gender matter?

    Am. Econ. Rev.

    (1996)
  • J.B. Dusek et al.

    The bases of teacher expectancies: a meta-analysis

    J. Educ. Psychol.

    (1983)
  • R.F. Ferguson

    Teachers’ perceptions and expectations and the black–white test score gap

    Urban Educ.

    (2003)
  • L. Jussim

    Grades may reflect more than performance: comment on Wentzel (1989)

    J. Educ. Psychol.

    (1991)
  • Cited by (2)

    View full text