Which teaching practices improve student performance on high-stakes exams? Evidence from Russia

https://doi.org/10.1016/j.ijedudev.2014.01.003Get rights and content

Highlights

  • We estimated the impact of teacher practices on Russian students’ college entrance exam scores.

  • We made these estimates separately for students in basic and advanced academic tracks.

  • Of three practices aimed at increasing scores, only high amounts of homework are effective.

  • The effective type of homework differs in the basic and advanced tracks.

Abstract

This study examines the relationship between student achievement and teaching practices aimed at raising student performance on a high stakes college entrance examination—the Russian Unified State Exam (USE). Data come from a survey of 3000 students conducted in 2010 in three Russian regions, and the analysis employs a student fixed effects method that estimates the impact of mathematics and Russian language teachers’ practices in advanced and basic tracks on students’ exam results. The study finds that the only practices positively affecting test outcomes are greater amounts of subject-specific homework, and that the most effective type of homework differs across tracks.

Introduction

Although a generation of research on “educational production” has greatly increased our knowledge of what works to improve student learning, there is little research on which teaching practices impact high school students who face high-stakes exit/entrance examinations. In most countries, including China, India, Russia, Germany, Brazil, and the United States, high school students must take entrance exams to qualify for college and particularly elite colleges (Carnoy et al., 2013). In other countries, such as France, Spain, and Italy, high school students are required to take high school exit exams to qualify for a degree.

Evidence suggests that, when preparing students for such high-stakes examinations, teachers use certain types of practices more than others to increase student achievement. Bishop, 1996, Bishop, 1997 showed that teaching practices in Canadian provinces with curriculum-based high school exit examinations were more likely to focus on more complex learning skills. Teachers also assign more homework related to the exam and give more practice exams compared to provinces without such examinations. Whether and which of these teaching practices in fact help students to improve their performance in a high-stakes environment, however, has not yet been rigorously tested in the empirical literature.

Given this lack of evidence, the goal of our study is to examine which teaching practices improve the performance of high school students on a high-stakes examination. We use a unique data set from a survey from three regions of Russia of almost 3000 final-year (11th grade) high school students who were preparing for the national college entrance exam (the Unified State Exam or USE) in 2010. We use the data and a cross-subject student fixed effects model (Clotfelter et al., 2010) to estimate the effect of three specific teaching practices on student examination outcomes: (a) the proportion of homework exercises targeting specific entrance exam items (hereafter known as “test-specific homework exercises”); (b) teachers’ use of practice (or mock) examinations; and (c) teachers’ use of websites geared to help students prepare for the exam.

We find that of the three practices only “test-specific homework exercises” has a positive and significant effect on student performance. The effect is rather large—about 0.2 of a standard deviation (SD) in exam score. Further, we find that the effectiveness of test-specific homework exercises is greater for students in the advanced track when homework exercises are focused on more difficult test items. Similarly, the effectiveness of test-specific homework exercises is greater for students in the basic track when homework exercises are focused on easier test items. The results suggest that we can identify those teaching practices that improve high school student performance on high-stakes exams.

The structure of the rest of the paper is as follows: Section 2 provides background on research that is relevant to this study and on the Russian education system, in particular the USE examination. Section 3 describes the data. Section 4 discusses the estimation strategy. Section 5 presents the results. Section 6 discusses the results and concludes.

Section snippets

Research on teacher impacts

Recent discussion on the effectiveness of school inputs in raising student outcomes focuses on teachers, showing that students with more effective teachers perform better on achievement tests (for example, Sanders and Rivers, 1996, Rockoff, 2004, Hanushek et al., 2005, Nye et al., 2004, Boyd et al., 2006). However, much of the emphasis in identifying effective teachers has been on teacher characteristics associated with higher student outcomes rather than on teaching practices. Such teacher

Data

In May 2010, we conducted a survey of 2927 students in 182 classes in 127 schools in three regions of Russia: Pskovskaya and Yaroslavskaya oblasts and Krasnoyarsky krai. The three regions were chosen because they represent a diversity of economic and educational contexts in a large and heterogeneous country. Krasnoyarsky krai in Siberia is Russia's second largest region (13 percent of the national territory) and is one of the richest in natural resources and industrial production. Yaroslavskaya

Estimation strategy

Estimating the impact of teaching practices on student performance can be complicated by omitted variables bias. Traditional (e.g. ordinary least squares or OLS) analyses of the relationship between school inputs and student outcomes often do not account for non-random assignment of students to schools. Many parents make an effort to choose their child's school (Lankford et al., 2002a, Lankford et al., 2002b, Bonesrønning et al., 2005). Children from families with greater economic and cultural

Results

The results of our model show that some of the tools available to high school teachers do have a positive effect on student performance on high-stakes tests. According to our estimates in Table 4, a high amount of exposure to test-specific homework exercises positively impacts student performance on the USE. High amounts of B-types of test-specific homework exercises, for example, increase USE scores by about 0.10–14 SDs (Table 4, columns 2, 4, 6, and 8) Similarly, high amounts of C-types of

Conclusions

We can draw three important conclusions from the results of our study. The first is that specific teaching practices can raise high school students’ outcomes on high-stakes exams. Specifically, targeting homework exercises of the appropriate level of difficulty at students of different levels of performance can increase student achievement on these exams. The impact of targeting is particularly notable for students in the advanced mathematics track where assigning more homework in the more

Acknowledgements

The authors wish to thank the National Research University Higher School of Economics and the U.S. Department of Education, Fund for the Improvement of Secondary Education, for supporting this research.

References (29)

  • M. Carnoy et al.

    University expansion in a changing global economy: Triumph of the BRICs?

    (2013)
  • C.T. Clotfelter et al.

    Teacher credentials and student achievement in high school: a cross-subject analysis with student fixed effects

    Journal of Human Resources

    (2010)
  • L. Darling-Hammond

    Educational Stanford, CA: Stanford University Graduate School of Education. A SCOPE Policy Brief

    (2009)
  • Cited by (16)

    • Does teaching to the test improve student learning?

      2021, International Journal of Educational Development
    • Is it the way they use it? Teachers, ICT and student achievement

      2017, Economics of Education Review
      Citation Excerpt :

      The importance of what teachers do in the classroom has been emphasized in the recent literature on the effects of teaching practices on students’ academic performance, which has focused on the effect of traditional versus modern teaching style. The results generally show that teaching style matters (Aslam & Kingdon, 2011; Schwerdt & Wuppermann, 2011; Zakharov, Carnoy, & Loyalka, 2014), but the empirical evidence is not conclusive with regard to the comparative effectiveness of modern and traditional practices (Lavy, 2011; Van Klaveren, 2011). Bringing together the literature on the effects of ICT at school and the literature on the effects of teaching practices on student performance, in this study we investigate the effect of ICT-related teaching practices on students’ achievement.

    • Does shadow education help students prepare for college? Evidence from Russia

      2016, International Journal of Educational Development
      Citation Excerpt :

      These studies are of limited external validity, however, since they are small-scale, involving a few hundred individuals, unrepresentative of the wider population of high school students in the United States, and mostly take place before 1990. In contrast to earlier studies (and as explained below in Section 3), we attempt to deal with selection bias by using a cross-subject student fixed effects model (see, for example, Zakharov et al., 2014; Van Klaveren, 2011; Schwerdt and Wuppermann, 2011; Clotfelter et al., 2010; Kingdon and Teal, 2010; Dee, 2005, 2007). Since the collapse of socialism, the market for shadow education has been growing rapidly in Russia (and indeed in the rest of Eastern Europe and Central Asia—Kozar, 2013; Silova, 2010; Silova et al., 2006).

    • The impact of teacher credentials on student achievement in China

      2015, China Economic Review
      Citation Excerpt :

      As such, the entire class would be exposed to the same math and Chinese teacher, and we would not have to worry about teachers of higher rank having a larger say in the assignment process within school. Taken together, this estimation strategy (like other studies employing cross subject variation across students, see e.g. Dee, 2005, 2007; Clotfelter et al., 2010; Kingdon & Teal, 2010; Van Klaveren, 2011; Schwerdt & Wupperman, 2011; Zakharov, Carnoy, & Loyalka, 2014) attempts to address endogeneity in three ways. First, it controls for a rich set of baseline (wave 1) characteristics including pre-test achievement scores to determine the impact on student test outcomes in the endline (wave 2).

    • Does homework design matter? The role of homework's purpose in student mathematics achievement

      2015, Contemporary Educational Psychology
      Citation Excerpt :

      These findings emphasize the importance of a homework preparation phase in which teachers calibrate the homework tasks' difficulty to match the students' proximal zone of development (Vygotsky, 1978) and promote homework engagement (e.g., adjusting the task difficulty and matching the students' educational needs). In fact, Zakharov, Carnoy, and Loyalka (2014) concluded that tasks that are adjusted for difficulty have a positive impact on students' performance. Our participants reported that this practice was valued; however, in everyday classes, they face several constraints in performing this instructional practice (e.g., the vast number of topics to discuss in class during the school year, the number of students per class (an average of 26), and the number of classes they teach at different grade levels (i.e., 5th, 6th and 7th grade).

    • The urban-rural divide in educational outcomes: Evidence from Russia

      2015, International Journal of Educational Development
      Citation Excerpt :

      The only systematic analysis of the quality of the Russian educational system is that of Amini and Commander (2012) that analyses Russian educational performance in a comparative light. Zakharov et al. (2014) estimate an educational production function to analyse what teaching practices can increase students’ individual performance. Carnoy et al. (2015) explore the reasons of Russia's poor performance in PISA scores by comparing students of medium schools to their Latvian and Estonian counterparts.

    View all citing articles on Scopus
    View full text