Keywords

1 Introduction

Opportunity to learn (OTL) can be considered the “observable structure” of teaching (Schmidt & Maier, 2009; Schmidt & McKnight, 1995) and thus makes a valuable contribution toward learning outcomes. OTL connects nationally prescribed contents and methods of teaching and learning (the intended curriculum) with learning contents, class environment, and school climate (the implemented curriculum), and students’ outcomes (the attained curriculum).

Alignment between educational goals, intended and implemented curricula, and educational outcomes is deemed an essential characteristic of effective education. The expectation is that better alignment between these leads to more effective education and hence better student performance. The concept of OTL is commonly used to compare content covered, as part of the implemented curriculum, with student achievement. As such, OTL can be viewed as a facet of the broader concept of alignment (Scheerens, 2017).

Accordingly, OTL can be characterized as the alignment between teaching processes and student achievement, or as “the alignment of standards and output measures, mediated by teaching processes” (Scheerens, 2017, p. 41), it can also refer to a wide range of inputs and processes within a school context that support intended student outcomes. In doing so, instructional time and content have also consistently been characterized as core elements of OTL, along with a number of instructional quality indicators. Carroll (1963) was first to define OTL as “the amount of time allowed for learning, for example by a school schedule or program” (see Carroll, 1989, p. 26). The central concept is the notion that students cannot learn content that has not been presented in class. Carroll (1989) included OTL as one of five variables in a formula that he used to express a student’s degree of learning. Some of these variables can be measured to a certain extent by time, in terms of the amount of time a student needs to be given to learn, understand, and master a task, and the amount of time allowed for this learning in the curriculum. This has led researchers to examine how time dedicated to instruction (the instructional time) may be related to student achievement. According to Kurz (2011, see also Elliott & Bartlett, 2016), Stevens (1996) distilled the first comprehensive conceptual framework of OTL, bringing together four elements: content coverage, content exposure (time on task), content emphasis (the emphasis on the cognitive processes required), and quality of instructional delivery (the methods and quality of the instructional practices used to deliver the content).

Content exposure refers to the estimated total amount of time actually devoted to covering the specific content (Leinhardt & Seewald, 1981). Terms like instructional time, amount of time devoted to teaching certain subject areas, and amount of teaching periods (or hours per week or year) are traditional measures of this particular dimension of OTL (Stedman, 1994; Wang, 1998). Brophy (2000) found that more time allocated to teaching specific content in classrooms positively contributed to student achievement. To provide students with the necessary opportunities to learn the intended curriculum, teachers must allocate instructional time toward addressing specifically prescribed teaching goals to achieve the requisite student outcomes (Elliott & Bartlett, 2016). Such measures may be categorized by teacher-oriented indicators, such as allocated time (the time scheduled for instruction), or by more explicitly student-oriented indicators, such as instructional time (proportion of allocated time used for instruction), engaged time (proportion of instructional time during which students are engaged in learning), and academic learning time (proportion of engaged time during which students are experiencing a high success rate of learning). Researchers have found time-based OTL indices to be moderately related to student achievement after controlling for student ability and socioeconomic status (Elliott & Bartlett, 2016, p. 5).

Teachers must also ensure that they cover the content outlined by the formal curricula, as these are the topics that students are likely to be assessed on. In 1964, IEA undertook the First International Mathematics Study (FIMS) in twelve countries, to investigate the outcomes of various school systems for mathematics, which was, at the time, undergoing reforms in many educational systems (Husén, 1967). This results from this early IEA study led to increased research interest in content-based conceptualizations of OTL, designed to evaluate the content overlap between enacted and assessed curricula (Elliott & Bartlett, 2016). Anderson (1986, p. 3682) noted that the “opportunity to learn from the Husén perspective is best understood as the match between what is taught and what is tested.” Perhaps the most important measure of content coverage in current policy efforts is the alignment of teachers’ instruction with state standards and/or assessments (Scheerens, 2017).

Substantial empirical evidence has documented the importance of OTL variables in explaining students’ test scores and found that students were more likely to respond to an item correctly if they had the opportunity to learn the tested concepts and skills, especially if students had this opportunity during the year the assessment was delivered. Wang (1998) examined the impact of Stevens’ (1996) four dimensions of OTL on outcomes and found that OTL was a significant predictor of student achievement in both written tests and other scores students received in schools. Variations in the effects of OTL could be attributed to differences in test format. In their study of English language learner achievement, Aguirre-Muñoz and Boscardin (2008) stated that content exposure was the most significant predictor of students’ written test scores, whereas the quality of instructional delivery was the most significant predictor of practical test scores.

IEA’s Trends in International Mathematics and Science Study (TIMSS) is one of the few international studies that are curriculum based, and it thus routinely collects information about OTL at different levels. As stated in the TIMSS framework, “TIMSS uses the curriculum, broadly defined, as the major organizing concept in considering how educational opportunities are provided to students and the factors that influence how students use these opportunities” (Mullis, 2017, p. 4).

Along with total instructional hours per year, TIMSS collects information, on a national level, on whether the curriculum or any other official document prescribes the percentage of total instructional time to be devoted to mathematics and science instruction at the grade four.

1.1 Framework and Research Questions

For our research, we considered a restricted concept of OTL. We used a conceptual framework that combined elements from educational effectiveness, a curriculum model, and the notion of OTL to analyze curriculum effects. In our model the intended curriculum is defined as the curriculum that an education system intends to implement, as stated in their official policy documents. At the school, classroom, and teacher level are the actors that actually implement that curriculum. Finally, the students (hopefully) attain the content taught. At each of these levels, we can observe specific curriculum factors that together define the whole curriculum (Bokhove et al., 2019).

We use the term opportunity to learn specifically with reference to the mathematics and science topics covered in classroom instruction. This reflects both the narrow curricular sense in which the concept was originally developed by Carroll (1963) and in the studies implemented by IEA. We chose to focus on the definition of OTL as time of exposure and quality of content for two reasons: (1) the provision of content is the fundamental rationale of schooling and the education system, and (2) this is an aspect of schooling that both reflects education policy and is amenable to education policy reform (see Scheerens, 2017, p. 41).

Seven participants from the Dinaric region took part in TIMSS 2019, namely Albania, Bosnia and Herzegovina, Croatia, Kosovo,Footnote 1 Montenegro, North Macedonia, and Serbia. Our analysis of the TIMSS 2019 data was designed to address two key research questions:

  1. (1)

    What can TIMSS 2019 tell us about students’ opportunities to learn mathematics and science across the Dinaric region?

  2. (2)

    Can students’ opportunities to learn be related to their mathematics and science achievement?

We explored key aspects of OTL included in the TIMSS international framework at grade four. In TIMSS, content exposure is defined as the instructional time prescribed and devoted to mathematics and science curricula at the grade four, and content coverage is defined as the number and content of mathematics and science topics intended to be taught and effectively taught in classes (Mullis & Martin, 2017). We explored both content exposure and content coverage at the level of intended curriculum and at the level of implemented curriculum. Our analyses conceptualized OTL as the overlap between intended content coverage and implemented content coverage; OTL is thus a measure of the relation between implemented content and intended content. Content coverage addresses the degree to which content was covered throughout the continuation of the school, in order to see if students have adequate opportunity to learn topics assessed by the test. Content exposure refers to the total amount of time spent on covering the specific subjects. Finally, we assessed the relationship between OTL and the attained curriculum, namely the mathematics and science achievement of TIMSS grade four students across the Dinaric region.

2 Methods and Data

As part of TIMSS, data about the contexts for learning are collected through questionnaires completed by students and their parents, teachers, and school principals (for more information, see TIMSS & PIRLS International Study Center, 2018). All the data used in our analyses were collected by the TIMSS 2019 grade four assessment. We carefully selected relevant data collected by the TIMSS 2019 background questionnaires to inform our analyses using variables at all three curriculum levels (Table 1).

Table 1 List of variables used in our analyses
Full size table

When examining the data available from TIMSS 2019 for grade four students, we limited our observations to two basic elements of the concept of opportunity to learn: time exposure (namely the time allocated to instruction in a topic in school) and content exposure (the amount of content presented to students by teachers). To observe the effect of OTL on final educational outcomes, we related the national averages for these elements to the national mean achievement of students in each topic, which was measured independently by TIMSS 2019 in all participating education systems.

Mathematics and science curricula were assessed using broad content domains: for mathematics, these were number, measurement and geometry, and data, and, for science, these were life science, physical science, and earth science. Each content was divided into specific topics, numbering 17 in total for mathematics (seven for number, seven for measurement and geometry, and three for data; Lindquist et al., 2017) and 26 in total for science (seven for life science, twelve for physical science, and seven for earth science; Centurino & Jones, 2017). These topics served as a basis for international development of items for students to solve in the TIMSS assessment (Mullis & Martin, 2017). Therefore they represent the content assessed by TIMSS.

In TIMSS, information on mathematics and science contents covered in national curricula up to grade four were collected at a system level by a curriculum questionnaire completed by the national research coordinator (NRC) in each participating entity from the Dinaric region. Such information reflects the content of the intended curricula. We assessed the implemented curricula using international data on principals’ and teachers’ responses to the TIMSS context questionnairesFootnote 2 (TIMSS & PIRLS International Study Center, 2018). The attained curriculum was evaluated using students’ mathematics and science achievement scores in the TIMSS 2019 assessment.Footnote 3 In addition, to analyze the achieved curriculum in more detail, we subdivided student achievement according to the specific mathematics (number, measurement and geometry, and data) and science (life science, physical science and earth science) content domains. The teacher questionnaire (see TIMSS & PIRLS International Study Center, 2018) also asked teachers to report which topics they had already taught to students participating in TIMSS prior to the assessment. These reports of the implemented curriculum can be compared with demonstrated knowledge measured by mathematics and science items in TIMSS tests.

During the development stages of every TIMSS assessment, a great deal of work is dedicated to ensuring comparability of achievement; here, the assessment content plays an important role. The test content is agreed by all participating education systems working in collaboration and aims to cover topics that are considered relevant by participating nations and that are also covered by the prescribed curricula in the majority of the participating systems. TIMSS provides additional information on the discrepancies between the assessment materials and national curricula, by undertaking a test-curriculum matching analysis of content topics (for further details and results, see Mullis et al., 2020, Appendix C). We used this data to also assess effect of content coverage on regional test scores.

Learning opportunities are not only provided to students in school but also outside formal settings, and especially at home. More books at home and higher education levels of parents have been clearly linked with more opportunities for children to learn at home (Chiu & Xihua, 2008; Eccles, 2005), but TIMSS provides an even more comprehensive scale that can be regarded as a measure of OTL at home, namely the TIMSS home resources for learning scale (HRL scale; Yin & Fishbein, 2020, p. 16.39). This scale covers the information on number of adult and children books and internet access at student home as well as parental education and occupations. Higher values on the HRL scale indicate greater access to home resources for learning, which, in turn, implies more OTL.

To investigate the relationship between OTL and achievement in more depth, as well as the percentages of students taught the topics, we also considered another aspect of OTL which can be linked to the classroom time provided to teaching content. In assessing the importance of higher content exposure, we also needed to consider how achievement may be affected when classroom teaching and hence content exposure is limited by the effects of lower home support and more problematic class climate. Among their responses to the TIMSS teacher questionnaire, teachers estimated how much they felt limited in teaching because of different student attributes. This was used to create a TIMSS scale, classroom teaching limited by students not ready for instruction (the LSN scale; Fishbein et al., 2021, Supplement 1, pp. 95–97). We included this scale in our analyses because additional instructional time is expected to be needed for students getting ready for instruction and consequently less time is available for teaching content; therefore lower values on this scale can be used as a negative measure of OTL.

To describe and assess the differences in students’ opportunities to learn about mathematics and science across the Dinaric region, we compared reports from each of the participating education systems derived from TIMSS data on the relevant factors. We undertook additional analyses to discover the relationships between the factors associated with OTL and student achievement, combining data from different sources reported on different scales. We calculated average time devoted to learning and percentages of the learning time devoted to mathematics and science, both according to national prescribed curricula and as reported at the school level by principals. We then used teacher reports on the actual content taught in the sampled schools and calculated mean percentages of students exposed to specific content for each education system. To examine the relationship between students’ opportunity to learn and their mathematics and science achievement, we calculated Pearson’s correlation coefficients between selected variables and student achievement. Finally, we used regression models to discover the relations between achievement and content exposure, taking two other factors into account: learning outside school and potential limitations to exposure to content taught in school. For the first factor, we used values on the TIMSS HRL scale to assess students’ external learning opportunities. The second variable we used was the TIMSS LSN scale. Student achievement was dependent variable in our regression models and we used t-test statistics to determine group differences (for a more information about the data sources and methodological tools, which account for the complex survey design, please see Sect. 5.

3 Results

3.1 Content Exposure

For all participating Dinaric education systems, we collected available data on agreed national curriculum targets or any other official document that prescribed the percentage of total instructional time that should be devoted to mathematics and science instruction at grade four. As this information was extracted from official documents, it can be considered a reasonable indicator of the intended or prescribed curriculum at the system level. In some of the Dinaric systems, schools were teaching an integrated curriculum (where school subjects are deliberately combined), so hours spent solely on mathematics or science instruction could not always be accurately reported. However, percentages of total instructional time prescribed nationally for mathematics and science instruction at grade four indicate that mathematics instruction was generally allocated about 20% of total instructional time in the participating systems and there was relatively low variation in the amount of time allocated, the lowest percentage being in Albania (17%) and the highest in Croatia (22%) (see Table 2). Science instruction was generally allocated approximately nine percent of total instructional time in most of the participating systems, although there variation was greater; notably, Croatia allocated 17% of grade four instructional time to science. Mathematics was thus generally allocated more learning time than science, and there was also more consistency among the Dinaric systems in the time allocated to this topic.

Table 2 Percentage of instructional time allocated at the system level to mathematics and science curricula at grade four
Full size table

At the level of curriculum implementation, we compared information collected from teachers and principals regarding the instructional time spent in individual sampled classes (Fig. 1), as the total instructional time across all subjects per year versus the instructional time devoted specifically to mathematics and science in the Dinaric education systems. Teachers reported the number of hours of instruction per week and principals reported the number of school weeks and days per year. The number of hours for mathematics and science per school year show that Croatia and Bosnia and Herzegovina provide the highest total number of hours of school instruction per year, and Albania and Montenegro provide the lowest number of hours of instruction per year. As a comparison, Croatian students received 206 hours more instruction per year than their peers in Montenegro. Across the Dinaric region, the distribution of time devoted to mathematics and science instruction varied considerably. Croatia devoted the highest number of hours per year to science instruction (82 h), while Kosovo spent the highest number of hours per year on mathematics instruction (150 h); Albania devoted the lowest number of hours per year to both mathematics and science instruction (113 h and 54 h, respectively).

Fig. 1
figure 1

Total instructional time received by students as reported by principals, and instructional time devoted to mathematics and science, as reported by teachers.

Notes In Kosovo and Serbia, the national defined population covers 90–95% of the national target population. In Kosovo, data for mathematics instruction were available for ≥70% but <85% of the students

Full size image

We compared information on prescribed hours reported at the system level by the national research coordinators with the information provided by principals and teachers about the implemented instructional hours and noted there were differences across the Dinaric region between the prescribed and implemented instructional time allocated to mathematics and science. On average, across the Dinaric participants, the prescribed instructional time was about 20% of the total instructional time for mathematics and about 10% of the total for science (see Table 2). In reality, the percentage of implemented instructional time (as calculated from the numbers reported by principals and teachers; see Fig. 1) was slightly lower than the prescribed instructional time in all participating systems. For mathematics, the time spent on instruction ranged from 14% of total instructional time in Croatia and Bosnia and Herzegovina to 19% in Kosovo. For science, this time ranged from seven percent in North Macedonia to 11% in Montenegro.

3.2 Content Coverage for Mathematics and Science Topics

Our first observation from analyzing the curricular data is that the intended curricula in the Dinaric region were defined at the system level and hence consistent for all students in each system. According to data collected by the TIMSS 2019 curriculum questionnaire (Fig. 2), there were large differences in the intended mathematics topics taught across Dinaric systems. Three of them, Albania, Bosnia and Herzegovina, and Croatia, indicated that seven out of the 17 TIMSS mathematics topics were not included in their grade four curricula. Kosovo reported that four topics were not included in their grade four curriculum. There was also considerable variation in the selection of topics that were taught. Bosnia and Herzegovina and Croatia flagged four out of seven number content topics were not included in their curricula. Regarding measurement and geometry content, Albania and North Macedonia marked four out of seven topics as not included in their curricula. It is worth noting that only Montenegro included all three data topics in their curriculum; the other Dinaric systems did not cover these topics at grade four, or only to minor extent, indicating there was generally low coverage of this content domain across the region. In North Macedonia, it was noteworthy that a number of topics were intentionally only taught to more able students; other Dinaric systems generally expected all topics included in the curriculum would be taught to all students (Fig. 2).

Fig. 2
figure 2

Intended curriculum coverage of TIMSS mathematics topics, as reported by national research coordinators.

Note Numbers in the bars indicate the number of TIMSS topics in each category

Full size image

According to the NRC reports, there were also large differences in science curricula topics across the Dinaric region (see Fig. 3). Only Kosovo and Montenegro reported covering all TIMSS science topics; in North Macedonia there were three topics that were only taught to the more able students. Bosnia and Herzegovina and Croatia omitted the largest number of TIMSS topics from their curricula (12 and 10, respectively). Bosnia and Herzegovina, Croatia, and North Macedonia did not cover a large number of the topics belonging to the physical science domain, but the other four Dinaric participants reported their curricula covered all twelve topics. Montenegro was the only Dinaric participant to cover all TIMSS earth science topics in their curriculum (Fig. 3).

Fig. 3
figure 3

Intended curriculum coverage of TIMSS science topics, as reported by national research coordinators.

Note Numbers in the bars indicate the number of TIMSS topics in each category

Full size image

Teachers reported the mathematics content in the TIMSS 2019 assessment that they taught their students (Fig. 4). Their reports revealed that there were quite substantial disparities in the implemented mathematics curricula across the Dinaric region. For example, Kosovo, Croatia, and Serbia focused more on measurement and geometry content, while North Macedonia, Bosnia and Herzegovina, and Albania focused more on number content topics. It is noteworthy that teacher reports of implemented curricula (Fig. 4) only partially aligned with the intended curricula (Fig. 2). Overall, the coverage of the mathematics topics assessed by TIMSS varied across the region, with particularly low coverage in Bosnia and Herzegovina, and high coverage in North Macedonia (Fig. 4).

Fig. 4
figure 4

Average percentages of topics covered by the TIMSS 2019 mathematics assessment that students were taught.

Notes The 17 TIMSS grade four mathematics topics are described in more detail in Lindquist et al., (2017, pp. 15–18). In Kosovo and Serbia, the national defined population covers 90–95% of the national target population (for actual percentages and standard errors, see Table S.1 in the supplementary materials available for download at https://www.iea.nl/publications/RfEVol13)

Full size image

Teachers also reported the science content in the TIMSS 2019 assessment that they taught their students (Fig. 5). On average, teachers in Bosnia and Herzegovina, Croatia, and Montenegro reported particularly low coverage of the TIMSS science topics by grade four, and there was some regional variation in the range of topics that were taught. Montenegro, Albania, Bosnia and Herzegovina, and Kosovo had a greater focus on the life science topics, while Serbia and North Macedonia focused more on teaching physical science topics. Serbian teachers seemed to have particularly concentrated on teaching physical science content, while Croatia focused on teaching earth science content. In Bosnia and Herzegovina, Montenegro, and Albania, teacher reports indicated low coverage of earth science topics (39%, 42%, and 46%, respectively).

Fig. 5
figure 5

Average percentages of topics covered by the TIMSS 2019 science assessment that students were taught.

Notes The 26 TIMSS grade four science topics are described in detail in Centurino and Jones (2017, pp. 32–38). In Kosovo and Serbia, the national defined population covers 90–95% of the national target population (For actual percentages and standard errors, see Table S.2 in the supplementary materials available for download at https://www.iea.nl/publications/RfEVol13)

Full size image

3.3 Relating OTL with Mathematics and Science Achievement

3.3.1 Relationships Between OTL and Achievement Within Education Systems

The TIMSS data provides an opportunity to compare teachers’ descriptions of content taught with the achievement of their students, and thus investigate whether there is a relationship between the amount of content taught and student achievement. While the design of international large-scale assessments (ILSAs) does not enable researchers to draw conclusions about the direct impact of learning factors on achievement, it is possible to use bidirectional correlations as indications. However, when we analyzed the correlations between teacher reports on content coverage and student achievement, we found only spurious statistical evidence of such a relationship across Dinaric participants (Tables 3 and 4).

Table 3 Correlations between teachers’ reports on average content coverage of TIMSS mathematic topics and TIMSS mathematics achievement (overall and disaggregated by content domains)
Full size table
Table 4 Correlations between teachers’ reporting on average content coverage of TIMSS science topics and TIMSS science achievement (overall and disaggregated by content domains)
Full size table

We found that correlation coefficients were very low and, in general, not significant. There was a very weak positive relationship (indicating that higher achievement was related to more exposure to the topic) in the mathematics content area of number in Bosnia and Herzegovina, and exposure to science content generally, as well as earth science content in particular, was positively related to higher student achievement in Montenegro. Counterintuitively, in Albania, the more students were exposed to life science topics, the lower their scores in this content domain, but this negative coefficient was of very low magnitude. Such results are unexpected, but there are potential explanations. Teachers may not have been sufficiently confident to confirm the basic content as taught, and may have instead chosen the option “not yet taught or just introduced”. Another explanation may that that the topic as described in the TIMSS teacher questionnaire also encompassed topics that were not yet included in the curriculum, leading the teacher to again report the topic as not yet taught (a specific example might be that the conductivity of heat or electricity within the topic of classifying materials based on physical properties was not yet taught, although other aspects, such as weight/mass, volume, and state of matter were covered by the curriculum). However, additional system-level research is required to establish whether such theories are valid.

3.3.2 Alignment Between OTL and TIMSS Test Materials at the System Level

We referred to the test-curriculum matching analysis result for the Dinaric participants (see Mullis et al., 2020, Appendix C) to evaluate whether the content of the TIMSS test items was covered in the regional curricula. Achievement scores for each education system are recalculated based on the inclusion of only those TIMSS achievement items that were reported as included in the intended curricula up to grade four. This information provides additional context regarding a student’s OTL, as it indicates whether the achievement of a participant would change if only “matched” items were considered, and provides a wider perspective on the average national achievement, which might be very different if students only responded to curriculum matched items. The analysis also reconfirmed the number of items covered by the regional curricula. We found large differences in coverage across the Dinaric region; as an example, almost all TIMSS science items were covered by the North Macedonian science curriculum, but only half of the science test content was covered by the Croatian curriculum (Tables 5 and 6). According our test-curriculum matching analyses, across most of the Dinaric participants, in general their mean achievement would be the same, even if the TIMSS test had been based on items tailored to their specific curricula. There were a few exceptions to this: for example, Montenegro and Bosnia and Herzegovina would have performed better if their students had been assessed on a selection of items tailored to the Montenegrin curriculum, and students in Croatia and Serbia would also have achieved better scores if they had been assessed on items covered by the Croatian curriculum.

Table 5 Test-curriculum matching analysis with the content of the TIMSS 2019 test materials for mathematics curricula across the Dinaric region
Full size table
Table 6 Test-curriculum matching analysis with the content of the TIMSS 2019 test materials for science curricula across the Dinaric region
Full size table

3.3.3 The Net Effect of OTL on Student Achievement

To better understand the varying importance of these different aspects of OTL across the Dinaric region, we used the TIMSS topics coverage variables as predictors of mathematics and science achievement in regression models, controlling for values on the TIMSS HRL and LSN scales (Tables 7 and 8). The LSN scale and TIMSS topics coverage variables were reported at the class level by their teachers, but treated here as student-level features. The results confirm that there were large differences across the region in the relation between content exposure and achievement, even after controlling for limitations related to home support and classroom climate, but, in general, content exposure was not significantly related to achievement (Tables 7 and 8). After controlling for the factors attributable to lower home support and limitations for teaching due to students not ready for instruction, there was no significant positive relationship between the percentages of students that were taught the mathematics or science topics and student achievement in any of the education systems that we studied. However, in Bosnia and Herzegovina, Montenegro, and North Macedonia, we found that lower scores on the LSN scale were weakly associated with higher TIMSS mathematics and science achievement. Across the Dinaric region, the strongest significant predictor of higher student achievement in mathematics and science was having access to more home resources for learning. This association was strongest in Serbia and North Macedonia, but relatively weak in Kosovo, indicating that achievement in Kosovo depends less on factors that lie outside formal school settings than it does in other parts of the region.

Table 7 Amount of variance in students’ mathematics achievement explained by the regression model, standardized regression coefficients for TIMSS mathematic topics coverage, classroom teaching limited by students not ready for instruction, and home resources for learning
Full size table
Table 8 Amount of variance in students’ science achievement explained by the regression model, standardized regression coefficients for TIMSS science topics coverage, classroom teaching limited by students not ready for instruction, and home resources for learning
Full size table

4 Discussion

Our aim was to describe the opportunities to learn provided to grade four students across the Dinaric region, and establish whether this could be linked to their demonstrated mathematics and science achievement in TIMSS 2019.

To assess the effect of time exposure, we compared prescribed and implemented opportunities to learn. Across the Dinaric region, many of the participants reported that the nationally prescribed percentage of teaching allocated to mathematics and science was similar and they also reported comparable disparities in the actual implementation of the timetable. Like many other participants in TIMSS, official documents for the Dinaric region suggested that more time is devoted to teaching mathematics than science. We also found that participants in the region reported that similar percentages of total instruction time were prescribed for mathematics lessons, while there was more regional variation in the time spent teaching science. In Croatia, the nationally prescribed time for science instruction was at least 50% higher than in the other participating systems. However, across the Dinaric region, the implemented curricula, as reported by teachers and principals, were found to deviate from the prescribed teaching time.

The results from TIMSS 2019 are similar to other sources. According to a Eurydice report (European Commission, 2018) that assessed all grades of primary education across European countries, the bulk of school teaching time was devoted to writing and reading, then to mathematics, followed by science. In 2018, in almost all European countries at each primary school grade, the number of hours of teaching officially recommended for science was significantly less than the number of hours recommended for mathematics. According to the report, the percentages of teaching hours recommended for mathematics in the Dinaric region at that time ranged from 17.9% in Albania to 22.2% in Croatia, and the percentages of teaching hours recommended for science ranged from 5.7% in Montenegro to 13% in Bosnia and Herzegovina.

We analyzed NRC reports on nationally prescribed content coverage and compared intended curriculum coverage with the list of content topics that were assessed by the TIMSS 2019 mathematics and science tests and with teacher reports of which TIMSS assessment topics were taught to classes.

We found that there was lower overall coverage of the TIMSS mathematics content than the TIMSS science content. Across the region, in general, we found that more than half of the mathematics and science topics were intended to be taught to all students. For mathematics, measurement and geometry topics had the best coverage, followed by the TIMSS number topics. The three data topics were not covered at all by four of the Dinaric participants, while one topic of the three topics was covered in another two participants. North Macedonia was the only education system that prescribed different content to be taught to more able students. In all other participating systems, the same curriculum was delivered to all students. Teacher reports on the delivery of topics in in the classroom suggest many topics were covered. On average, teachers reported that almost two-thirds of students were taught all TIMSS mathematic topics in five of the Dinaric systems, while teachers from Bosnia and Herzegovina and Croatia reported fewer students received instruction related to the TIMSS content topics. Across the region, the TIMSS content related to the data domain was least likely to be covered.

Regional curricula were better at covering TIMSS science topics than TIMSS mathematics topics, but teachers’ reports of classroom delivery of content were more inconsistent. In Kosovo and Montenegro, NRCs reported that all of the 26 science topics used in the TIMSS assessment were covered in the curricula, and, generally, all participants reported good coverage of the science content topics. However, unlike mathematics, there were very obvious mismatches between the reports of the intended and the implemented curricula. While NRCs reported most science topics were covered, teachers from Bosnia and Herzegovina, Croatia and Montenegro reported that the percentages of students taught different science content topics in the classroom ranged from only 30 percent to 60% and, for the earth science content area, the coverage reported by teachers was less than 60% in another three participants. When we disaggregated the responses to establish which domains were taught, we found great variation in the attention given to physical science and life science across the region. In general, the earth science and physical science topics were less likely to be taught than the life science topics. Physical science topics were taught to most students in North Macedonia (85% of students) and Serbia (90% of students), while the life science topics were taught to most students in Albania (80% of students) and Montenegro (75% of students).

We also analyzed relations between opportunities to learn and learning outcomes. While there is ongoing debate about whether increasing instruction time in school increases student achievement (Andersen et al., 2016; Jez & Wassmer, 2013; Yeşil Dağlı, 2019), evidence has suggested that the quality of instruction and the time available for learning may have a positive effect on student achievement and even compensate for weaknesses in other areas, such as ability or willingness to learn (Gettinger, 1985). However, we found that, in the Dinaric region, the TIMSS 2019 data provided no evidence that spending more hours on mathematics or science led to higher achievement. This confirms data from earlier cycles of TIMSS, which showed that mean national achievement was not positively associated with average hours of instruction (see Martin et al., 2016, exhibit 9.1 and Mullis et al., 2016, exhibit 9.1). Our study also showed that, in Dinaric region, there was no unequivocal correlation between instructional time and achievement.

International research shows that other important elements can have a mediating effect, such as the quality of instruction or time and opportunities available for learning outside school (Jacob & Ryan, 2018; Özek, 2018). The positive relationship between increased instruction time and student achievement is more apparent when the increase is accompanied by other support measures and directed at disadvantaged students, for instance, those who come from less privileged families or home environments (European Commission, 2018). We therefore investigated the relationship of OTL with achievement taking into account two mediating variables: the accessibility of home resources for learning, as reported by parents in the TIMSS home questionnaire, and teacher reports of classroom teaching being limited by students who were not ready for instruction, which we used, respectively, as proxy measures of opportunities given to students outside school and quality of instruction. We found no strong general relationship between content coverage or topics taught and achievement. Although some Dinaric systems covered a large proportion of topics in their curricula or allocated more time to instruction, these differences did not translate into differences in student achievement. The presence of a weak relationship between content coverage and achievement showed that, along with teachers’ reports of which topics were taught, some students had not yet mastered topics reported as taught, and likewise did not know how to solve items that required knowledge of content not yet taught in school. The results showed that higher home support was an important predictor of higher science and mathematics achievement for students in all seven education systems, and an especially strong factor in Serbia and North Macedonia. Quality of teaching, as assessed by the teacher reports of feeling limited by students not ready for instruction, was found to be significantly related to lower achievement in Bosnia and Herzegovina, Montenegro, and North Macedonia. This suggests that the science and mathematics capital that students bring with them to school is potentially important. As student achievement seems to be strongly linked with opportunities available to them outside school, teachers and schools need to be especially aware of the importance of their role in providing quality OTL to students who have low levels of home support.

5 Conclusions

Our research demonstrated there was some variation in OTL across the Dinaric region. We noted discrepancies between intended, implemented, and attained curricula in terms of instructional time and content coverage in mathematics and science at grade four. Across the Dinaric region, there was no significant common relationship between the percentage of students that were taught the topics and mean national achievement in TIMSS 2019, but the reports of officially prescribed curricular contents did not align with teachers’ reports of the content taught in schools. School content coverage was not related to achievement. We found achievement showed a strong positive association with home learning resources and a weak association with content exposure in school. Establishing the teaching characteristics that could provide the best OTL requires further in-depth research, as learning can be facilitated in many ways and may also depend on teacher characteristics. Our central message to teachers is that more attention should be given to learning more about students’ existing knowledge, whether that was acquired in the classroom or outside school. Quality OTL in school is especially important for those students who lack the requisite home resources to support their learning.

This study demonstrates that the concept of OTL is extensive and plays an important role in student achievement. OTL also embraces the opportunities students have to learn outside schools. Teachers and policymakers therefore need to consider the mathematics and science capital that students are exposed to both inside and outside formal school settings, and provide additional school support for those students who lack the requisite resources and assistance in the home. Although new national policies may be necessary to achieve overall improvement, our analyses of the TIMSS 2019 data suggest that better teaching and learning of mathematics and science in the Dinaric region may be achieved by more rigorous focus on narrowing the disparities between the intended and implemented curricula. Understanding the reasons behind the observed differences is an important first step. Teachers may wish to carefully review the content of the prescribed curricula and compare this with what is presently taught in the classroom.