COVID-19 and U.S. Schools: Using Data to Understand and Mitigate Inequities in Instruction and Learning

Abstract

Shortly after the COVID-19 pandemic arrived in the United States, schools across the country had to enact significant, rapid changes to their instructional models, and schools varied widely in their access to the resources needed to support these efforts. Researchers across the U.S. quickly launched surveys, website reviews, and other data-collection methods to document these shifts. In this chapter, we draw on this research to describe the U.S. K-12 educational context, the policies states adopted, the practices and resources schools offered, and the potential effects on students’ academic, social, and emotional learning. In these discussions we draw particular attention to inequities in educational opportunities across schools serving different student populations. We then discuss how different sources of data will be needed to help identify educational needs and mitigate disparities in instruction and learning post-pandemic.

13.1 Introduction

In the U.S., the announcement of the COVID-19 pandemic by the World Health Organization in early March 2020 brought the significance of this health crisis to national attention. A combination of factors, including a fractured media landscape, low levels of trust in institutions, and lack of coordinated messaging across different levels of government resulted in widespread disagreement among Americans about the severity of the pandemic and how they should respond to it (Ipsos, 2020; Kavanagh, 2020). Moreover, there was slow recognition of the pandemic as a national emergency and limited guidance for and attention to the unprecedented disruption in all sectors of society, including education.

This lack of clear, centralized guidance, combined with the U.S. education system’s history of local control and the substantial differences in access to resources across schools and districts, led to wide variation in how schools responded to the pandemic (Audrain et al., This volume). Understanding these responses is crucial for informing future efforts to address the educational harms caused by the disruption to learning. Fortunately, researchers, education advocates, and others across the U.S. quickly launched data-collection activities to document schools’ responses and the effects of the pandemic on educators and learners. These data will play an essential role in helping policymakers and practitioners identify what interventions and other resources need to be provided and which groups of learners have the most urgent need for these resources.

In this chapter, we draw on several of these data sources to describe (1) the K-12 educational context in the United States when the pandemic hit and state policies and guidelines for remote learning, (2) how school closures and shift to remote learning impacted students and teachers, and (3) the ways that data can help identify educational needs and inform policy and practice to mitigate disparities in instruction and learning.¹ The large number of data-collection efforts undertaken in the wake of the first school closures have provided timely and crucial information about the pandemic’s effects. We do not attempt to review all these data sources for this chapter.² Instead, we draw on a small number of them to highlight some themes that are especially relevant to understanding the pandemic’s effects on educational opportunities.

These data allow us to paint a broad picture of the pandemic’s effects on educators and students and to identify potential sources of inequity. As we discuss throughout the chapter, however, the available data are insufficient for developing a thorough understanding of how the pandemic affected students’ learning opportunities and how it is likely to affect their social, emotional, and academic development. The final section of this chapter builds on what we learned about the utility of existing data on responses to the pandemic in the U.S., offering guidance for policymakers and others to promote a more comprehensive strategy for tracking inequities in the short and longer terms.

13.2 The U.S. Public Education Context and Policy Response to COVID-19

To make sense of schools’ responses to the pandemic, along with the challenges they faced in pivoting to new instructional delivery models, it is important to understand the broader policy context and education landscape in which public schools in the United States were operating. Since the founding of the U.S. public education system, the federal role in influencing school policies and operations has been limited. Instead, state and local (i.e., district) agencies and school boards have primary responsibility for setting school policy. Moreover, systems of school funding rely heavily on state and local taxes, resulting in substantial inequities in school funding that favor schools in wealthier communities (Baker & Corcoran, 2012; Ercikan & Elliott, 2015). Thus, when COVID-19 arrived on American shores, students were already experiencing unequal educational opportunities, with students of color and those from low-income families having less access to high-quality learning opportunities than other students (National Academies of Sciences, Engineering, and Medicine, 2019).

These pre-existing inequities and the lack of a strong federal role in education provide important background for understanding the pandemic’s effects. In the next section we briefly describe relevant aspects of the educational context as the pandemic started disrupting education, including school and family access to technology that could support remote learning. We then summarize the instructional policies that states, districts, and schools adopted in response to this disruption. This overview is intended to lay the groundwork for the subsequent discussion of the impact of the pandemic on instruction learning.

13.2.1 Educational Context in the Early Phases of the Pandemic

Policies regarding when and how to shift operations in response to COVID-19 varied greatly across states and school districts in the U.S. Starting the second week of March 2020, schools on the East Coast of the U.S. closed their buildings and transitioned to remote learning. Within a few weeks, school closures expanded to other parts of the country, resulting in closures of at least 124,000 of the estimated 132,000 U.S. public and private schools. These closures were expanded through the end of the 2020 academic year in 48 states and affected over 50 million public school students (Education Week, 2020).

The sudden transition to remote learning³ has been marked by disparities among schools. Schools in the poorest neighborhoods experienced the most significant challenges in moving to remote teaching and learning, exposing deep inequities that exist in the American education system. Examples of such challenges were observed in the state of Michigan. In this state, school districts were given flexibility of implementing different forms of remote learning, including delivering learning and instruction through cell phones, online classes, and by mailing materials to homes. In Detroit, one of the poorest school districts in Michigan and in the country, when schools closed in March, nine out of 10 students lacked access to digital devices (e.g., tablets or computers) and also lacked internet access (Associated Press Wire Content, 2020). The severity of the challenge was recognized by the state, which invested $23 M to get 51,000 Detroit students tablets by late April.

In addition to access to digital devices for remote learning, access to the internet has been another significant barrier to learning opportunities. Lack of internet access is not rare in the U.S. In February 2020, shortly before the first known cases of the pandemic in the U.S., the BroadbandNow research estimated that 42 million Americans did not have access to internet (Busby & Tanberk, 2020). Nationwide, 13% -26% households did not have access to internet in major cities in the U.S., with Memphis (26%) at the top, and New York city (13%) at the bottom. The combination of access to devices and internet can magnify the challenge of delivering instruction and learning to students at home.

As expected, the challenges associated with device and internet access affected school districts serving large proportions of low-income students at greater levels than other districts. Education Week conducted a survey of 2,600 teachers and school district leaders, first during the period of March 24–25, and a second round on April 7–8. The survey findings indicate enormous differences between school contexts that serve low-income students and others. In particular, 64% of school district leaders in school contexts with 75% or higher percentage of students from low-income backgrounds reported that technology access was a major problem, compared to 21% of district leaders in school contexts with 25% or fewer low-income students (March 24–25, 2020 data).⁴

13.2.2 Policies for Remote Learning

In April 2019, the Center on Reinventing Public Education (CRPE) reviewed information provided on district websites for 100 districts across the country (Dusseault, Heyward, Jochim, & Pillow, 2020). These districts served close to 10 million students, covered 50 states and 7 U.S. territories, and included the 30 largest districts in the country.⁵ Within a month of the COVID-19 pandemic this review revealed that even though the states made efforts to provide guidance and resources to districts, they fell short of providing clear expectations or adequate instructional resources to support remote learning or to remediate lost learning opportunities. Fifteen out of 50 states provided no directives requiring remote learning, leaving plans up to the local districts. In 29 states, state departments of education required districts to provide remote learning. However, little to no guidance or instruction was given for what was meant by remote learning. For example, Maryland required districts to provide remote learning but did not define what “remote learning” should include. Iowa removed requirements for instruction as long as districts provided a plan for “any methodology used to extend learning beyond brick-and-mortar district building.” Half of the districts in CRPE’s review did not require any remote learning plans.

In keeping with the long tradition of state and local control in the U.S. public education system, there was great variation in states’ approaches to providing guidance on what kinds of learning experience districts must offer. For example, the state education agency in Delaware required districts to provide detailed plans for their remote learning, including number of hours and days, by grade. A less directive approach was followed by the Alabama Department of Education, which provided a checklist of instructional delivery options, their curriculum providers, and the mechanisms of instruction.

CRPE’s review showed lack of clear guidance by state departments of education on remote learning, along with lack of centralized efforts in providing resources for supporting districts and schools in most states. This context, combined with existing disparities in education resources and the societal inequities in general, created a perfect storm for school-level decision-makers. In the next section, we summarize data on instruction and learning, beginning with a description of the opportunities schools provided for students to learn during the pandemic.

13.3 Impact of COVID-19 on Instruction and Learning in U.S. K-12 Schools

Although many of the short-term–and certainly the longer-term–impacts of the COVID-19-induced disruptions to education remain to be seen, early data suggest numerous troubling signs. The pandemic affected virtually all the nation’s K-12 students in some ways, and its effects are likely to be especially pronounced for groups of students who have been underserved by the education system in the past, given the longstanding, systemic inequities that had deprived these students of resources available to their more-advantaged peers.

In this section, we first summarize data on how opportunities for students to participate in education changed as a result of the pandemic, focusing on public schools serving students in grades kindergarten through twelve. We also present some data on how responsibilities for provision of instruction shifted from institutions to families and discuss potential implications of this shift for equity. We then discuss research that provides suggestive, early evidence on how these disruptions to schooling might have affected student learning. We conclude this section by exploring likely longer-term effects on students’ learning and well-being. Although we focus this discussion on instructional opportunities, it is important to recognize that schools in the U.S. provide social services, including meal provision and mental and physical healthcare, and that schools also serve as many families’ childcare providers. We do not address these aspects of schooling due to space limitations, but they are important considerations for understanding the pandemic’s broader effects on schools, students, and families.

13.3.1 Opportunities to Learn During Widespread School Closures

Shifts in Instruction during COVID-19

In the early phases of the pandemic, in addition to variations in access to digital devices and internet, there was significant variation in how teachers engaged and interacted with students online (Herold, 2020). Based on the Education Week survey, only 67% of teachers in schools serving high percentages of low-income students were engaging in any kind of instruction during school closures in March, whereas in schools with less than 25% low-income students, 89% of teachers engaged in instruction.⁶ By April, in all school settings instruction increased, though the disparities between lower-income schools and others remained. The lower rates of online instruction were accompanied by lower rates of in-person instruction and online opportunities for learning for low-income schools (see Fig. 13.1).

Fig. 13.1

Percentage of teachers engaging in any kind of instruction during school closures, by school-level family income in March and April 2020

The rapid shift to remote learning and the lack of universal access to technology almost certainly resulted in changes in curriculum coverage and other features of typical school-based instructional programs. Indeed, fewer than half of school principals reported having a plan in place pre-pandemic to deal with prolonged school closures, and many lacked other supports for remote learning, such as online learning management systems or relevant teacher training (Diliberti, Schwartz, Hamilton, & Kaufman, 2020).⁷ Even after schools shifted to remote learning, both teachers and principals reported significant gaps in training and other supports for remote learning, and they were especially in need of additional training to support students with disabilities and other groups with special needs (Hamilton, Kaufman, & Diliberti, 2020).

The near-universal closures of schools to in-person learning in spring 2020 was followed in fall 2020 by a variety of instructional delivery models. Some schools continued to provide fully remote instruction while others opened to fully in-person learning, at least temporarily. Hybrid models in which students split their time between in-person and remote learning became common, and in some schools, students or their families could choose from among various modes. Because of the planning and resources required to engage in fully in-person or hybrid models, students in under-resourced schools were less likely to have those opportunities (Diliberti & Kaufman, 2020). Data from RAND’s American Educator Panels reveal that smaller percentages of schools serving high proportions of low-income students (regardless of racial/ethnic composition) and those serving high proportions of students of color offered hybrid or fully in-person options compared with the full sample of schools (see Fig. 13.2).

Fig. 13.2

Note This figure is based on the following survey question to school principals: “Which of the following most closely reflects how instruction is provided to students at your school as of today?” Principals were asked to select from the following response options: “Fully remote instruction, where a large majority or all students receive at least one synchronous class each school day”; “Fully remote instruction, where a large majority or all students receive less than one synchronous class each school day (i.e., instruction might be distributed via paper workbooks or asynchronous videos)”; “Hybrid model, where a majority or all students receive some in-person instruction and some remote instruction”; and “Fully in-person instruction each school day for the majority, if not all, students.” FRPL stands for free or reduced-price lunch, a common but somewhat coarse indicator of low income. Source Diliberti and Kaufman (2020), Creative Commons Attribution 4.0 International License. Technical details including survey administration and weighting methods are provided in Kaufman et al., 2020)

Disparities exist in which students received remote instruction in Fall 2020.

Although we lack detailed national data on teachers’ pedagogical practices or the content they covered during the pandemic, survey data provide some evidence regarding the basic features of that instruction. In surveys administered at various time points during the pandemic, teachers reported that they spent more time reviewing old content and less time presenting new content than they had in the past, they were less likely to assign letter grades or provide feedback on student work, they interacted with their students less, and they were unable to reach all of their students or students’ families (Diliberti & Kaufman, 2020; Hamilton, Kaufman, & Diliberti, 2020; Kurtz, 2020; Lieberman, 2020). Only 12% of teachers reported in spring 2020 that they were able to cover all or most of the curriculum they would have covered had schools remained open, suggesting the potential for widespread gaps in student knowledge and skill development (Hamilton et al., 2020).

Reviews of school and district websites provide additional information about instruction during the pandemic. These websites are often a primary means of communicating with all stakeholder groups and can shed light on factors such as scheduling, grading policies, digital tools adopted, and attendance-tracking methods. An analysis of a nationally representative sample of school websites (Harris et al., 2020) found that the extent of schools’ personalization and engagement (e.g., use of live, synchronous instruction, provision of feedback on student work) was positively associated with local internet access but did not relate to schools’ racial/ethnic composition or socioeconomic status. The district website review by CRPE shows some system-level conditions that are aligned with school staff survey responses, particularly regarding inadequate resources and supports for vulnerable student populations including students with disabilities (CRPE, undated). Their longitudinal analysis indicated that districts’ strategies and resources improved from spring to fall 2020 but that districts continued to struggle to provide technology and keep track of all their students. An additional concerning set of statistics comes from a separate review of school district websites that estimated lost instructional time (Malkus, 2020). The study found that between March 2020 and the end of the 2019–2020 school year, students in the most economically disadvantaged districts lost 20 days of instruction on average, and even the most economically advantaged lost 16 days during the same period. This lost time was due to a combination of cancelled instructional days and student nonparticipation in instruction.

Together, the data summarized above point to widening opportunity gaps across schools. But because these data do not provide information about differences in students’ experiences within schools or classrooms, we are unable to assess the full impact of the pandemic on inequity. Even within the same classroom, some students will have received higher-quality instruction than others due to factors such as their ability to access remote instruction from home or their opportunities to study in quiet, sufficiently spacious environments. We return to the need for better data on within-school differences later in this chapter.

Addressing Educators’ Needs

Both teachers and principals have reported needing a wide range of supports for teaching and learning during the pandemic, including additional resources related to both academic instruction and broader student and educator well-being. One of the most significant needs, according to teachers, was strategies to keep students engaged and motivated to participate in remote learning, as shown in Fig. 13.3. Sizable percentages of teachers also expressed a major need for strategies to address the loss of hands-on learning opportunities, guidance or tools for assessing students’ social and emotional well-being, and tools and resources to enable counselors or school psychologists to support students. Moreover, 68 percent of teachers reported that the well-being of their students was an area of major concern, and another 26 percent described it as a moderate concern (Hamilton et al., 2020). These responses demonstrate that teachers’ concerns for their students extend well beyond academic performance. Many teachers also indicated a need for assessments, both for academic achievement and social and emotional well-being.

Fig. 13.3

Note This figure is based on the following survey question: “Please indicate your current level of need for additional support from school or district leaders in each of the following areas.” Response options were “No need,” “Very minor need,” “Moderate need,” “Major need,” and “Very major need.” Source Hamilton et al., (2020), Creative Commons Attribution 4.0 International License

Percentages of teachers indicating need for support from their District or school in each of the following areas, Spring 2020.

Interest in student well-being was not limited to teachers. In spring 2020, principals were more likely to report at least a “major need” for SEL instructional materials than for academic materials (Hamilton & Opfer, 2020). And 60% of district leaders in the fall 2020 American School District Panel Survey reported that addressing students’ SEL and mental health needs was an area for which they had the greatest need for support–higher than any of the other needs included in the survey (Schwartz et al., 2020). Addressing students’ SEL needs can be especially challenging when opportunities for in-person interaction and relationship-building are so limited. Elimination of non-academic activities such as classroom parties or extracurricular activities has understandably received less attention than changes to academic instruction, but these can be a valuable source of social, emotional, and academic development, and their absence in students’ school lives results in additional missed learning opportunities.

It is important to recognize that teachers’ capacity to support their students is influenced by teachers’ own well-being (Hamilton & Doss, 2020; Oberle & Schonert-Reichl, 2016), and surveys suggest that the pandemic harmed teachers’ well-being in several ways. Majorities of teachers surveyed in spring 2020 identified the following factors as areas of moderate or major concern: their own health or the health of loved ones (75%), responsibilities to care for their own children or other loved ones (59%), and feelings of burnout (54%) (Hamilton et al., 2020). By fall 2020, a full 80% of teachers reported burnout as a moderate or major concern, and roughly one quarter reported that they were likely to leave the teaching profession (Diliberti & Kaufman, 2020). As the pandemic dragged on, its negative effects on teachers’ well-being continued to mount, as evidenced by the three-quarters of teachers who reported in an EdWeek survey of 817 teachers in November 2020 that their morale was lower than before the pandemic (Will, 2021). School principals, too, reported struggling with low morale and anxiety (Brackett, Cannizzaro, & Levy, 2020). It will be important for those who support schools to keep these data in mind and recognize the value of addressing the well-being of both students and educators.

What Families are Saying

Considering the growing role that families have played in providing time and space for schooling, if not delivering instruction themselves, any predictions about educational effects of the pandemic would be incomplete without considering the challenges facing families. Surveys of families have revealed widespread concerns about students falling behind in school, and those whose children attended school partly or fully remotely were more likely to express these concerns than parents whose children received in-person instruction (see Jochim, Gundapaneni, & Pangelinan, 2020, for a synthesis of numerous family surveys).

These concerns are undoubtedly attributable in large part to the challenges families and students have encountered. The Understanding America Study, a nationally representative household pulse survey project, has tracked the perceptions of K-12 students’ families, finding that access to supports such as devices and internet improved from spring to fall 2020 but that many students still lacked access to a device they could use consistently or to reliable internet service (Saavedra, Rapaport, & Silver, 2020). This was especially the case for students of color and those in low-income households. In addition, although majorities of parents were satisfied with the instruction their children’s schools provided, parents of students of color and lower-income students reported greater concerns about several aspects of schooling, including access to social supports, than other parents (Jochim, Gundapaneni, & Pangelinan, 2020).

Among families of students whose schools were not offering fully in-person instruction, more than ten percent reported participating in “learning pods”–groups of students who learn together with support from one or more family members, tutors, or teachers. These pods have the potential to further exacerbate inequities, especially if more-affluent families have better access to networks and resources to support pods. However, the practice was more commonly reported by less-affluent families (Saavedra, Rapaport, & Silver, 2020). Many families have also switched schools or at least considered doing so, and again we see gaps, with affluent families more likely to consider private schools or full homeschooling and less-affluent families more likely to enroll students in charter schools. These changes to students’ learning environments and the accompanying differences by race or economic status complicate efforts to monitor students’ learning opportunities and mitigate opportunity gaps.

13.3.2 Likely Short-Term Effects of the Pandemic on Student Outcomes

The previous discussion highlighted the substantial challenges that schools faced and the widespread disparities in students’ opportunities to participate in high-quality educational environments. Even if schools returned to their pre-pandemic status by the start of the 2021–2022 academic year, some scholars predicted that the effects of the missed learning opportunities in spring 2020 and during the 2020–2021 school year will result in long-term, if not lifelong, disadvantage for the generation of K-12 students who experienced the pandemic (OECD, 2020). Our ability to understand effects on students’ academic learning and other outcomes is limited by a lack of high-quality data in addition to the short timeline, but evidence from earlier work on topics such as online learning and effects of natural disasters provides some basis for speculating about what is likely to happen, and more recently gathered data offers an early look at the pandemic’s effects.

What we Can Infer from Pre-Pandemic Student Achievement Data

The unprecedented nature of COVID-19’s impact on schools across the U.S. precludes any effort to draw strong inferences about likely effects on learning from pre-pandemic data. The most relevant evidence comes from two bodies of research on conditions that share at least one feature with the current context. First, scholars have turned to data on students’ academic achievement trajectories over summer break to predict what might happen to students who receive no instruction for weeks or months (Kuhfeld et al., 2020). Recent research suggests that differences in summer learning are not as great a contributor to racial/ethnic and socioeconomic achievement gaps as once thought (von Hippel & Hamrock, 2019) and that there is extensive variation in summer learning trajectories that is not easily explained by measurable demographic characteristics such as race/ethnicity or socioeconomic status (Atteberry & McEachin, 2020). In fact, the magnitude of this variation can exceed what is seen during a typical school year (Kuhfeld et al., 2020). This work suggests that the learning trajectories of individual students during school closures will differ substantially, but the sources of these differences are not clear from existing research. Of course, most students received at least some instruction while schools were closed in spring 2020, and even more did in fall 2020. Thus, the relevance of these summer learning trends is somewhat limited.

A second data source that might be more relevant, particularly when it comes to instructional delivery, is data gathered from students who participated in fully virtual schooling. Research has consistently linked students’ participation in fully virtual schools to lower rates of learning in several academic subjects, relative to the performance of similar students attending in-person schools (Ahn & McEachin, 2017; Fitzpatrick et al., 2020; Woodworth et al., 2015). The reasons for these differences are not clear but are likely to stem in part from high student–teacher ratios that limit students’ contact with teachers, challenges associated with monitoring and promoting student engagement in learning, and the inability of students with special needs to participate fully in online instruction (Gill et al., 2015). As we discussed above, these concerns are similar to ones that have arisen during COVID-19 remote learning, so the lessons from this body of research have clear relevance. At the same time, the instructional models schools adopted in response to the pandemic are almost certainly unlike the intentional, online-only models that most fully virtual schools have adopted, and the virtual schooling sector tends to serve a group of students that is not representative of the U.S. population of public-school students.

Kuhfeld and colleagues (2020) present additional relevant research on achievement effects of school absences, weather-related closures, and school disruptions stemming from natural disasters. The findings generally indicate negative effects of not being in school, but none of these situations closely match the COVID-19 context. Moreover, the COVID-19 period coincided with the widely publicized Black Lives Matter protests that brought significant attention to racial injustice. As Kuhfeld et al. (2020) point out, past research on killings by police suggests reason to believe these recent events could contribute to COVID-19’s negative effects on student achievement. Together, prior research points to a high probability of significant missed learning opportunities resulting in lower achievement test scores than students would have attained under more normal circumstances, but the magnitudes of these likely effects, and their relationships with pre-existing achievement disparities, are unclear.

What we are Learning from Early Data in the COVID-19 Era

Fortunately, data on how students are performing during COVID-19 are starting to become available. Using NWEA MAP© Growth™ assessment data for approximately 4.4 million students, Kuhfeld and colleagues examined fall 2020 reading and mathematics scores for students in grades three through eight, comparing them to the scores of students in the same grades in fall 2019 (Kuhfeld et al., 2020b). Their findings paint a somewhat more promising picture than the estimates discussed in the previous section: students’ reading scores in fall 2020 were similar to those in fall 2019, whereas their math scores were roughly 5–10 percentile points lower in 2020 than in 2019. The findings suggest students’ growth in math achievement was lower during the 2019–2020 school year than it would have been in a typical year. The authors note although the assessment was administered both remotely and in-person, there was a fairly high rate of missing data that disproportionately consisted of students of color, those with relatively low prior achievement, and those in schools that served large proportions of economically disadvantaged students. The likelihood that these groups of students are the ones most vulnerable to negative effects of the pandemic creates a reason to be less optimistic than the initial findings might suggest.

There is also evidence of negative effects on another measure of students’ academic performance: course grades for many students are lower than before (Strauss, 2020). In addition, survey data make it clear that teachers have concerns about their students’ learning. In the RAND fall American Teacher Panel survey, for instance, approximately a quarter of teachers said that most of their students were significantly less prepared to participate in grade-level work in fall 2020 than they had been at the same time last year, and these percentages were higher in schools serving majorities of students of color and economically disadvantaged students (Diliberti & Kaufman, 2020). Given the continued impact of the pandemic on school operations and the probable need for significant acceleration of learning to ensure that students get back on track, it will be crucial to monitor achievement test scores, grades, and other measures of student performance over the coming years.

13.3.3 Potential Longer-Term Effects on Student Learning and Well-Being

The findings discussed above begin to fill in some of the gaps in our knowledge about the pandemic’s effects, but what we do not yet know swamps what we do. We have not yet been able to examine the learning trajectories for different groups of students as conditions for learning in both schools and homes have shifted, and we do not know how persistent the effects of the pandemic on academic learning will be. Moreover, we know almost nothing about national and subgroup trends in areas that are not measured by existing academic achievement tests, including social and emotional learning as well as academic performance in non-tested subjects and grade levels. Finally, much of the existing research relies on a small set of tests that are particularly well-suited to examining learning trajectories but that do not capture the full range of competencies that students typically develop in their mathematics and reading classes.

Although these gaps in our knowledge make predictions difficult, existing data provide reasons for both concern and optimism. The group disparities in learning opportunities and outcomes discussed above are a source of significant concern: Unless policies and funding are intensively targeted toward supporting the students who suffered most from the effects of the pandemic, it is likely these disparities will remain, with significant consequences for students’ opportunities to pursue high-quality postsecondary education or rewarding careers once they complete their K-12 education. Beyond academics, these indicate the potential for negative effects not just on students’ mental health and well-being, but that of educators as well.

Despite this relatively gloomy outlook, a few bright spots provide some reason to believe that the pandemic could lead to better, more equitable learning opportunities if we respond constructively to what we are learning. One reason for optimism is that despite innumerable challenges, educators across the U.S. and at all levels of the education system sprang into action to shift instructional models in significant ways and with unprecedented speed (Hamilton & Opfer, 2020). The dedication that many educators displayed, and their willingness to try new strategies for connecting with their students, are promising signs for a post-pandemic education system–provided teachers receive the necessary supports, including training and working conditions, to sustain their work. These instructional shifts also spurred some innovations in remote-learning technologies along with increased attention to the factors that contribute to effective remote learning, such as family engagement and high-quality assessments. For instance, some educators experimented with breakout rooms in videoconferencing software or with virtual-reality platforms to facilitate small-group interactions (Allen, 2020).

In addition, although remote learning did not work well for all students, in some cases students appeared to perform better in their new home-learning settings and to enjoy school more as a result of conditions like reduced distractions or increased flexibility (Gilman, 2020). Teens, in particular, fared better than we might have predicted, according to a survey of approximately 1500 adolescents conducted in summer 2020 (Twenge et al., 2020). On average, these teens reported lower rates of loneliness and depression, less use of social media, and more time spent sleeping and engaging in activities with family members compared with pre-pandemic results–though, as with all the data we have discussed in this chapter–results varied significantly by respondent characteristics and should not be interpreted as evidence of a generally salutary effect.

A final reason for hope is that when schools closed their doors and families lost access to crucial supports, other community-based organizations such as afterschool programs often stepped in to help, offering resources such as a summer learning toolkit developed by the Denver Afterschool Alliance.⁸ Combined with the increased family involvement in instruction, these connections might lead to a deeper appreciation among policymakers that schools cannot meet all students’ needs on their own and that partnerships among families, schools, and other community organizations should be prioritized as part of a strategy to ensure equity and opportunity for all.

These small silver linings in no way compensate for the significant, widespread interruption to learning that most students experienced and that disproportionately affected students who lacked access to well-resourced schools before the pandemic. Up to this point, this chapter summarized what we have learned about COVID-19’s effects on learning in the U.S. public school system. In the next section, we discuss how the research and policy communities can contribute to better information and solutions going forward.

13.4 Advancing Measurement and Data Systems to Promote Equitable Opportunities in the Wake of the Pandemic

The widespread loss of learning opportunities, combined with substantial disparities in access to high-quality supports for learning and well-being, point to a need for both continued monitoring of opportunities and outcomes along with a well-coordinated, ambitious effort to ensure widespread access to needed supports for all students. Providing a comprehensive overview of what these supports should be and how they should be deployed is beyond the scope of this chapter, and we refer readers to other sources of such guidance.⁹ We instead discuss a particular need that educators and policymakers face as they attempt to recover from the wreckage of the pandemic–the need for high-quality data to mitigate harms while creating new, improved learning opportunities in the coming years.

The significant changes in when, where, and how learning occurs will have long-term effects even after the pandemic ends (Schwartz et al., 2020), and educators will need new ways of monitoring learning in non-traditional settings, along with sources of data that provide consistent information about student learning. As harmful as COVID-19 has been to students, families, and educators, its effects on the educational landscape provide an opportunity to rethink what and how we measure learning opportunities and outcomes. In this final section of the chapter, we highlight some of the most important ways in which measurement can be applied to promote equitable, high-quality learning for all students.

13.4.1 Monitoring Opportunity to Learn

Data on opportunity-to-learn (OTL) will be more important than ever given the widespread shift to different modes of schooling (Marion, 2020). Efforts to address students’ learning needs will require understanding the learning environments they have experienced, both at school and elsewhere, and the gaps in their exposure to grade-level content and other instructional supports. And data will need to reveal not just between-school differences but also differences in students’ access to instruction within schools and even within the same classroom.

Monitoring OTL requires ongoing collection of data on the kinds of instructional activities in which students engage and the learning resources to which they have access. The expansion of learning in digital environments is likely to continue to influence instruction post-pandemic. An advantage of many digital learning environments is the explicit documentation of implemented curriculum and instruction and the opportunities to observe student engagement in learning activities. For OTL data to inform practice and policy, features of digital learning and student engagement will need to be systematically measured and analyzed at student, classroom, and higher levels. In addition, the digital learning resources can be examined to identify the degree to which these resources covered targeted curriculum and learning outcomes.

Another source of OTL data is documentation of student learning activities in the log files that many software packages generate. Such log data can provide information about whether and when students log onto learning environments, how much time they spend on different activities, and how they navigate through the learning activities, among other types of engagement with the learning environment. All these data about students’ interactions with learning tools can help evaluate the degree to which and how students have been engaging in the digital learning resources on an ongoing basis. In light of the earlier discussion of challenges related to student engagement and access to remote instruction, such data could help policymakers and education leaders target resources to the students who are most in need of additional learning support.

13.4.2 Monitoring Learning Outcomes to Inform Policy and Practice

In general, assessments at all levels need to be considered with their targeted uses and impact in mind. In the current educational context, dire educational needs and the role of assessment in meeting those needs should determine which, how, and when assessments are conducted. Two educational needs are of primary relevance: (1) the learning opportunity loss due to school closures; and (2) growing disparities in learning outcomes for students from different racial/ethnic and socioeconomic backgrounds. We highlight key information requirements and the role of educational assessments for both of these needs below.

Information needs and role of assessment for addressing learning opportunity losses

Currently we do not have adequate information about the degree of learning opportunity losses and how they varied across student groups and at individual student levels. Addressing these losses requires actions at the individual student level as well as the group and system levels. Accordingly, different types of assessment information are needed for these uses. In particular, to address individual student needs, educators will benefit from assessment information that is tied to students’ ongoing learning activities and that provides fine-grained information about how students progress through the curriculum. Such an intended use is referred to as formative assessment and requires the assessments to be closely tied to instruction both in terms of what they assess but also with respect to when the assessment is conducted and how the assessment information is used. Greater use of digital platforms for learning and assessment can facilitate access to such types of assessments by teachers and students. These can be in the form of well-designed assessment tasks that can be embedded within learning activities.

In addition to formative uses of assessments, information is needed to inform policy and planning at classroom, district, and state levels. Well-designed, large-scale assessments at state levels can provide such information. Several aspects of these assessments are critical in order to meet such goals. Ercikan and Barclay-McKeown (2007) identify five requirements in order for large-scale assessments to guide educational policy and practice:

• 1. assessments need to assess valued outcomes;

• 2. assessments need to be aligned with learning, instructional, and curricular goals;

• 3. assessments need to provide accurate estimates of student knowledge and competencies;

• 4. reports of assessment results need to be informative and timely;

• 5. intended and unintended consequences of uses of assessment results need to be important considerations (p. 58).

Each of these requirements gain heightened importance and meaning in the current educational context as they are discussed below.

Valued learning outcomes. Focusing on valued learning outcomes such as problem solving and critical thinking, instead of recall of factual knowledge, has been the focus of great discussion in education for over several decades. In the current educational context, focus on these types of learning outcomes is important for effective preparation of students for higher levels of learning, preparation for higher education and careers. In addition, in light of the prevalence of misinformation and mistrust of evidence about COVID-19, and other natural, societal and political phenomena (Hamilton, Kaufman, & Hu, 2020; Kavanagh & Rich, 2018), other constructs such as civic knowledge, skills, and dispositions have particular relevance.

Alignment. In the current pandemic context, alignment with learning, instruction, and curricular goals requires a greater focus on OTL given the disparities in learning opportunities. There is great research evidence that assessments that are not closely aligned with instruction and learning opportunities do not provide valid evidence of student learning (Ercikan, 2006; Moss, Pulin, Gee, Haertel & Young, 2008; Linn, Baker & Dunbar, 1991). This alignment is critical if the assessments are to inform and support learning.

Accuracy of measurement. Great disparities in opportunities to learn have added challenges for assessment with respect to the range of knowledge and skills the assessments need to cover and variations in student learning outcomes. These disparities require assessments to be adaptable for different levels of learning outcomes, with difficulty levels appropriate for student levels, and to provide accurate and generalizable information about student competencies.

Reporting of assessment results. Usefulness, interpretability, and timeliness of reports are critical for informing policies for addressing the impact of learning opportunity losses and closing disparities in learning outcomes. The reports need to provide actionable information at different levels of the education system to identify the degree of impact of the learning opportunity losses, which student groups, schools, and districts have been disproportionally impacted, at points in time in the school year for effective implementation of the policies.

Intended and unintended consequences. The focus here should be on designing assessments that will support intended consequences and avoid inappropriate interpretations and uses of assessment results, and therefore minimize unintended consequences. Informing strategies to mitigate learning losses should be the central focus of large-scale assessments. One aspect of supporting the intended uses involves assessments to allow comparisons of learning outcomes with previous years so that educators can identify the greatest gaps in students’ knowledge and skill development. Having a pre-COVID-19 baseline will help users of the data understand the effects of the pandemic while also providing helpful information for addressing longstanding differences in performance.

Users of assessment data should be wary of ways in which assessment use can lead to inaccurate inferences and to potential harms for students or educators. Use of assessments for school or teacher accountability, in particular, has been associated with negative effects on instruction as well as threats to the validity of inferences that the scores will support (Ercikan & Barclay-McKeown, 2007; Koretz, 2008; Stecher et al., 2018). In the current context, risks associated with accountability uses of tests are exacerbated. An expert panel on assessment in the context of the pandemic laid out several key principles, one of which was “do no harm” (Lake & Olson, 2020). With disparities in digital device and internet access, in addition to educational support resources, the focus should be on using assessments to support learning and development of all students.

Those who mandate, administer, and use assessments must also consider the unique threats to validity that the pandemic has created. As we discussed earlier, efforts to track learning trajectories during the pandemic have been thwarted by incomplete data from students, including those who were unable to access the assessment remotely or have been unable to attend school at all. Other concerns include lack of validated approaches to assessing English learners, students with disabilities, or other students who need assessment accommodations, as well as potential security threats to remote testing. A long-term investment in better assessment options will be needed to accommodate future shifts in instructional models, but in the meantime, users of test-score data will need to apply appropriate caution when interpreting the results.

13.4.3 Monitoring Social and Emotional Learning

The concern that teachers, principals, and district leaders expressed about promoting students’ social and emotional learning (SEL) reflect a widespread understanding that all learning brings together aspects of students’ academic, social, and emotional competencies (Aspen Institute, 2019). The expansion of instructional delivery into new modes and new settings could have implications for students’ opportunities to establish supportive relationships and to develop competencies such as teamwork and self-regulation. A comprehensive approach to data collection to inform decisions about teaching, learning, and resource allocation should include both academic and SEL measures (Lake & Olson, 2020).

Selecting the right SEL measures and using them in ways that will benefit rather than harm students, can be especially challenging due to the lack of widely available, validated assessments, the minimal training most educators receive to assess SEL, and a lack of clear guidance regarding how to use the results of assessments to inform instruction (Hamilton & Schwartz, 2019). It can be especially difficult to find SEL assessments that are appropriate for all students, regardless of cultural background and other personal circumstances (Jagers et al., 2018). And of course, the remote-learning context creates additional barriers to SEL assessment.

As with the other types of assessment discussed above, the shift to remote learning, and the growing prevalence of digital instructional materials and communication methods, creates opportunities to re-evaluate traditional approaches to monitoring SEL. Informal data-collection practices such as regular individual check-ins between teachers and students can provide information that teachers can use to inform their decisions about SEL instruction while also enabling them to identify students who might be in need of additional resources such as counselors or other mental-health professionals. Educators have been creative about exploring new ways foster SEL, such as by encouraging student collaboration and teamwork through monitored breakout rooms in videoconferencing software. Moreover, a growing number of digital instructional tools are available to promote and assess SEL (see, for example, Zoo U¹⁰).

Beyond the resources provided to students, data on educators’ social and emotional well-being is also crucial for monitoring educational opportunities. As we discussed earlier, educators’ well-being can influence their instruction and can be associated with attrition, so it is inherently linked to the quality of student learning opportunities. Online tools such as peer learning communities can help mitigate the stress associated with teaching (especially but not only during a pandemic), and inexpensive assessments approaches such as regular brief surveys can provide data on educators’ emotional well-being so that policymakers and education leaders can intervene as needed.

It is important to monitor for mental-health problems in both students and educators, but this does not mitigate the need for asset-based, universal SEL that builds the competencies that all students will need for success in school, the workplace, and the broader community. Information about SEL should be gathered in an ongoing way rather than only at the end of a course or semester, when it might be too late to intervene. Finding ways to build SEL data collection into existing tools and assessment systems can help address time and resource constraints while providing educators and policymakers with crucial information to inform their work.

13.5 Conclusion

COVID-19 caught nations–including the United States, the wealthiest nation in the world-off guard in meeting the needs of their populations. As the schools closed, along with workplaces and businesses, the impact of shutting down the education systems were felt not only in well-being of students and teachers and learning outcomes, but in economies of all the countries. There is now much greater understanding and appreciation of the role of education in the development of children that goes beyond learning and facilitating opportunities for parents to participate in the labor market.

We propose data–on both student outcomes and learning opportunities-as key to addressing the short-term and likely long-term impact of COVID-19 on learning opportunity losses and on disparities in educational outcomes. As the modes and forms of learning change over time, as does the concept of schooling, there is a need for a continuous-improvement approach to trying out innovative data-collection and assessment strategies, monitoring their utility, validity, and fairness, and modifying them in response to evidence. Those who develop or deploy new assessments will face growing pressure to document validity, reliability, fairness, and utility of scores on these tests and to provide users of the data with supports and guidance to ensure appropriate interpretations and uses.

This data-informed approach will require coordinated action on the part of several groups. Policymakers and funders should explore ways to support both large-scale data collection for monitoring purposes and smaller-scale assessments for formative, instructional purposes, along with research and development to ensure that these data produce the best-possible evidence. Educators must gather and use data in ways that benefit all students, and organizations that support and train educators need to equip them to do so. Furthermore, students should be encouraged to share their perspectives and experiences. Thoughtful, judicious use of assessments that monitor students’ academic, social, and emotional learning, as well as their learning environments, can be a cornerstone of a broader strategy to help the education system not only recover from the pandemic, but thrive in the post-pandemic era.