Introduction

During the early spring of 2020, as a reaction to the COVID-19 pandemic, colleges, and universities across the USA (as well as the entire world) switched from traditional face-to-face instruction to remote (or online) learning. A combination of public health measures imposed by federal, state, and local authorities and a desire to minimize any human-to-human interaction prompted a radical change in content delivery. Employing an isolation strategy, students, faculty, and staff, to the highest degree possible, were instructed to avoid contact. Designed to stop, or at least slow down, the spread of a deadly virus, a “lockdown” went into effect.

Aware that the challenges of the pandemic, including the lockdown and sudden change to remote learning, would, with a high probability, affect the mental health of students and the quality of learning, universities across the USA-adopted some version of a flexible grading policy for students. Flexible grading included giving the students the option to maintain their earned letter grades in the course, opt for a pass/no-pass course grade, or retroactively withdraw from the course, i.e., earn no credit. At the heart of the change in grading policy was the desire to allow students to continue to learn while ensuring safety and wellness. The most prestigious universities (Basken, 2020), state universities (Venable, 2020) and community colleges (Bird et al., 2022) offered a version of flexible grading.

Enacting radical changes to policy in a short period of time produces a variety of outcomes, whether the outcomes are well-intended or not. Large-scale shifts in policy over a short time have significant implications for undergraduate students, academic programs, extracurricular endeavors, and college or university administrators. This study analyzes the implications of the COVID-19-induced flexible grading policy at a public Historically Black College and University (HBCU) located in the southeastern region of the USA. We study the differential selection of flexible grading options by course characteristics, students’ sociodemographics, and students’ academic profiles over three semesters, Spring 2020 through Spring 2021. We also study the impacts of the flexible grading policy on sequential courses (e.g., Calculus I to Calculus II). Specifically, we address two main research questions:

RQ1: How were the course characteristics, students’ sociodemographics, and students’ academic profiles associated with the utilization of the flexible grading policy options at different stages in the lifespan of the policy?

RQ2: How did the use of the flexible grading policy impact sequential courses?

By examining the ways in which the utilization of the flexible grading policy options varied across courses and among students of varying sociodemographic backgrounds and academic profiles, and the impacts on sequential courses, the study aims to provide valuable insights into the ways in which such policy may impact student success and promote academic equity during periods of upheaval. Through this rigorous analysis, the study seeks to inform the development of evidence-based practices in higher education.

The current study adds to a growing body of literature investigating the impacts of the COVID-19 pandemic on higher education (e.g., Altindag et al., 2021; Aucejo et al., 2020; Bird et al., 2022; Hartzell et al., 2021; Iglesias-Pradas et al., 2021; Jaeger et al., 2021; Kofoed et al., 2021; Rodríguez-Planas, 2022a, 2022b; Shin and Hickey, 2021; Whitelaw et al., 2022; Zhou & Zhang, 2021). Despite the great attention given to documenting the consequences of the pandemic on college education, few studies discussed the implications of the flexible grading policy induced by the pandemic (e.g., Rodríguez-Planas, 2022b). Our study aims to fill this gap by conducting a focused analysis of the COVID-19 flexible grading policy at a public university in the USA.

The rest of this paper is organized as follows. In the background and context section, we briefly review two grading systems that have been employed in higher education. As described above, the COVID-19-induced flexible grading policy blended these two systems. In the same section, we describe the flexible grading policy at the study institution and more broadly at the state’s university system. A summary of previous work on the flexible grading policy and its connections with the current study concludes the background and context section. The methodology section describes the data sources and analysis methods. A results section presents the outcomes of our analyses. A discussion section contextualizes the findings of the paper. Finally, the conclusion section summarizes the key findings and implications.

Background and Context

Two Familiar Grading Systems in Higher Education

Grades are considered a measurement of progress and academic achievement (Venable, 2020). They are useful and deeply engrained in university life. Here we review two systems that are commonly used to assign grades in higher education: namely, the A-F letter grading system and the pass-fail grading system.

The A-F letter grade system is a short, quick, easily understood communication device. The letter grade acts as a facile method to communicate either progress during the course (a formative grade) or overall achievement (a summative grade). The basis for a letter grade can be norm-referenced (e.g., a bell-shaped curve) or criterion-referenced (e.g., performance against a standard) (Gronlund, 1982). Letter grades can be linked to standards (Chappius & Stiggins, 2016); they serve as a tool for scholarships, matriculation, transfers, and post-graduate work; and they communicate standing and rank (Schneider & Hutt, 2014). For an undergraduate student in the USA an “A” in a Calculus course, for instance, represents an achievement of 90% (or better) of the course goals or outcomes.

The letter grade from a course can be quantized and grades from multiple courses can be averaged. Administrators and advisors currently find it convenient to equate a letter grade to a number, where an A, for example, is assigned 4.0 grade points. Then, grades from other courses are combined to compute a grade point average (GPA). The GPA allows for quick comparisons. Consequently, students are conveniently ranked, sorted, and categorized (Schneider & Hutt, 2014; Soh, 2011). Due to this operational convenience and many other attractive features of the A-F grading system, this system is firmly entrenched in the United States (Schneider & Hutt, 2014).

On the other hand, A-F letter grading has been repeatedly criticized for its negative impacts on student well-being and learning due to the competitiveness it places on students. Chamberlin et al. (2018) give a long list of negative effects of letter grades which includes enhancing stress and anxiety, encouraging cheating and reducing academic motivation, critical thinking, and cooperative learning (see Chamberlin et al., 2018 and references therein). An alternative grading system that was deemed to address these shortcomings of letter grades is the pass–fail (P–F) grading system.

The P–F system was extensively tested and studied in the sixties (e.g., Karlins et al., 1969; McLaughlin et al., 1972; Bain et al. 1973). Under this system, students earn a “Pass” if they score above certain cutoff (e.g., D or better letter grade) and a “Fail” if they do not meet the cutoff. By eliminating the unhealthy competition inherent in the A-F grading system, the P–F system is advocated to have several desirable effects on students’ well-being and learning outcomes. Studies on P–F grading have found that it can help reduce students’ anxiety, stress and burnout (e.g.,Bloodgood et al., 2009; Rohe et al., 2006; Spring et al., 2011), allow students to explore courses outside the majors without penalizing their GPAs (e.g., Bain et al. 1973; Weller, 1984), encourage collaboration and self-regulated learning (e.g., White & Fantone, 2010), and increase first-year student retention (Stiller & Paguyo, 2012). For these attractive qualities, the P–F grading system is quite prevalent in medical school programs, especially in pre-clerkship years, in the United States (e.g., AAMC, 2020; Rohe et al., 2006; Smith & Piemonte, 2022). Indeed, the vast majority of research concerning P–F grading after 1980 has been concentrated in medical education.

The P–F system is not without faults. Among the shortcomings of the P–F system is that it was frequently found to be associated with lower student achievement (e.g., Sgan, 1970; Gold et al., 1971; Wittich, 1972; Bloodgood et al., 2009), as students aim to meet the minimum requirement. P–F grading was also criticized for its inability to identify at-risk students due to those students being shrouded within the broad “pass” category (e.g., Gonnella et al., 2004). These flaws and the relatively insufficient research supporting P–F grading have led this grading system to diminish from American colleges and universities apart from medical programs.

Flexible Grading at the Time of COVID-19

The COVID-19 pandemic represented a catastrophe that overwhelmed and incapacitated local, state, and national agencies. In response to this catastrophe, institutions of higher education employed several measures to mitigate the spread of the virus and help students overcome the related challenges. Such measures included expanding the options to convert A-F grades to P–F grades. Given the Colleges and Universities' preference for A-F letter grades, they consequently restricted the number of P–F courses used toward graduation. Universities give credit for the work but exclude a passing grade from GPA calculations. Hence, P–F options are usually shunned, and for good reason, too. In a very practical sense, P–F courses hinder the student’s post-baccalaureate ambitions. Many graduate schools do not accept P–F courses. Even within the same university, if a student decides to switch majors or change programs, acceptance into the new major can be predicated on strength of the GPA. Hence, the action of a change in grading policy across the university to allow for a larger number of P–F courses impacts GPA’s, opportunities for transfer, and graduate or professional school admissions.

To date, there is dearth of literature on the implications of the COVID-19-induced flexible grading policy. At the time of writing this paper, we are aware of only two peer-reviewed studies discussing flexible grading, namely, Gelles et al. (2020) and Rodríguez-Planas (2022b).

Gelles et al. (2020) conducted qualitative interviews with 11 second-year Engineering students from a private university in California to explore how they adapted to the transition to remote learning during the Spring 2020 semester. They noted that students generally appreciated the flexible grading option, especially females who frequently acknowledged utilizing the P–F option.

Rodríguez-Planas (2022b) used administrative records from Spring 2017 to Spring 2020, complemented with transcript data for Spring 2020, to study the effects of the pandemic on students’ academic performance at Queens College, New York. She focused on investigating the differential effects of the pandemic on students’ Spring 2020 GPA by their pre-pandemic income status. Her analysis grouped students into lower- versus higher-income classes based on their financial aid status with those who ever received a Pell grant classified as lower-income students. She found that students used the flexible grading policy to improve their GPA. Specifically, she reported that the flexible grading policy was responsible for 43.7% of the Spring 2020 GPA increase among higher-income students (never Pell recipients). Additionally, she found that during Spring 2020 lower-income students earned a 5.1% higher GPA and failed 28% fewer credits relative to their higher-income peers. However, when she estimated the effect of the pandemic on Spring 2020 GPA prior to factoring in students’ elections of the flexible grading option, this GPA differential by income status vanished. That is, the difference in academic performance by income status was fully explained by the selection of the flexible grading policy. Overall, Rodríguez-Planas’ (2022b) analysis reveals the students’ differential use of the flexible grading policy based on their financial and academic needs. It also suggests that the flexible grading policy was effective in counteracting the negative shocks of the pandemic, especially among economically disadvantaged students.

Flexible Grading at the Study Institution

This study focuses on a public Historically Black College and University (HBCU), which is one of 16 universities under the auspices of the University of North Carolina (UNC) System. Within the UNC System, each institution crafted its own unique flexible grading policy during the COVID-19 pandemic. See Table 6 in Appendix for a list of UNC institutions and their flexible grading policies. Policy variations included the length of time to invoke the policy options (i.e., before finals, during finals, or even weeks after the semester ended); the name of flexible grading options (“pass-fail” or “satisfactory-unsatisfactory”); and the number of semesters offered (e.g., UNCW for only Spring 2020, ECSU for Spring and Fall 2020, and NCAT for Spring 2020, Fall 2020 and Spring 2021).

As time passed, many universities modified their flexible grading policy. Factors such as better understanding of the virus transmission, increase in vaccination, adherence to social distancing protocol, and epidemiological data on infection rates impacted the policy. For example, at the flagship institution, UNC at Chapel Hill (UNC-CH), the flexible grading policy for Spring 2020 allowed students to decide whether to opt for the P–F option or retroactive withdrawal by August 7, 2020, i.e., over two months after the end of semester. Then, for Fall 2020 and Spring 2021, the administration modified the flexible grading policy, requiring a decision to opt for the P–F option before the end of the term (Flaherty, 2020). In addition, course withdrawals made after August 31st for Fall 2020 or after February 1st for Spring 2021 were excluded from the 16-hour withdrawal limit (UNC-CH Registrar, 2020a, 2020b).

At the study institution, the flexible grading policy was most similar to the UNC-CH policy with some notable differences. In Spring 2020, students were able to select the P–F option or a retroactive withdrawal by June 30, 2020. A student with a grade A–D in any course could choose to transform their course grade into a P1 (Pass) which earns them course credit but does not count in GPA calculations. Students with an F grade could retain their F grade, which would count in their GPA calculations, or request a retroactive withdrawal (WE) which does not count towards their GPA nor their 16-hour withdrawal limit but may affect their financial aid eligibility and their satisfactory academic progress status. There was no limit on the number of courses where students can use the P–F option. The policy was amended to limit the P–F option to a maximum of 3 courses in Fall 2020 and a maximum of 1 course in Spring 2021 and to remove the retroactive withdrawal option. Unlike UNC-CH, students at the study institution were permitted to select the P–F option after the end of semester for Fall 2020 and Spring 2021. Similar to UNC-CH, the study institution excluded course withdrawals made by the university’s normal withdrawal deadline from the 16-hour withdrawal limit. These withdrawals were also coded as WE on the student transcript. It is worth noting that the WE category in Spring 2020 only includes withdrawals made after the end of semester (retroactive withdrawals) whereas the WE category in Fall 2020 and Spring 2021 includes only course withdrawals made by the normal withdrawal deadline.

The primary objective of the current study is to gain a comprehensive understanding of the differential utilization of flexible grading options across courses and among students of varying sociodemographic backgrounds and academic profiles at an HBCU. Furthermore, our study presents a novel analysis of how the use of the flexible grading policy impacted students’ performance in sequential courses in mathematics, chemistry, and English. Similar to Rodríguez-Planas (2022b), our study uses transcript data and administrative academic records for undergraduate students at one institution. In contrast with Rodríguez-Planas (2022b), which utilized data from Spring 2017 to Spring 2020 to study the immediate impact of the pandemic on performance while controlling for pre-pandemic academic performance, our study utilizes data spanning from Spring 2020 to Spring 2021 to provide in-depth, albeit descriptive/correlational, analysis of the flexible grading policy. Additionally, our study used data from all undergraduates who were enrolled at the study institution in any of the three semesters from Spring 2020 to Spring 2021 whereas Rodríguez-Planas used a sample of 2817 students (23% of all undergraduates at Queens College) who voluntarily participated in an online survey on their experiences during Spring 2020. Moreover, the socio-demographic characteristics of our student population differ from Rodríguez-Planas’ student population in many ways. The study institution is an HBCU with 83% of the student population being Black and only 4% Hispanic, whereas Queens College is a more diverse Hispanic Serving Institution (HSI) with 30% Hispanic, 29% Asian, 23% White and only 9% Black. The study institution has a significantly higher average annual cost and somewhat lower graduation rate than Queens College ($10,563 vs $3469 and 50% vs 56%, respectively). The two student populations have similar socioeconomic distribution with 61% of our student population being Pell recipients compared to a 58% of Queens College’s students.

Methodology

Data

The study was reviewed and approved by the University’s Institutional Review Board (IRB-19-0202). The study used two types of data for the first three semesters of the pandemic, Spring 2020, Fall 2020, and Spring 2021: course-level data obtained from transcripts and student-level data obtained from administrative records. The course-level data contained information on the following variables:

  • Course subject: representing the subject of the course (e.g., biology, mathematics, psychology, etc.),

  • General education (Gen Ed) status: whether the course meets a general education requirement, and

  • Course grade: the letter grade (e.g., A, A-, B+, etc.) students earned in the course along with whether they used the flexible grading option noted as P1 for the “pass” option and WE for the “withdrawal” option.

The student-level data included sociodemographic and academic profile variables. The sociodemographic variables were:

  • Sex: student's sex with categories “female” and “male,”

  • Rurality: whether the student is from a rural area with categories “no” and “yes,”

  • Residency: whether the student is “in-state” or “out-of-state” student, and

  • Financial aid status: whether the student received financial aid (Pell grant) during the semester.

The academic profile variables were:

  • Classification: student’s classification at the institution with categories “Freshman,” “Sophomore,” “Junior” and “Senior,”

  • Full-time status: whether the student is a full-time or part-time student during the semester, and

  • STEM status: whether the student is a Science, Technology, Engineering or Mathematics (STEM) major.

Additional covariates that were controlled for in our analyses included:

  • Online experience: whether the student has taken any online course over the past 5 years,

  • ACT/SAT: standardized ACT/SAT composite score (Most of the students had SAT scores, but some students had ACT scores. Both ACT and SAT scores were changed to z-scores. Then, SAT z-scores were used for students with SAT scores available, and ACT z-scores were used for those without SAT scores), and

  • Gen Ed Courses: the number of general education courses attempted during the semester.

Course-level and student-level data were merged for analysis using the anonymous students’ IDs generated by the registrar's office in the original data files. The data used in the study included all undergraduate students and the courses taken by these students at the study institution from Spring 2020 to Spring 2021 (See Table 2 for sample size).

Methods

To address the research questions of this study, we used a combination of descriptive statistics, data visualization, and statistical modeling techniques. We started our analysis by cross-tabulations describing trends and variations in the selection of flexible grading options by course subject area and students’ sociodemographic characteristics over three semesters; Spring 2020, Fall 2020, and Spring 2021. We then used two types of statistical models to identify the factors associated with increased likelihood and/or frequency of selection of the flexible grading options. Specifically, we used logistic regression (e.g., Hosmer et al., 2013) for modeling whether the student selected a flexible grading option (binary response variable; yes or no) as a function of several explanatory variables including the course subject area and the student’s sociodemographic characteristics. We also used negative binomial regression (e.g., Hilbe, 2012) for modeling the frequency of selecting a flexible grading option (count response variable; number of courses in which the student chose to apply a flexible grading option), as a function of the student’s sociodemographic characteristics, and their academic profile (e.g., pre-college preparation, prior online course experience, STEM status, and course load). To capture possible changes in the association between the selection of flexible grading options and the explanatory factors at different stages of the lifespan of the flexible grading policy, we built separate models for each semester. Alternatively, we could build one model for the data from all three semesters and include semester as an explanatory variable in the model. However, the first approach provided a simpler way to explore and present the changes in the associations over time.

Finally, we analyzed the longitudinal implications of the selection of the flexible grading options through correlational analysis of students’ course grades in popular course sequences, namely, general mathematics, general chemistry, and English sequences (see Table 1). These course sequences have the greatest numbers of enrolled students at the study institution. Moreover, MATH, ENGL, and CHEM are the subject areas with the highest student enrollment as shown in Table 1.

Table 1 High enrollment course sequences
Full size table

All analyses were conducted using the open-source statistical software R (R Core Team, 2022). Results were considered statistically significant if the p value was below the 0.05 significance level.

Results

Trends in the Selection of Flexible Grading Options by Semester

The results in the first line of Table 2 summarize the trends in students’ utilization of the flexible grading options by semester. Comparing Spring 2020 and Fall 2020, students chose the Pass (P1) option or the Withdrawal (WE) option at a similar rate, respectively, 17.4% and 15.6%, yet the rate declined significantly during the Spring of 2021 (8.8%) partially due to evolution of the policy, i.e., the stricter limit on the number of courses the flexible options can be applied. A closer look at the utilization rates of the P1 option versus the WE option reveals an interesting pattern where the rate of P1 declines from 14.4 to 11.3% while the rate of WE rises from 3.0 to 4.3% between Spring 2020 and Fall 2020. This pattern may be attributed to multiple factors that varied between the two semesters. First, at least half of the Spring 2020 semester, prior to the lockdown, was conducted using normal face-to-face instruction whereas the entire Fall 2020 semester used a hybrid of face-to-face and remote instruction model, a model that was unfamiliar to both students and instructors. Second, it is not unreasonable to assume that instructors may have been more lenient when assigning semester grades in Spring 2020 than in Fall 2020, especially with students on the pass/no-pass borderline (Karadag, 2021). Finally, the negative impacts of the pandemic on students and instructors are likely to have been more significant in the Fall 2020 semester, after five months full of harsh pandemic protocols, job losses, and death and illnesses that exhausted students and instructors alike, than in the Spring 2020 semester.

Table 2 Students’ selection of the flexible grading options by the course general education status and subject area
Full size table

Variations in the Selection of Flexible Grading Options: A Descriptive Analysis

Table 2 also reports the rates of selection of flexible grading options by the type of course and the course subject area. For general education courses, the flexible grading options (P1 or WE) were utilized 19.6%, 15.8%, and 9.8% of the time for the three semesters, Spring 2020, Fall 2020, and Spring 2021, respectively. Relative to general education courses, a lower rate of flexible grading selection was observed for non-general education courses in Spring 2020 (16.1%). The differences shrunk in the other two semesters. Additionally, the declining pattern of P1s and the rising pattern of WEs noted above between Spring 2020 to Fall 2020 were also present when considering the general education status of the course.

Focusing on the course subject area, MATH, ENGL, and CHEM had the largest enrollment of students in each semester of analysis. More than a quarter (28.5%) of the students taking a MATH course relied on the P1/WE options during the first semester of the lockdown (Spring 2020). As the pandemic progressed and the flexible grading policy was modified, students steadily invoked the policy less and less (23.4 % in Fall 2020 and 14.1 % in Spring 2021). We see a parallel trend in CHEM and ENGL courses, but the selection rate of the flexible grading options was somewhat lower for CHEM courses and much lower for ENGL courses relative to MATH courses. Overall, the results in Table 2 show significant variations in the utilization of the flexible grading options by the course subject area with MATH, CHEM, MEEN, ECON, and COMP witnessing higher rates of options utilization than other subject areas across the three semesters.

The variations in students’ utilization of the flexible grading options by students’ sociodemographic and academic profile characteristics are summarized in Table 3. Across the three semesters, male students used the flexible grading options at significantly higher rates than female students. On the other hand, considering the place of residence factors (rurality and residency), no noticeable differences were observed. Similarly, financial aid status and STEM major did not seem to be associated with notable variations in the utilization of the flexible grading options. In Spring 2020, an interesting trend emerged among students of varying classifications, i.e., freshman, sophomore, junior, and senior, with the flexible grading utilization rate declining as students’ classification gets higher. Specifically, the utilization rate was 21.2% for freshmen, 18.7% for sophomores, 16.6% for juniors, and 14.2% for seniors. Similar trends appeared in Fall 2020 and Spring 2021. Another notable pattern from Table 3 was that full-time students used the flexible grading options at a higher rate than part-time students (17.6% vs 12.6% in Spring 2020; 15.9% vs 10.8% in Fall 2020). However, this difference diminished by Spring 2021.

Table 3 Students’ selection of the flexible grading options by their sociodemographic characteristics and academic profile factors
Full size table

Factors Associated with Selection of Flexible Grading Options: A Multivariable Analysis

In this section, we discuss the factors associated with students’ selection of the flexible grading options. Unlike the univariate results described in the previous section, the results of this section are derived from multivariable analyses that account for the correlation between the covariates when exploring their associations with students’ selection of the flexible grading options. Therefore, the results presented in this section offer a more accurate depiction of the association between the selection of flexible grading options and the considered factors.

Table 4 summarizes the results of logistic regression models for the likelihood of selecting the flexible grading options. We modeled the general selection of flexible grading using the binary response variable “selected flexible grading”, which was coded as “1” if the student selected any flexible grading option P1/WE in any of their courses during the semester and as “0” otherwise. To provide a more detailed and nuanced understanding of the selection of flexible grading, we also modeled the selection of each flexible grading option separately using two additional response variables: “selected the P1 option”, which was coded as “1” if the student selected the P1 option in any of their courses during the semester and as “0” otherwise, and “selected the WE option”, which was coded similarly.

Table 4 Results of logistic regression models for the likelihood of selecting the flexible grading options by course characteristics and students’ sociodemographic characteristics and academic profile factors
Full size table

Course Characteristics

After controlling for the course’s subject area and the student’s characteristics, general education courses were less likely to apply the flexible grading options than core courses as seen from the odds ratios in columns 3, 6, and 9 of Table 4 (OR = 0.91, 0.78 and 0.84 in Spring 2020, Fall 2020 and Spring 2021, respectively). This result is statistically significant at the 5% level across semesters. Considering the type of option (P1/WE), the association between exercising flexible grading and the course’s general education status becomes insignificant in Spring 2020, remains significant in Fall 2020, and is partially significant (only for the WE option) in Spring 2021. We also note that in Fall 2020 and Spring 2021, the gap between general education and non-general education courses in selecting flexible grading is magnified when focusing on the WE option (OR = 0.74 and 0.75, respectively). With respect to subject area, across the three semesters, students were more likely to select the flexible grading options for MATH courses than for other subjects with few exceptions (ECEN in Spring 2020, and COMP and ECON in Fall 2020). For instance, students were twice as likely to employ flexible grading for MATH courses than for ENGL or HIST courses and three times more likely than for PSYC courses. For some subjects, the differentiating option varied by semester. A remarkable example appears in the case of COMP where in Fall 2020, students were 3.23 times more likely to select the WE option for COMP courses compared to MATH courses with no significant difference in selecting the P1 option. One semester later, the odds of selecting the P1 option for COMP courses were 29% less than for MATH courses with no significant difference in selecting the WE option.

Sociodemographic Factors

Considering students’ sociodemographics, the results in Table 4 show that male students were consistently more likely (64%, 40%, and 31% more in the three semesters, respectively) to utilize the flexible grading options than female students. Regardless of the flexible grading option chosen, the statistical significance of this association persisted, albeit with minor alterations observed in the option-specific odds ratios. Although rurality was not significantly associated with students’ selection of the WE option, students from rural residential areas were significantly less likely (12%, 16%, and 12% less) to utilize the P1 option than urban students. Only in Fall 2020, state residency (in-state vs out-of-state) showed a statistically significant association with the selection of the flexible grading options overall with out-of-state students being 6% less likely to select the flexible grading options. Considering the specific flexible option selected reveals an interesting observation where out-of-state students were always less likely to select the WE option (34%, 39%, and 24% less in Spring 2020, Fall 2020, and Spring 2021, respectively) than in-state students. While overall there was no significant association between the selection of flexible grading and the student’s financial aid status in Spring 2020, students on financial aid (Pell grant recipients) selected the WE option at a significantly higher rate (36% higher with p value < 0.0001) than their non-financial aid peers. On the contrary, in Fall 2020, we find a slightly, but statistically significant, higher rate of P1 option selection among students on financial aid (7% higher with p value < 0.05). Spring 2021 did not witness any significant association between financial aid status and the selection of flexible grading options.

Academic Profile Factors

Focusing on factors in the student’s academic profile, we found an interesting trend in the selection of flexible grading among students of different classifications. Across all three semesters, we noticed a negative association between students’ classification levels and the selection of flexible grading. This type of trend was very pronounced in Spring 2020 where freshman students were more likely to use the flexible grading options than sophomore students, junior students, and senior students (11%, 21%, and 36% more likely, respectively). It was also interesting to note that compared to freshman, sophomore, and junior students, senior students were much less likely to use the WE option in any of the three semesters. Considering the student’s full-time status, full-time students selected the P1 option at a significantly higher rate (38%, 68%, and 51% higher, respectively) than part-time students. Furthermore, in Spring 2021, full-time students were less likely to employ the WE option compared to part-timers (34% less odds with p value < 0.001). While it was noted from Table 3 that STEM students had a somewhat higher rate of flexible grading option selection than non-STEM students, the results in Table 4 suggest that STEM students were always less likely (12%, 23%, and 23% fewer odds in Spring 2020, Fall 2020, and Spring 2021, respectively) to exercise flexible grading than non-STEM students across all three semesters. Surely, the result of Table 4 is to be used as it accounts for the potential influence of other covariates, including the general education status and subject area of the course.

Frequency of Selecting the Flexible Grading Options

The results in Table 5 are derived from student-level negative binomial regression models for the frequency of selecting the flexible grading options (i.e., the number of courses with flexible grades employed per student) offset by the total number of courses the student attempted in the given semester. The distribution of the frequency of selecting the flexible grading options is depicted in Fig. 1. The negative binomial model was found to be a good fit for this count data; the \({\chi }^{2}\) goodness-of-fit p values were nearly equal to \(1.00\) in all models in all three semesters. We also tested the Poisson model and it was not deemed a good fit for the data. Unlike the course-level results in Table 4, the results in Table 5 are student-level, and hence, course-level factors (i.e., course general education status and subject area) are not present.

Table 5 Results of negative binomial regression models for the frequency of selecting the flexible grading options by students’ sociodemographic characteristics and academic profile factors
Full size table
Fig. 1
figure 1

Flexible grading options selection by semester

Full size image

According to the results in Table 5, male students used the flexible grading options at significantly higher rates (on average, 0.46, 0.35, and 0.27 more courses in Spring 2020, Fall 2020, and Spring 2021, respectively) than female students. A similar trend was present for each of the specific options (P1 and WE). Rurality was also significantly associated with the rate of selecting the flexible grading options. Precisely, across the three semesters, rural students exercised the P1 option less frequently than urban students, but there were no significant differences in the selection rates of the WE option. Furthermore, out-of-state residency was associated with significantly lower rates of exercising the WE option. On average, out-of-state students selected the WE option in 0.39, 0.49, and 0.28 fewer courses than in-state students in Spring 2020, Fall 2020, and Spring 2021, respectively. Only in Spring 2020—when students transitioned to emergency remote instruction at beginning of the pandemic– we observed an association between the student’s financial aid status and the rate of selecting flexible grading with financial aid students employing the WE option at a significantly higher rate (an average of 0.31 additional courses) than their non-financial aid peers.

Parallel to the results noted for the likelihood of option selection, the student’s classification was significantly associated with the rate of exercising the flexible grading options. Most notably, seniors consistently used the flexible grading options, particularly the WE option, at significantly lower rates than freshmen. Full-time status was also associated with differential rates of selecting the flexible grading options where full-time students exercised the P1 option more frequently (an average of 0.24, 0.45, and 0.36 additional courses) than part-timers in all three semesters. Additionally, in Spring 2021, full-time students selected the WE option at a significantly lower rate (an average of 0.44 fewer courses) than part-timers but no significant differences were observed in the other two semesters. Despite showing no significant association with the rate of selecting flexible grading in Spring 2020, the student’s STEM status was significantly associated with varying rates of option utilization in Fall 2020 and Spring 2021. Specifically, in these two semesters, STEM students employed the P1 option at modestly higher rates than non-STEM students. Note that this result does not align well with the results in Table 4 regarding the likelihood of option selection where in Fall 2020 STEM students had significantly lower odds (12% lower) to select the P1 option than non-STEM students. This discrepancy is likely due to the omission of the course subject area and general education status from the student-level models in Table 5. On the contrary, STEM students employed the WE option at significantly lower rates (an average of 0.17 and 0.32 fewer courses in Fall 2020 and Spring 2021, respectively) than non-STEM students.

Flexible Grading and Performance in Sequential Courses

In this section, we present and discuss the implications of the flexible grading policy in terms of students’ performance in sequential courses. We focus on popular course sequences in three subject areas, MATH, CHEM, and ENGL, at the study institution. Two MATH sequences are considered, the algebra sequence and the calculus sequence. The algebra sequence consists of MATH 103 and 104 which focus on algebraic and trigonometric concepts and are preparatory courses (i.e., pre-calc classes). They fulfill the “Mathematical, Logical, and Analytical Reasoning” general education requirement. However, all engineering and most science majors require the completion of the calculus sequence; MATH 131, MATH 132, and MATH 231 (Calculus I, II, and III). The Chemistry sequence consists of five courses. CHEM 103 is an introductory chemistry course. CHEM 106 and CHEM 107 are classic General Chemistry I and II courses. They are open to students who completed CHEM 103 or have an ACT or SAT score above a minimum threshold. Specific majors, such as Biology, Chemistry, Agricultural and Environmental Sciences, and other pre-professionals (pre-med, pre-vet, and pre-dental) will, additionally, take CHEM 221 and 222, the first and second semester courses in Organic Chemistry. ENGL 100 and ENGL 101, Ideas and Their Expressions I & II, respectively, are the only two general education courses that fulfill the “Written Communication” requirement. Consequently, every undergraduate student, apart from transfer students, must complete ENGL 100 and 101.

Flexible Grading in Mathematics Sequences

Figure 2 presents the associations between students’ grades (including the flexible grade options) in sequential MATH courses. One striking feature in Fig. 2 is the scarcity of two letter grades, C− and D’s. This trend derives from a graduation policy, namely many programs of study require a passing grade for core content courses, with passing being a “C” or better. Hence, students in MATH courses (wisely) selected the P1 option over a C− or D grade. With few exceptions, this occurred in every sequence.

Fig. 2
figure 2

Utilization of the flexible grading options in MATH course sequences: Spring 2020 to Fall 2020

Full size image

Additionally, four distinct clusters emerged over the calendar year of Spring 2020 to Fall 2020 (Fig. 2). Students comprising the green cluster earned A’s, B’s, and C’s in both courses; however, the correlation between the two grades is not strong. This cluster succeeded regardless of the format of the class, and regardless of the upheaval caused by the pandemic. They navigated the transition from face-to-face to remote learning and did not invoke the flexible grading options. A reasonable assumption is that the students completed each course in the sequence with a solid grasp of the content.

The second cluster, blue, succeeded during the outbreak of the pandemic and the transition to remote learning (received passing grades during the spring of the lockdown) but struggled during the fall when they were learning in the hybrid format. This trend might be attributed to two reasons: (1) the introduction of the hybrid learning environment negatively impacted students’ performance, and (2) some students expected the flexible grading options they were offered in Spring 2020 to be available in Fall 2020 and they decided to rely on them.

The third cluster, students within the red loop, invoked the flexible grading options during both the spring and the fall. They struggled before the lockdown and continued to struggle thereafter. Clearly, this cluster needed focused support, and it is most concerning to the administrators and instructors because they completed the semester without meeting the minimum standards for passing. They not only contribute to the college’s and the university’s rate of “D/F” grades and Withdraws (also known as the DFW rate) but they may also be placed on academic probation or dismissal.

Finally, the orange cluster represents the students who selected the flexible grading options in the spring of 2020 (for example, MATH 104) and then kept the earned letter grade in the subsequent course in the fall of 2020 (for example, MATH 131). Reasons for the emergence of this cluster may include; (1) the change in the flexible grading policy limiting the number of eligible courses to three in Fall 2020 as compared to an unlimited number of courses in Spring 2020, and (2) students in that cluster are likely to be science, engineering or math majors (taking the calculus sequence) and high-achieving students who were able to improve in the fall of 2020 as compared to spring 2020.

The span of time shown in Fig. 3 is a traditional “fall to spring” academic year. Those in the MATH 103 to 104 and the MATH 131 to 132 sequences sorted out into similar clusters as those in Fig. 2. On the other hand, unlike Fig. 2, there is clear data scarcity in the MATH 104 to 131 and the MATH 132 to 231 sequences due to the students being out of sequence. The normal sequence for students advancing to calculus after the algebra sequence would be to take MATH 103 in Fall, MATH 104 in Spring, and MATH 131 in Fall, whereas for students required to take the complete calculus sequence, they would normally take MATH 131 in Fall, MATH 132 in Spring and MATH 231 in Fall. An intriguing cluster in Fig. 3 is the blue cluster where a sizable number of students appear to have utilized the P1 option to progress in completing their algebra or calculus sequence. The orange cluster in Fig. 3 represents the ideal scenario for the P1 option utilization where students opting for the option in their first course did well enough in their second course to a level where they can rely on their earned grades. Finally, similar to the pattern observed in Spring 2020 to Fall 2020, we observe a cluster of students (red cluster) who were not well-served by utilizing the P1 option in their first course. We believe that it would have been to the advantage of these students to retake the first course to build proper foundations for the second course instead of using the P1 option to advance to the second course while underprepared.

Fig. 3
figure 3

Utilization of the flexible grading options in MATH course sequences: Fall 2020 to Spring 2021

Full size image

Flexible Grading in Chemistry Sequences

Chemistry is one of many general education options, along with physics, biology, astronomy, and other science courses, that can be used to fulfill the scientific reasoning requirement. Students can enter the CHEM sequence either at CHEM 103 or CHEM 106 and may conclude their studies of chemistry after CHEM 107, CHEM 221, or CHEM 222.

The grade pattern in Fig. 4 resembles that of the MATH sequences. However, it is noteworthy that the cluster of “green” students is strongly shifted to the A and B grades. Few students earned a C (or C−/C+) in their first course and none of those that earned a C grade during the Spring of 2020 attained a better grade in their second course in the Fall of 2020. Notably, those that selected the P1 or WE options for Spring 2020, split into two clusters (red and orange clusters) during the Fall of 2020. One cluster (red) used the P1 option again, failed, or passed with a D grade while the other (orange) did very well, earning mostly A and B grades in Fall 2020. Apart from 3 or 4 students, the absence of students in the C+ to D range is interesting. The last cluster (blue) in Fig. 4 represents those who did well in their first course of the sequence in Spring 2020 but struggled in the second course in Fall 2020. This cluster is most dense for the CHEM 107 to CHEM 221 sequence where we see three subclusters emerging; 1) those who managed to pass CHEM 221 with the help of the P1 option, 2) those who withdrew from CHEM 221 by the mandated withdrawal date, and thus received a WE, and 3) those who failed CHEM 221. This third subcluster is likely to be composed of those who thought they can finish the course with a grade that qualifies for the P1 option (i.e., D or better) but they could not.

Fig. 4
figure 4

Utilization of the flexible grading options in CHEM course sequences: Spring 2020 to Fall 2020

Full size image

Figure 5 depicts the utilization of the flexible grading options in CHEM course sequences between Fall 2020 to Spring 2021. It mostly repeats the distribution observed in Fig. 4, with some notable differences. One such difference is the lower P1 rate which might be attributed to the more stringent limit on the number of P1’s for Spring 2021 (limit of 1) as compared to Fall 2020 (limit of 3). As with many sequentially offered courses, CHEM 106 is an introductory college-level general chemistry course, with the expectation that successful students take CHEM 107 in the spring. Most students follow this pattern. The majority of students who were in the CHEM 106 to 107 sequence fall in the green cluster (i.e., earned a C or better grade in both courses), where the relationship between their grades in the two courses was mostly linear. It is notable that most students who opted for P1 in their first course in Fall 2020 earned respectable passing grades in the second course in Spring 2021 (orange cluster). This is the one group of students for whom the P1 option was “most beneficial”.

Fig. 5
figure 5

Utilization of the flexible grading options in CHEM course sequences: Fall 2020 to Spring 2021

Full size image

Flexible Grading in the English Sequence

Figure 6 provides insight into the ENGL 100 to ENGL 101 sequence over the course of three semesters. As previously noted, all students at the study institution must pass ENGL 100 and ENGL 101 to graduate.

Fig. 6
figure 6

Utilization of the flexible grading options in the ENGL course sequence. * It is worth noting that while passing ENGL 100 with a minimum “D” grade is a prerequisite for progressing to ENGL 101, we see a sizable fraction of students who did not pass ENGL 100 in Fall 2020 and yet enrolled in ENGL 101 in Spring 2021. We speculate that this may be a result of system error where the system failed to enforce the prerequisite requirement.

Full size image

First, we note a high sparsity in enrollment in the ENGL sequence in the Spring 2020 to Fall 2020 cycle compared to the Fall 2020 to Spring 2021 cycle due to ENGL 100 being a first-semester course for first-year students. Remarkably, a similar pattern to that of the MATH sequences emerges for the ENGL sequence in the Spring 2020 to Fall 2020 cycle. Students with strong grades, i.e., A and B grades, in ENGL 100 were split into two clusters in ENGL 101: the green cluster with strong grades in ENGL 101 and the relatively smaller blue cluster with either P1 or non-passing grades in ENGL 101. The cluster encircled with the red box represents students who either opted for the P1 or WE or failed ENGL 100 and then failed ENGL 101. For the Fall 2020 to Spring 2021 cycle, no relationship emerged between performance in ENGL 100 and ENGL 101. Nearly every permutation that can exist for grades within this sequence does exist (e.g., A grade to F and F to A). Specifically, selecting the P1 option in the fall does not seem to have impacted grades in the spring.

Discussion

In this study, we examined the flexible grading policy induced by the COVID-19 pandemic at a medium-sized public university in the United States. Specifically, the study documented the evolution of the policy over three semesters at the study institution, the variations in the utilization of the flexible grading options by course characteristics, students’ sociodemographics, and students’ academic profiles, and the longitudinal impacts of the flexible grading policy on various course sequences. We used a combination of descriptive statistics (e.g., cross-tabulations and visualizations) and multivariable regression models to illustrate the variations and trends in the utilization of the flexible grading options.

Major Patterns in Policy Utilization

In general, we found that students selected the flexible grading options at a similar rate in Spring 2020 and Fall 2020. The options selection rate declined significantly during the Spring of 2021 as the policy evolved with greater restrictions. The utilization of flexible grading options varied significantly between subject areas with MATH, CHEM, MEEN, ECON, and COMP witnessing higher rates of flexible grade option utilization than other subject areas across the three semesters. Additionally, controlling for the course’s subject area and the student’s sociodemographics and academic profiles, we found that students were less likely to apply the flexible grading options, particularly the WE (withdrawal) option, in general education courses than in core courses. These findings suggest that the flexible grading policy has supported students as they navigated through certain areas of their curriculum during the pandemic.

We also noted considerable variations in students’ utilization of the flexible grading options by students’ sociodemographics. Across all three semesters and regardless of the flexible grading option chosen, male students used the flexible grading options at significantly higher rates than females. This pattern is consistent with the well-documented observation that male college students have lower achievement than female students, especially among minorities (e.g., Tani & Ray, 2018; NCES, 2021). Considering the place of residence, although rural students faced greater challenges from transitioning to remote instruction during the pandemic (Vogels, 2021), they were less likely to use the flexible grading options, particularly the P1 (pass) option, than urban students. We speculate that rural college students, who are typically high-achievers (e.g., Provasnik et al., 2007), tended to eschew the flexible grading options and instead seek traditional letter grades. However, we acknowledge that rural and non-rural communities are multi-faceted, particularly those from the institution’s home state who are ethnically and culturally diverse (Showalter et al., 2019), which makes providing a deeper explanation beyond the scope of this paper. Even though in-state and out-of-state students exhibited a comparable selection of flexible grading options overall, out-of-state students consistently showed a much lower likelihood of opting for the WE option. This can be plausibly explained by the considerably higher expenses associated with course retakes for out-of-state students, relative to their in-state peers.

Students’ economic status, as determined by whether they received a Pell grant during the semester, was significantly associated with differential odds and rates of selecting the WE option in Spring 2020. Specifically, students who were on financial aid were significantly more likely (36% more) to opt for the WE option than students who were not on financial aid. Additionally, accounting for the number of courses the student was taking in Spring 2020, financial aid students opted for the WE option more frequently than their non-financial aid peers. In Fall 2020, financial aid students were marginally (7%) more likely to employ the P1 option than their non-financial aid peers. This is in line with the results of Rodríguez-Planas (2022a, b) who found that, in Spring 2020 at Queens College, lower-income students (defined as those who ever received a Pell grant) from the bottom GPA quartile were 8.1% (p value < 0.05) more likely to utilize the flexible grading options than their higher-income counterparts. We note one important distinction between Rodríguez-Planas’s analysis and our analysis. Rodríguez-Planas’s analysis controlled for pre-pandemic academic performance (i.e., Fall 2019 GPA quartiles) and considered the interaction between economic status and pre-pandemic performance, whereas our analysis considered students’ pre-college preparation metrics (i.e., standardized ACT/SAT) and other academic profile factors (e.g., classification, STEM status) but not their pre-pandemic performance since our study population includes a large portion of new freshman students for whom the pre-pandemic college GPA data was not available.

Several factors in the student’s academic profile were also found to be associated with differential odds and/or rates of selecting the flexible grading options. Freshman students were more likely to select the flexible grading options, in particular the WE option, than all other students in Spring 2020, juniors and seniors in Fall 2020, and only seniors in Spring 2021. This is not surprising as first-year students lack the wide range of skills and experiences needed to succeed in college. They were simultaneously learning to be college students as they reacted to the changes brought on by the pandemic, and hence, they needed the flexible grading options to manage these challenges. Furthermore, full-time students were significantly more likely to exercise the P1 option than part-timers across the three semesters. This association was substantiated in the models accounting for the number of courses enrolled where full-time students were found to exercise the P1 option at significantly higher rates than part-time students. Lastly, STEM students were less likely to utilize the flexible grading options overall than non-STEM students–the gap was particularly pronounced under the WE option. The reduced odds of using the P1 option among STEM students could possibly be due to the fact that more STEM, than non-STEM, students aspire to pursue graduate studies, where excessive P1’s on their transcripts could prove disadvantageous. On the other hand, the differential utilization of the WE option is harder to explain without further information.

Longitudinal Impacts of the Policy

Considering the longitudinal impact of the flexible grading policy, we investigated the correlation between grades in sequential courses in three popular course sequences at the study institution, namely, mathematics, chemistry, and English sequences. In general, four clusters of students emerged when looking at the correlation between grades in a pair of sequential courses taken in two consecutive semesters. These four clusters were (1) students who opted for the earned letter grades (C or better) in both courses; (2) students who opted for earned letter grades in the first course but either utilized the P1 option or earned a non-passing grade in the second course; (3) students who opted for the P1 option in the first course and maintained an earned letter grade of C or better in the second course; and (4) students who opted for the P1 option in the first course and either utilized the P1 option or earned a non-passing grade in the second course. Students in cluster 4 are the most problematic and should have received special attention from advisors and school administrators as they went through these course sequences. The analysis reported in this paper did not thoroughly examine the characteristics of those students. Further data and analysis are needed to address this important point. This point will be sought further in our future research.

Significance

The flexible grading policy was a generous and empathetic overture by higher education institutions to help students overcome the challenges of the COVID-19 pandemic. Although a plethora of studies examined the effects of the pandemic on college education, few explored the implications of the flexible grading policy. To address this gap, the current study presents a detailed analysis of the COVID-19 flexible grading policy at a public university in the United States. By investigating how the use of flexible grading options varied among students with different sociodemographic backgrounds and academic profiles, as well as across courses and its impact on subsequent courses, the study provides insights into how this policy may have supported student success and promoted academic equity in a time of turmoil.

Implications

The study has several implications for higher education policy and practice. Our study uncovered significant variations in the utilization of flexible grading options by course and student characteristics. These findings underscore the need for more individualized and proactive counseling efforts to help students make informed decisions about whether to exercise these options. Academic advisors, for example, could play a crucial role in providing personalized guidance to students on the potential benefits and drawbacks of the flexible grading options, and in helping them weigh the trade-offs between short-term gains and long-term consequences. These efforts could help all students make informed choices, hence, meeting the needs of diverse student populations and promoting equity and fairness. Moreover, our study revealed a complex picture of the effects of the flexible grading policy on student success in sequential courses. While some students benefited from the policy and were able to complete their course sequences in a timely manner, others were put at a disadvantage. Specifically, the pass option, which allowed students to progress in their course sequences without earning a letter grade, proved detrimental to some students who were unprepared for subsequent courses. These findings highlight the need for careful monitoring and tailoring of such policies to ensure that all students are adequately prepared for future coursework.

The current study motivates further research on the flexible grading policy. One future direction may focus on investigating the psychological aspects of the policy. For instance, it is imperative to assess whether the policy has provided students with a sense of assurance or a safety net, hence, giving them a confidence boost and improving their psychological well-being. Additionally, an in-depth analysis of the financial impacts of the flexible grading policy warrants further investigation. Such analysis may further develop our findings, which showed that out-of-state students avoided the no-credit option, possibly to skip the high cost of repeating the course, and the findings of Rodríguez-Planas (2022a, b), which suggested that lower-income students were more reliant on the flexible grading options, possibly to maintain their merit-based financial aid benefits.

Limitations

This study is not without limitations. First, the study relies solely on data from students’ academic records. The study could benefit from the inclusion of data obtained from a students’ survey or focus groups, such as data about students’ awareness of the flexible grading policy and its ramifications, to develop a more comprehensive depiction of students’ decisions to exercise the flexible grading options. Second, the scope of the study was confined to students enrolled in a single institution, particularly an HBCU, which could potentially restrict the generalizability of the study to other contexts or populations. Given the differential configuration, implementation, and evolution of the flexible grading policy at different institutions, the results of our study may not be directly transferable to other institutions, including HBCUs despite the similarity in socioeconomic and racial diversity. However, our study has the potential to motivate other studies evaluating the impacts of the policy at other institutions which shall enhance our understanding of the implications of the policy.

Conclusion

The present study provided a detailed analysis of the COVID-19-induced flexible grading policy at a large historically black college and university in the USA. Despite the limitations noted above, our study has the potential to add to the knowledge base in many ways.

Our study demonstrated that the utilization of the flexible grading policy options varied significantly by course characteristics and students’ sociodemographic characteristics and academic profiles. The study also showed that the flexible grading policy had mixed effects on student success in sequential courses. These findings suggest that the implementation of the flexible grading policy could benefit from careful monitoring, tailoring, and counseling to ensure that all students can take full advantage of the policy options, thereby contributing to greater student success and increased equity.

Overall, we believe that at the study institution, the policy provided a much-needed safety net for students while allowing them to continue progressing in their education. In light of the unprecedented challenges presented by the COVID-19 pandemic, the policy had a decidedly positive impact by helping to mitigate the disruptive effects of the pandemic on students' academic trajectories. Particularly noteworthy was the policy’s ability to safeguard and support vulnerable student populations, such as freshmen.