A Study of Students’ Readiness to Learn Calculus

Abstract

The Calculus Concept Readiness (CCR) instrument assesses foundational understandings and reasoning abilities that have been documented to be essential for learning calculus. The CCR Taxonomy describes the understandings and reasoning abilities assessed by CCR. The CCR is a 25-item multiple-choice instrument that can be used as a placement test for entry into calculus and to assess the effectiveness of precalculus level instruction. Results from administering the CCR to first semester calculus students at the beginning of the semester revealed severe weaknesses in students’ foundational knowledge and reasoning abilities for learning calculus. Correlating CCR results with course grades revealed that students with higher CCR scores are better prepared to succeed in beginning calculus. The CCR data further identified specific ways of thinking and concepts for which precalculus instruction could be improved to influence student learning and preparation for calculus.

Introduction

Precalculus courses in the United States (US) are not achieving their educational potential, especially with regard to preparing students to succeed in calculus (e.g., Breidenbach et al. 1992; Carlson 1998; Moore 2012; Moore and Carlson 2012). One consequence of this is high attrition from precalculus to calculus, but the major consequence is the lost learning opportunities that would benefit precalculus and calculus students. The attrition and missed educational opportunities are enormously expensive because they delay STEM students’ progress toward degrees, lower the learning in some degrees, or cause students to drop out of STEM altogether. Research over the past few decades point to ways precalculus and calculus courses can be strengthened to address this alarming situation (e.g., Carlson and Rasmussen 2008). This research has informed the development of the Calculus Concept Readiness (CCR) instrument,¹ while curriculum and instruction in precalculus level courses have not been noticeably impacted by research on learning key ideas of precalculus level mathematics.

This article provides an overview of literature that has identified foundational precalculus level reasoning abilities and understandings that students need for understanding key ideas of calculus. The Calculus Concept Readiness (CCR) Taxonomy is presented to detail the specific abilities assessed by the CCR instrument. We conclude by sharing results from administering the CCR to US college students prior to taking beginning calculus.²

Background

Over the past 25 years many mathematics education researchers have found that student difficulties in understanding key ideas of calculus are rooted in their weak understanding of the function concept (e.g., Breidenbach et al. 1992; Carlson 1998; Carlson et al. 2010; Tall and Vinner 1981; Tall 1992, 1996; Thompson 1994a; Smith 2008; Zandieh 2000). Early studies of students’ understandings of the function concept revealed common misconceptions among students (e.g., Monk 1992; Sierpinska 1992; Vinner and Dreyfus 1989) including their: i) strong tendency to view a graph as a picture of an event (Monk 1992), rather than a representation of how two quantities change together; and ii) viewing a function as a recipe for getting an answer instead of as a process that maps input values to output values (Breidenbach et al. 1992; Carlson 1998). These weaknesses in students’ understanding of the function concept are contributing to students being unprepared to understand ideas in beginning calculus.

Other studies have investigated student thinking in the context of curriculum tasks designed to develop student understanding of ideas of function (Dubinsky and Harel 1992; Moore and Carlson 2012), function composition and inverse (Engelke et al. 2005; Engelke 2007), quantity (Moore and Carlson 2012; Moore 2012, 2014), exponential growth (Castillo-Garsow 2010; Strom 2008), and central ideas of trigonometry (Moore 2013, 2014). These studies consistently report that when students conceptualize a function as a process that maps input values from a function’s domain to output values in a function’s range, they are able to understand and use the idea of function, function composition and function inverse to solve novel problems (Engelke 2007; Moore and Carlson 2012). As students begin to reason about how the input values and output values of a function change together they are able to distinguish between different function types, explain the meaning of a concave up graph and conceptualize the covarying quantities related by trigonometric functions in precalculus (Moore 2012, 2014) and the Fundamental Theorem of Calculus (e.g., Carlson et al. 2003; Oehrtman 2008; Smith 2008; Thompson 1994a). This is encouraging since it suggests that student learning in precalculus level mathematics can be affected by instructional interventions that support students in understanding and reasoning with the function concept.

Foundational Reasoning Abilities and Understandings for Learning Calculus

This section provides a more detailed description of the reasoning abilities and understandings that students need to develop prior to beginning a course in calculus. In particular we describe what is involved in conceptualizing quantities and make an argument for the importance of covariational reasoning in defining meaningful functions to model relationships in word problems. We discuss the importance of developing a process view of function and what it means. We also describe complexities of engaging in proportional reasoning and conclude by discussing understandings of other key ideas (e.g., constant rate of change, average rate of change) that are needed for learning calculus.

Covariational Reasoning

In the context of mathematics it is common that students need to conceptualize quantities in a problem situation and to consider how those quantities are related and change together (Carlson et al. 2002; Thompson 1994b). This ability to both conceptualize a situation and imagine the measurable attributes of the objects (quantities) in a situation is referred to as quantitative reasoning (Smith and Thompson 2007; Thompson 1993, 2011, 2012). For example, when watching a race one might initially observe runners and a starting line, and then when the starting gun is fired be interested in how a quantity such as distance of a runner from the starting line changes as the runner is moving down the track. One might also notice that the elapsed time since the runner started the race is increasing and that the length of the race is 100 m. The observer has conceptualized two varying quantities, the elapsed time since the start of the race and the distance of the runner from the starting line, and one fixed quantity, the length of the race.³ There are many other varying and fixed quantities that could be conceptualized in this situation (e.g., the height of the runner or the distance of the runner from the finish line). However, it is important that students learn to focus on and conceptualize the quantities that are relevant for the line of inquiry they are pursuing.⁴

The mental process of relating two varying quantities requires that students think about how the two quantities are changing together. Gaining clarity about how the runner’s distance from the starting line is changing with the elapsed time since the runner started running might involve considering fixed amounts of change in one quantity while considering how much the other quantity is changing. In this example we might consider fixed amounts of time (e.g., ½ sec) while considering how much distance the runner traverses for each successive ½ sec of the race. If we were to observe that the runner was traveling a greater distance, over some interval of time for each successive ½ sec since starting the race, we might conclude that the runner’s rate of change of distance with respect to time is increasing over that interval of time.

Carlson (1998) documented that both precalculus and second semester calculus students had difficulty creating a graph to represent the height of water in a spherical bottle as a function of the amount of water in the bottle. Carlson et al. (2002) later described 5 mental actions associated with covariational reasoning in the context of making or interpreting a graph of two quantities that change together. These mental actions include: i) conceptualizing the quantities in the situation that are to be related; ii) imagining how the direction of the two quantities change together (e.g., as the elapsed time since the race started increases the distance of the runner from the finish line decreases) iii) imagining how the amount of change of one quantity changes while considering contiguous fixed amounts of the other quantity on intervals of that quantity; and iv) imagining how the average rate of change of the output variable with respect to the input variable is changing on small contiguous intervals of the input variable.

A student who considers how two quantities in a dynamic situation change together is said to be engaging in covariational reasoning (Carlson et al. 2002; Thompson 1994b). This is a foundational way of thinking that is needed to construct meaningful formulas and graphs to model relationships in applied contexts (Carlson et al. 2002; Moore and Carlson 2012). Covariational reasoning has also been documented to be an essential reasoning ability for understanding and using ideas in beginning calculus (Carlson et al. 2003; Engelke 2007; Smith 2008; Thompson 1994b; Zandieh 2000).

When students first encounter word problems in precalculus, regardless of the type of function model that is needed, they must first conceptualize the quantities to be related (e.g., length of the radius of an expanding sphere and volume of the expanding sphere, angle measure and vertical distance, ambient temperature and the temperature of an object, amount of time since making an investment, the value of the investment). Once the relevant quantities in a word problem have been conceptualized students are able to think about how the quantities are related and how they change together. These conceptualizations describe the reasoning that students must engage in to construct a formula to model the relationship between two quantities in an applied context (Moore and Carlson 2012).

Understanding the Function Concept

Carlson and colleagues (Carlson 1998; Carlson et al. 2002) found that many precalculus level students have difficulty using and interpreting function notation, and many are not clear on what it means to express one quantity as a function of another. Precalculus students sometimes confuse the output of a function with the function name and interpret the equal sign as a statement of equivalence rather than as a means of defining a relationship between two quantities that change together (e.g., Carlson 1997, 1998). Students who exhibit a process view of function are better prepared to understand and use function notation to relate values from a function’s domain to its range. This is because they see both a function’s defining formula and graph as specifying how to process the input values to produce output values. This image supports students in thinking about function composition as the stringing together of two function processes for the purpose of relating two quantities that cannot be directly related by a single formula. Students who have a process view of function are also able to understand the idea of function inverse as a new mapping that reverses the process of the original function.

Carlson (1995) investigated precalculus level students’ understanding of function. She found that when function names such as f, g, and h were used, students were unclear as to what the letters meant and that some students believed that the letters represented variables. This is not surprising because up to this point in students’ experiences in algebra, letters had been used to define variables. This conception does not support students in viewing a function that has been named with a letter and defined with a formula as representing a process that maps input values in a function’s domain to output values in a function’s range.

Extending the process view of function to reason about how the values in a function’s domain covary with values in a function’s range involves considering how function values change over a continuum of values, rather than just imagining one input value being mapped to one output value one value at a time. This extends the process view of a function so that students are able to both construct meaningful graphs, and to describe what a graph’s concavity conveys about how the function’s output value is changing relative to its input value. Several studies (e.g., Carlson 1998; Carlson et al. 2002; Carlson et al. 2010; Moore and Carlson 2012) revealed that attention to the quantities being modeled in a situation, and the ability to think about how one quantity is changing while imagining fixed incremental changes of the other quantity, enables students to understand and use ideas of constant rate of change and to interpret and construct meaningful function formulas and graphs to represent specific linear, exponential, quadratic, rational, and periodic growth using functions.

The idea of average rate of change on an interval of a function’s domain is also a key idea of precalculus that is needed for learning calculus. It is common for precalculus level students to associate the word average with a procedure to add numbers in a list and then divide by the number of items added (Carlson et al. 2010). In contrast, one accurate and meaningful understanding of the idea of average rate of change of a function on an interval, is that the average rate of change is the constant rate that would achieve the same change in both the input and output quantities as the actual function on the interval of interest (Thompson 1994b). Studies (e.g., Carlson et al. 2002; Carlson et al. 2010) have consistently revealed that the vast majority of students at many different universities across the US exit precalculus courses without understanding the idea of average rate of change.

Proportional Relationships

The ability to recognize situations in which two varying quantities are related proportionally has been documented to be problematic for precalculus level students (Castillo-Garsow 2010). They might be able to solve for x when given a situation in which x/y = 3/7 and, but if given a problem that requires students to recognize that the ratio of two varying quantities is a constant, or that two varying quantities are related by a constant multiple, the vast majority of precalculus level students have difficulty recognizing proportional relationships in dynamic situations (Carlson et al. 2010). For example, when administering the rain-gauge problem (Fig. 1) to 1205 precalculus students at the end of the semester, only 43 % of these students provided the correct response of $7\frac{1}{3}$ (Carlson et al. 2010). Most of these students recognized that when the wide cylinder had 6 inches, the narrow cylinder had 4 inches. However, they failed to recognize that as the amount of rain increased, the ratio of the number of inches of rain in the wide cylinder to the number of inches of rain in the narrow cylinder had to remain in a constant ratio of 4:6. It was common for students who provided an incorrect response to reason that, if the water rises to the 11th mark on the narrow cylinder it would rise to the 9th mark on the wide cylinder since the amount of water originally in the wide cylinder was 2 less than what was in the narrow cylinder.

Fig. 1

The rain-gauge problem (Piaget et al. 1977; Lawson 1978)

To the right are drawings of a wide and a narrow cylinder. The cylinders have equally spaced marks on them. Water is poured into the wide cylinder up to the 4th mark (see A). This water rises to the 6th mark when poured into the narrow cylinder (see B). Both cylinders are emptied, and water is poured into the narrow cylinder up to the 7th mark. How high would this water rise if it were poured into the empty wide cylinder?

Recent work to improve student learning in precalculus (Carlson et al. 2015) has highlighted the important role that proportional reasoning plays in understanding and using the idea of constant rate of change. If two quantities are changing at a constant rate of change, the changes in the two quantities are proportional. It is this understanding that is needed to determine a new value for one quantity when the constant rate of change and a value of the other quantity is known. Many applied problems in precalculus and beginning calculus require use of proportional reasoning. Recognizing proportionality of quantities and using proportional reasoning is also key to understanding and using the idea of angle measure in trigonometry (Moore 2013).

Angle Measure and Sine Function

The ideas of angle measure and trigonometric functions have been documented to be under-developed in inservice teachers (Thompson 2008; Thompson et al. 2007) and precalculus level students (Moore 2012, 2013) and preservice teachers (Moore 2012; Moore et al. 2012). Students often do not conceptualize an angle measure as an amount of openness between two rays with a common endpoint; nor do they recognize the need to use an arc of a circle (with its center at the angle’s vertex) to measure an angle’s openness.

According to Moore (2013), approaches that help students view an angle’s measure in terms of the relative length of circle’s arc that the angle subtends, are able to understand the idea of radian measure and effectively use the radius of a circle as a unit for measuring angles. Examination of student thinking during a teaching experiment designed to support student understanding of trigonometric functions revealed that when students are able to reason about how an angle’s measure and the vertical coordinate of the arc’s terminus (measured in units of the arc’s radius) covary, they are better able to understand the sine function and use it meaningfully to model periodic motion (Moore 2012). This image of the sine function in the unit circle context also was useful for students in connecting their unit circle conceptions of the sine and cosine functions to their conceptions of these functions in the triangle trigonometry context. These students came to understand specific right triangles as corresponding to specific input–output pairs of the trigonometric functions.

The CCR Taxonomy

The CCR Taxonomy (Table 1) includes three primary reasoning abilities that are foundational for learning and using key ideas of calculus. The taxonomy includes understandings of various function types that emerge from examining growth patterns in data, and other understandings that have been identified in research studies to be essential for either constructing or interpreting meaningful function formulas and graphs. Lastly, the taxonomy has a category that describes the trigonometric ideas that are needed to model periodic growth and to understand and connect unit circle and triangle trigonometry.

Table 1 The CCR taxonomy

The CCR is a 25 item multiple-choice exam with each question having five answer choices. Eighteen of the twenty-five CCR items assess or rely on student understanding of the function concept. Five items assess student understanding or use of trigonometric functions, and four items assess student understanding or ability to use exponential functions. Ten items are situated in an applied (or word problem) context and require students to reason about quantities and use ideas of function, function composition, or function inverse to represent how the quantities change together. There are other items that provide information about such things as students’ understanding of notational issues, their ability to interpret the meaning of an absolute value inequality such as |x − 3| < 5, and whether they can determine the inverse function of an exponential function. In some problems students are expected to recognize equivalent expressions, perform translations on a known function, and/or use structural equivalence in their reasoning, such as recognizing that 3(x+2)²–4(x+2)+7 = 0 is a quadratic function in (x+2).

The Process of Developing the CCR

The literature review and CCR taxonomy informed the development of items produced by a committee of five mathematicians and one mathematics educator. Four of the mathematicians have participated in developing and scoring items for Advanced Placement Calculus, one has worked in placement testing for over 20 years, and the mathematics educator had led the development of the Precalculus Concept Assessment (PCA) instrument (Carlson et al. 2010).

Following methods of instrument development used to develop the Precalculus Concept Assessment (PCA) (Carlson et al. 2010), the Mechanics Baseline Test (Hestenes and Wells 1992) and the Force Concept Inventory (Hestenes et al. 1992), open-ended questions were designed to assess the reasoning abilities and understandings that had been revealed to be most critical for learning calculus. According to Lissitz and Samuelsen (2007), the development of a valid examination should always begin by identifying the constructs worthy of assessment. Question wording and item distractors for the multiple-choice items were based on student interview data that illustrated common student thinking when responding to each open-ended item. Further interviews were conducted with students after the initial multiple-choice items were developed to identify or refine item distractors so that they were representative of five common student responses, and to verify that questions and answer choices were interpreted as we intended. The taxonomy and CCR items went through multiple cycles of refinement. Clinical interviews with students were conducted repeatedly until each CCR item had been validated to: i) be consistently interpreted, ii) assess the knowledge intended by the item designer, and iii) have distractors that were representative of student thinking as revealed during the interviews.

After validating that the CCR items assess what we claimed, we administered the CCR exam to 631 students at three public universities and one private university. The data was subsequently analyzed to examine trends in the CCR data.

Analyses of CCR Results

We administered CCR to 601 Calculus 1 students at three different universities during the first week of the fall 2009 semester. Our primary goal was to determine how well CCR scores predicted grades in Calculus 1 courses. We administered CCR to thirty precalculus students in the last week of classes, comparing student course grades with their CCR scores. We used these quantitative data to: (1) estimate the reliability of the test as a whole; (2) measure how each of the 25 items was functioning, individually and as a part of the test; and (3) measure validity of the instrument as a measure of readiness for success in learning calculus.

Reliability is a measure of consistency of scores on repeated administrations of the test. Since repeated administrations pose logistical difficulties, reliability estimates from a single administration of the test are made from correlations of scores on subsets of the test. For example, split-half reliability is a correlation of scores on halves of the test. The estimate of reliability we used is an extension of the split-half reliability, called Cronbach’s alpha. To measure item functioning, we computed a difficulty index (the percent correct), a discrimination index, and the point-biserial coefficient for each item. The point-biserial coefficient of an item is the correlation between the scores on the item and the scores on the entire test with the item deleted.

We measured validity of CCR as a predictor of course grades in Calculus 1 (predictive validity), correlated CCR scores with the American College Testing (ACT)⁵ mathematics scores (concurrent validity), and compared grades in a prerequisite precalculus course to CCR scores (criterion validity). Each of these analyses indicated that CCR scores are useful when deciding on readiness for success in the study of calculus.

Establishing CCR Validity⁶ and Predictive Potential

During the fall semester of 2009, CCR was administered to 215⁷ Calculus 1 students at a large public university during the first week of class. Approximately 70 % of these students were freshmen (149 freshmen and 66 upperclassmen) and were placed in the course by virtue of having an ACT mathematics test score of at least 26. Most of the 66 upperclassmen were in the class by virtue of passing a prerequisite course. Final course grades were obtained for 214 of these 215 students. The statistics for these students are given in Table 2.

Table 2 Basic statistics on CCR and ACT scores and course grades

Table 3 provides the Pearson correlation coefficients between ACT mathematics scores (most recent and maximum in 2007–2009), the score on CCR, and the course grade (0–4 with withdrawal (W) being 0). These are all significant at the 0.0001 level under the null hypothesis that the correlations are 0. The number of records for each correlation is given as N.

Table 3 Pearson correlation coefficients

The above results (Table 3) indicate that course grades are moderately correlated to CCR scores and ACT, while the correlation of grades with ACT scores is somewhat weaker. This raised the question of using both ACT scores and CCR scores to explain variation in course grades. The results of two multiple regressions are given in Tables 4 and 5.

Table 4 CCR score and ACT recent as explanatory variables

Table 5 CCR score and ACT max 07–09 as explanatory variables

These results indicate that using an ACT score (most recent or maximum in 2007–2009) in conjunction with the CCR score does not add much to the explanation of the variation of course grades. There is considerable overlap in the predictive information from CCR scores and ACT scores, but the predictive power of CCR is better, with CCR scores contributing significantly to the explanation of the variation of course grades.

We provide a brief description of the 25 items, the percent correct (P), a measure of discrimination (D), and the point-biserial coefficient (PBS) (see Appendix A). The percent correct is a measure of item difficulty. The Discrimination Index (D) (Kelly et al. 2002) is computed from equal-sized (27 %) high and low scoring groups on the test by subtracting the number of successes by the low group on the item from the number of successes by the high group, and dividing this difference by the size of a group. The range of this index is +1 to −1, and values of 0.4 and above are regarded as high and less than 0.2 as low discrimination. We observed that 12 of the items have high discrimination indexes and four (20, 21, 23, 24) have low indexes. Not surprising these four items with low discrimination indexes correspond to the ones on which our interviews revealed severe weakness in most students. Three of these four are trigonometry items, corroborating our findings from the clinical interviews that even high performing students had weak understanding of ideas of angle measure and trigonometric functions in the unit circle context.

The Pearson point-biserial for each test item is a correlation of the scores on that item (dichotomous at 0 or 1) and the scores on the test with that item deleted. A point-biserial of at least 0.15 is recommended but “good” items have point-biserials greater than 0.25 (Varma 2012). It is noted that all of the items have biserial coefficients greater than 0.15 and nineteen of them have coefficients greater than 0.25 and moderate to high discrimination indexes. The six items 18, 19, 20, 21, 23 and 24 with a point-biserial lower than 0.25 are items on which the percentages of correct answers were less than 20 %, similar to what one would expect if students were selecting an answer without even reading the question.

Reliability Measure

The Cronbach coefficient, alpha, is an estimate of test reliability, i.e., the internal consistency of the test. The raw Cronbach coefficient from this test administration is 0.665 (p < 0.0001); standardized it is 0.658. The raw Cronback alpha of 0.665 increases slightly with items 18, 19, 20, 21, 23, and 24 deleted (one at a time), consistent with the low correlation of these items with the rest of the test as indicated by the point-biserial coefficients. These data provide further support of qualitative studies that have revealed severe weaknesses of students’ understanding of trigonometry. Since our the research literature and our own qualitative data support that the understandings assessed by these items are important for learning key ideas of calculus we are retaining these items on CCR to raise awareness of the need for greater curriculum focus with ideas of angle measure (T1 and T2), function translations (U7), and trigonometric functions (T3 and T4). We are also hopeful that shifts in the conceptual focus of curriculum will lead to improved results on these items.

Validity Measures

Our analysis of CCR scores from assessing 215 Calculus 1 students at the beginning of the course shows a reasonably strong connection between levels of CCR scores and success in their Calculus 1 course, as measured by course grades. The mean CCR score of students whose course grades were A, B, or C is 11.83, while the mean CCR score for students whose course grades were D, W, or F is 8.49. Tables 6 and 7 give the grades sorted by CCR score and the mean CCR scores for students with grades of A, B, C, D, F, and W. These tables indicate a reasonably strong correlation between CCR scores and course grades, and it is noted that each of the 15 students with CCR scores of 17 or more earned a grade of A or B. This finding suggests that CCR items assess prerequisite knowledge for learning key of ideas of calculus.

Table 6 Course grades for various CCR scores

Table 7 Summary CCR score statistics for course grades

Factor Analysis

A conceptual analysis of what is assessed by individual items revealed that each item assesses a unique combination of reasoning abilities, understandings, and notational issues, and that this uniqueness of individual items results in low correlations between items. Carlson et al. (2010) report similar findings that four PCA items that primarily assessed students’ ability to use function composition were weakly correlated. Their clinical interviews revealed differences in the complexity of the items that might have attributed to the low correlation.

Criterion Validity

Another type of validity study involved comparison of the CCR results to the outcome of the prerequisite precalculus course at the university where this study was conducted. Since students who earn grades of A, B, or C in a precalculus course are presumably ready to learn the content taught in a Calculus 1 course, testing precalculus students at the end of the course should reveal a correspondence between success in the course and a CCR score. Data from 30 precalculus students (Tables 8 and 9) at the same university were tested at the end of their course. The correlation between CCR test scores and course grades in precalculus was 0.58 with the following results that, when combined with the predictive study, yields reasonably consistent results.

Table 8 Basic statistics from precalculus testing

Table 9 Distribution of precalculus grades by CCR score

Our data analysis illustrates two approaches for validating a placement instrument. One is to compare placement test scores to an existing criterion for placement in calculus, namely successfully completing a precalculus course. Since this placement testing is administered at the end of a precalculus course, students who have dropped out are not available for the study. The second approach for validating a placement instrument is to compare students’ placement test scores with their grades in calculus. There are also limitations to this approach since factors other than knowledge of and facility with precalculus concepts influence grades in calculus. As a result, it is unlikely that a test administered at the beginning of a course will predict accurately all students’ course grades. Since students in this study were placed in Calculus 1 by using ACT mathematics scores, and in one institution, a procedurally oriented placement exam, these conditions likely reduced the variation in the CCR scores of our sample, resulting in the likelihood that the predictive power of CCR scores was reduced.

These results of this analyses revealed that students who receive higher CCR scores generally performed better in Calculus 1. In fact, students who scored 11 or higher on CCR passed calculus (grade of A, B, or C) at rates of 66 %, 86 % and 91 % at the three universities participating in this study. This finding is consistent with the results reported by Carlson et al. (2010) using the Precalculus Concept Assessment (PCA). The authors reported that 77 % of 248 Calculus I students who took PCA at the beginning of a fall semester, and received a score of 13 (out of 25) or higher, were awarded a course grade of A, B, or C. However, the 0.51 correlation coefficient between the initial CCR score and final course grade was slightly higher than the correlation coefficient of 0.47 between PCA scores, also administered at the beginning of a fall semester, and final course grade. The success rate (at least a grade of C) of 95 % or greater for students who scored a 15 or higher on CCR suggests that collectively the items on CCR assess essential understandings that are used in beginning calculus. However, a success rate of 27 % for those with CCR scores less than 9 indicates that some students are achieving grades of A, B, or C in Calculus 1 without initially understanding many fundamental function concepts. This finding might be due to the strong procedural emphasis in this calculus curriculum, rather than an indication of the overall efficacy of CCR to detect strengths and weaknesses in students’ foundational knowledge for calculus.

Beginning Calculus Students are Not Prepared to Understand Calculus

An examination of the CCR response patterns of the 601 students completing CCR at the beginning of calculus revealed severe weaknesses in these calculus students’ understandings and reasoning abililities of ideas on which calculus is built. The majority of students were unable to answer both proportional reaosning questions correctly and only 9 % of the 631 students answered all three function word problems (See Appendix A, Items 2, 7, 12), suggesting weakness in their ability to construct meaningful formulas by examining the quantities in a dynamic word problem context. Another area of difficulty was in students’ ability to compose two functions. Only 28 % of students provided a correct response to the item that asked students to define the area A of a circle in terms of its circumference C (Fig. 2) and only 29 % of the students selected the correct answer to a question that asked them to determine the area of a circular oil spill that traveled outward from the center of the spill at a speed of 2 feet per second. In another function composition item that provides a table of values for the functions f and g and asks students to determine the value of f (g (3)), only 37 % of the students chose the correct answer. Interviews with students on this item revealed that students who selected the correct answer spoke about the functions as a means of processing input values to produce output values, while students who selected incorrect answers did not. They appeared to use the table as a way to look up answers without any guiding principles for how quantities in one column were mapped to quantities in another column. We also observed that the students in this study did not perform any better on items that required them to execute standard procedures or remember common definitions. Only 21 % of the students selected the correct answer to the item that asked them to solve f (t) = 100^t for t, while the answer f ⁻¹ (t) = 1/100^t was selected by 53 % of the students.

Fig. 2

Area-circumference item

An examination of student responses on the four trigonometry function questions, 3 of which were asked in the context of unit circle trigonometry (Appendix A, Items 21, 23, 24), further revealed severe weaknesses in their covariational reasoning abilities, with 16, 21 and 17 % respectively of the students selecting the correct answers. On a fourth more traditional trigonometry item that asked students to identify the formula for the graph of a cosine function with a period 2π/3, only 21 % of the students selected the correct formula.

The following sections provide a more detailed description of student thinking on three CCR items. These descriptions reveal the nature of students’ weakness and specific reasoning abilities and understandings assessed by these CCR items.

Proportional Reasoning

Two varying quantities are related proportionally if their ratios remain constant or if they are related by a constant multiple. The CCR assesses whether students can recognize this structure in an applied context, and then set up and solve an equation stating the proportional relationship when the value of one of the proportional quantities is known. The CCR has two items that assess students’ ability to apply proportional reasoning. One item is in the form of a capture-release problem (item #1, Appendix A) with 61 % of the students in our study selecting the correct response.⁸ The other item asked students to construct a formula to represent the driving distance d on a road in terms of the number of centimeters n between two points on a map, given that 3 cm on the map corresponds to 114 km of actual driving distance. We expected that almost all beginning calculus students would select the correct response and were suprised that only 65 % of beginning calculus students selected the correct answer of d = 38x The most common incorrect choices were $d=\frac{1}{38}x$ and d = 114x. These students were clearly not imagining the quantities in the situation and how they are related and change together.

Function as Process

The CCR contains items that require students to compose two functions in a graphical, tabular, and word problem context. One of these items was a function composition word problem (Fig. 2) that prompted students to define the area of a circle in terms of its circumference. Interviews conducted with students who answered this question in an open-ended format revealed that the item assesses students’ ability to identify the quantities to be related (U1) in a word problem. The item also assesses student ability to interpret the phrase, “express area A as a function of circumference C” as a prompt to construct a formula of the form, A = <some expression that contains C> (A5). Students must recall the formulas for the area and circumference of a circle, and view these formulas/functions as processes that map values of one quantity to values of another quantity (R2). To obtain the function that relates the area and circumference of a circle, students must recognize that the two formulas can be combined it they first invert the formula that defines a circle’s circumference in terms of its radius (U5). They also recognize that by composing (U4) the area formula with the inverted circumference formula they are able to define a new formula that defines the circle’s area in terms of its circumference.

Follow-up interviews were conducted with 19 students, 7 students had provided the correct answer and 3 students had constructed each of the four most common incorrect responses. Analysis of interview data with students who provided a correct answer revealed a common approach of verbalizing the problem goal (to express area in terms of circumference). These students typically wrote the formulas A = πr ²and C = πr ², and eventually recognized that they needed to re-express the circumference formula by solving for r (or determining the inverse of the circumference formula) to obtain r = C/(2π). When students were prompted to explain how they knew to solve the circumference formula for r, a common response as articulated by one student was,

I know that A = πr ² and I need a formula for area in terms of circumference. I then need to solve C = 2πr for r so I can put r = C/(2π) in for r. This will give me a formula that takes values of C and computes values of r. When I put C/(2π) in for r we get a formula that takes values of the circumference C and computes the area A.

The students who selected this response appeared to be thinking of formulas as processes for determining values of one quantity when values of another quantity are known. Various misconceptions and impoverished ways of thinking led students to select the incorrect responses. Students selecting answers (c) and (d) were unable to say what it means to express one quantity in terms of another and students who selected answer choices (b) and (e) appeared to view the letters in the formulas as something to solve for. Interview data further revealed that students who selected the incorrect responses did not view C/(2π) as an expression that determines values for r when values of C are known. It was interesting that many students wrote the formula C = 2πr but did not recognize that they could reverse the process of the formula to obtain r = C/(2π) to express r in terms of C.

When administering the item to 631 students who completed CCR, only 28 % of these students selected the correct response. Since at least 13 % of the students (out of 631) selected each answer choice, this quantitative data (Table 10) further supports the findings from analyzing the qualitative data that the five answer choices are representative of common student answers.

Table 10 Area-circumference answer percentage

Idea of Quantity and Covariational Reasoning

Ten of the 25 CCR items (1, 2, 6, 7, 8, 11, 12, 17, 20, 23) require students to conceptualize quantities as a first step in responding to the question. In each of these items the student must first imagine some measurable attribute of a situation and subsequently define a variable to represent the values that this attribute’s measure can assume. Such an item is often followed by a request to define a formula to relate two quantities. These items are typically applied problems that also assess whether students have a process view of function. An item that assesses whether students are able to engage in covariational reasoning asks that students think about how the two quantities in a situation or applied context are changing together. One example of this type of problem is the bottle that asks students to construct a graph of the height of water in the bottle in terms of the volume (Carlson 1998).

Another item on quantity and covariational reasoning is a question that asks students to describe how the top of a 10-foot ladder, leaned in a vertical position against a wall, changes as the distance of the bottom of the ladder is pulled away from the wall at a constant rate (Carlson 1998; Kaput 1992). It is noteworthy that some items that require students to use covariational reasoning might not require that students initially identify quantities in the problem context. One example is a CCR item in which students are asked to describe the behavior of the rational function, f defined by f(x) = x ²/(x – 2). In this question students need to think about how the values of the output f(x) change while imagining changes in x. They might begin by reasoning that, as x decreases from 4 to 3, f(x) increases from 8 to 9, as x decreases from 3 to 2.5, f(x) increases from 9 to 12.5, as x decreases from 2.5 to 2.4, f(x) increases from 12.5 to 57.6, etc. CCR has 7 items (4, 6, 7, 11, 16, 21, 23) in which students need to reason covariationally to provide the correct response.

A Quantitative Reasoning and Slope Item

The following item (Fig. 3) requires that students conceptualize two quantities (the distance of the top of the ladder from the floor and the distance of the base of the ladder from the wall) (U1). They then need to imagine how the distance of the base of the ladder from the wall changes as the top of the ladder increases to twice its original distance from the floor (R3). As they imagine how these measurements change together (engage in covariational reasoning), they also need to think about how the quotient of the ratio of the changes in these two quantities (slope) (U3) changes as the distance of the ladder from the wall decreases (R3). Another correct justification involved imagining how the ratio that represents the slope of a line changes if the algebraic form of the slope is changed by doubling the value of the numerator and decreasing the value of the denominator. Students who used this approach verbalized that doubling the numerator would produce a slope twice what it was before, but doubling the numerator and decreasing the value of the denominator would produce a slope that is more than twice as large as what it was before. These students were able to imagine how concurrent changes in two lengths on the illustration impacted the values of the numerator and denominator and how these values impacted the value of the quotient.

Fig. 3

Slope of ladder item

Only 27 % of 631 students who completed CCR (Table 11) provided a correct response to this item. The most common incorrect response was choice (b), exactly twice what it was. Interviews with seven students who selected this choice revealed that these students were not conceptualizing the slope of the straight line (ladder) as a ratio of two quantities.

Table 11 Slope of ladder answer percentages

Follow-up interviews revealed that most students who chose answer (b) were only focusing on the amount of increase of the top of the ladder. When prompted to elaborate on their rationale for this choice, one student explained, “since the top of the ladder is two times higher when the ladder is repositioned, the slope will be twice as large.” The student failed to consider the effect of the shortened distance of the base of the ladder from the wall and never considered the slope as representing a ratio of the two distances. Students who selected answer (e), not enough information to determine, thought they needed exact numbers for the distances to be able to compute and compare the slopes. Students who thought the slope was less than twice what it was (answer (a)) thought that the smaller denominator made the slope less than twice what it was. Students who indicated that the ladder’s slope had not changed (answer (d)) indicated that the ladder itself did not change because the positioning of the ladder on the wall had no effect on the shape of the ladder. These students were not thinking about the slope as representing a ratio of two quantities.

A Trigonometry Item: Assessing Ideas of Angle Measure and Sine Function

The following trigonometry item (Fig. 4) relies on students having developed robust conceptions of function (R2), angle measure (T1), radian as a unit (T2) and the sine function (T3), in addition to being able to imagine and fluently reason about how two quantities change in tandem (an angle measure and distance) (R3). Responding to this item requires that students first have a conception of the sine function as representing the covariation of an angle measure and the distance of a point on the unit circle (positioned at the end of the terminal side of the angle) from a horizontal line through the circle’s center (T3, R3). Determining the function that expresses d in terms of k also requires that students conceptualize an angle measure in relation to arc length that is cut off by the rays of an angle positioned at the center of a circle and measured in lengths of the circle’s radius (T1 and T3) (k needs to be divided by 47). As students imagine k varying, they also imagine how d varies (R3) and recognize that, since the output of the sine function is measured in lengths of the radius (radian), this value needs to be multiplied by 47 to express the value of d in feet.

Fig. 4

Periodic motion item

Only 21 % of 631 students who completed CCR selected the correct answer, (e) (Table 12). As noted above, responding to this question requires that students understand ideas of angle measure, radian and sine function. They must also reason about quantities, their variation and their covariation—how the values of two quantities, k and d, change in tandem.

Table 12 Periodic motion answer percentages

Students who selected answer (a) did not understand the idea of angle measure and that an angle measure can be expressed in lengths of the radius. The interview data also revealed that these students did not understand that the input to the sine function must be expressed as an angle measure. Students who selected answer (b) did not understand the idea of angle measure. Students who chose answer (c) or (d) had a weak understanding of angle measure and radian as a measure of the number of radius lengths subtended by the rays of an angle with its vertex positioned at a circle’s center.

Concluding Remarks

A CCR cut score of 11 out of 25 (44 %) is a relatively low score on an exam that assesses fundamental ideas and reasoning abilities for calculus, suggesting that many students are succeeding in Calculus 1 without the prerequisite knowledge. The CCR break points of 11 and 9 that our data suggests could be used to advise students relative to whether they should (or should not) enroll in Calculus 1 are separated by only two CCR items. This implies small differences in the initial knowledge base of students who pass Calculus 1, and those who fail Calculus 1. This observation could be explained by the fact that the Calculus 1 courses in this study did not rely on student understanding of foundational knowledge. Our observation that 27 % of students who received a nine or below were able to pass calculus with an A, B, or C also raises questions about the conceptual focus of these Calculus 1 courses.

These data in combination with our examination of the percentages of correct CCR answers further supports that many student are taking and passing first semester calculus with severe deficiencies in their reasoning abilities and knowledge base. One explanation for this finding is suggested in a recent study (Tallman and Carlson under review) that examined 150 randomly chosen Calculus 1 final examinations selected from 246 Calculus 1 final examinations administered at institutions of higher learning across the United States in the fall of 2010. They found that the exams were highly procedural in their focus—87 % of the items were coded as recall a fact or carry out a procedure. Our results combined with Tallman and Carlson’s findings strongly suggest a potentially serious shortcoming in the conceptual focus of Calculus 1. They also point to the need for further investigation of the content focus and student learning in both precalculus and beginning calculus in the United States. Another noteworthy finding of this study is that all students who received a 17 or higher on CCR received an A or B in Calculus 1. This finding supports that CCR is assessing relevant knowledge for succeeding in calculus, whether the course has a conceptual focus or not.

Examination of student responses on the collection of items that assess students’ function conception revealed that the vast majority of students in our study did not view a function as a process. This finding corroborates results that have been previously reported in the literature (e.g., Breidenbach et al. 1992; Carlson 1998; Dubinsky and Harel 1992). We found similar data trends for the collection of items that require students to use covariational reasoning to consider growth patterns in two quantities changing together. Students were asked to describe the behavior of the function f defined by f (x) = 1/(x–2)². Only 37 % of students selected the correct answer—as the value of x gets larger, the value of f decreases, and as the value of x approaches 2, the value of f increases. Another noteworthy result is the high percentages of students who selected incorrect answers for the proportional reasoning and exponential growth items. These findings suggest a need for higher standards for curriculum and courses prior to calculus in terms of the degree to which they support students’ development of fundamental reasoning abilities and understandings needed for learning and using central ideas of calculus.

Our analysis of the CCR data suggests that it is useful as a tool to assess the effectiveness of a precalculus course or curriculum in preparing students for calculus. It can also be used to advise students about their readiness for calculus. We expect that CCR correlations with success in calculus will be higher when administered as a pre-test to students enrolled in calculus courses that emphasize understanding (making connections) and reasoning with ideas. Even though calculus courses vary in the amount of emphasis placed on skills, techniques, and understanding and using key concepts, we believe that CCR is a good measure of whether students are prepared to learn and understand calculus. However, we encourage those who administer CCR to Calculus 1 students to use the cut scores that we have suggested as advisory, and to consider local constraints and current curriculum foci in precalculus and beginning calculus to adjust break points accordingly.

Appendix

Table 13

Table 13 The nature of items, Percent Correct (P), Discrimination index (D), and Point-Biserial Coefficient (PBS)