Resources for Teaching and Assessing the Vision and Change Biology Core Concepts

INTRODUCTION

Nearly 10 years ago, the Vision and Change national report (American Association for the Advancement of Science [AAAS], 2011) called for the life sciences community to improve undergraduate biology education by organizing instruction around five core concepts that every undergraduate biology major ought to know upon graduating: (1) evolution; (2) structure and function; (3) information flow, exchange, and storage; (4) pathways and transformations of energy and matter; and (5) systems. These core concepts are mirrored in the big ideas outlined by the Next Generation Science Standards (Next Generation Science Standards Lead States, 2013; National Research Council, 2015) and the AP Biology Curriculum Framework (Wood, 2009). Together, these documents provide a consensus framework for designing instructional materials, assessments of student learning, and evaluations of the effectiveness of academic programs.

In response to these calls to action, multiple teams of researchers have developed several complementary resources focused on the Vision and Change core concepts (Figure 1). However, the life sciences community may not be aware of these resources or may not see the unique benefits of each resource. This article highlights these resources that biology instructors and departments can use to teach, develop curricular maps, and assess student learning. The resources presented here were specifically designed to align with the core concepts of Vision and Change. Notably, they focus on multiple core concepts, distinguishing them from other resources, such as concept inventories, which focus exclusively on one domain (e.g., evolution). While the Vision and Change report provided a visionary blueprint for undergraduate biology education reform, the resources presented here provide tools that are needed to enact widespread change across life sciences departments.

UNPACKING THE VISION AND CHANGE BIOLOGY CORE CONCEPTS

The Vision and Change report identified core concepts with brief descriptions that were intentionally left broad so the biology community could elaborate on them. Two separate efforts have unpacked the core concepts and articulated their scope. The BioCore Guide (Brownell et al., 2014) was created by iteratively incorporating the feedback of more than 240 biologists from across the country. It is a set of general principles and specific statements that expand upon the core concepts for three major subdisciplines of biology that approximate the diversity of biology: molecular/cellular biology, physiology, and ecology/evolutionary biology (Figure 2). Three to four specific statements for each core concept were created for molecular/cellular biology, physiology, and ecology/evolutionary biology. The specific statements can be used by instructors to develop learning goals that are aligned with the core concepts for topics within those subdisciplines. The CE Framework (Cary and Branchaw, 2017), similar to the BioCore Guide, presents a series of general principles that expand the core concept definitions and was developed with iterative input from a separate national pool of expert biologists (n = 60). However, unlike the BioCore Guide, the principles, referred to as conceptual elements, are general and can be used across biological scales and subdisciplines. They represent the components or elements that contribute to each core concept (Figure 3). As such, they could be applied to any biological scenario.

Both the BioCore Guide and the CE Framework provide outlines for what a graduating general biology major should know and could be used by biology instructors at any course level to guide the development of instructional materials that align teaching efforts with the core concepts of Vision and Change. If instructors prefer to think along the lines of the three major subdisciplines of biology (molecular biology, physiology, or ecology/evolutionary biology), then the BioCore Guide statements may be most relevant and useful for them. If instructors are teaching more specific courses (e.g., immunology or microbiology) and want to emphasize the core concepts in ways that do not align with the three major subdisciplines of biology, then the CE Framework may be most useful for them, because it is more applicable to more specialized areas of biology. Additionally, some instructors may prefer not to teach within the artificial, yet typical organization of subdisciplinary silos of biology, so the CE Framework may be most appropriate for them to use. In general, we recommend that instructors review both of these resources and choose the resource that resonates most with their teaching philosophy. However, if departments are interested in coordinating learning goals in multiple courses in the biology major and tracking student progress over time in learning the core concepts, we encourage instructors in departments to consider using the same framework (either the BioCore Guide or the CE Framework) in multiple classes for consistency.

RESOURCES TO ALIGN COURSE CONTENT AND CURRICULA WITH VISION AND CHANGE BIOLOGY CORE CONCEPTS

At the individual course level, instructors who are interested in reflecting on or monitoring their teaching of the core concepts can map the content of their syllabi or lessons onto the BioCore Guide and/or the CE Framework to identify gaps and to guide ongoing development of their courses. For example, instructors can circle the boxes on the BioCore Guide or use the course and curricular mapping tools of the CE Framework (Supplemental Material in Cary and Branchaw, 2017) to determine which components of each core concept they are teaching. It is important to note that both of these resources are comprehensive and therefore represent what biology majors should know at the end of a 4-year curriculum. Consequently, it is unreasonable, and in fact not recommended, to teach all aspects of every core concept in a single course.

Across multiple courses at the curricular level, departments can use the Partnership for Undergraduate Life Sciences Education (PULSE) Vision & Change Rubrics (Aguirre et al., 2013), specifically the V&C Curriculum Rubric, in combination with the BioCore Guide and/or CE Framework, to self-assess their progress toward implementing curricular and other recommendations outlined in Vision and Change (AAAS, 2011). The PULSE V&C Rubrics, which were created by the PULSE Fellows, were intended to be used by departments as a comprehensive self-assessment tool to generate evidence to support their curricular review (Figure 4). Specifically, the PULSE V&C Curriculum Rubric can be used for several self-assessment purposes by a department: 1) as an initial assessment to draw attention to potential curricular gaps, 2) to inform instructional sequencing, and 3) to track curricular improvement over time (Peteroy-Kelly et al., 2019). Instructors use the PULSE V&C Curriculum Rubric to give their courses a score ranging from zero (baseline) to four (exemplar) for each of the five core concepts. For department- or program-level review, individual scores are compiled to create a single curricular map illustrating the depth and frequency at which core concepts are taught in a given department or program. This type of curricular map can then be used to facilitate discussions among faculty about whether the overall curriculum is meeting the Vision and Change recommendations. We recommend that instructors and departments use the BioCore Guide and/or the CE Framework as a reference when self-assessing their progress to help standardize their conceptions of the core concepts.

In addition to their use as a self-assessment tool for departments, the complete set of PULSE V&C Rubrics is also a component of the PULSE Recognition Program (Pape-Lindstrom et al., 2015). The goal of the PULSE Recognition Program is to provide commendation for life sciences departments that are transforming their curricula in accordance with the recommendations of Vision and Change. PULSE progression levels (modeled after Leadership in Energy and Environmental Design certification for green construction) are assigned after a site visit by PULSE Fellows, which includes review of documentation to support the self-reported rubric scores, visits to classrooms and labs, and meetings with the institution’s faculty, students, staff, and administrators. Participation in this program is meant to be an iterative process with 5- to 7-year cycles, and participating departments will aspire to increase their progression levels over time. Initial consensus rubric data collected across multiple institutions have been published (Brancaccio-Taras et al., 2016), including scores from 57 institutions on the V&C Curriculum Rubric. For more information, see the PULSE website: https://pulse-community.org/recognition.

RESOURCES TO ASSESS STUDENT UNDERSTANDING OF THE VISION AND CHANGE BIOLOGY CORE CONCEPTS

While Vision and Change defined the core concepts that undergraduate biology students should learn and tasked the biology community to teach the core concepts, it did not provide tools to assess student learning of these core concepts. To support this need, biology education researchers have developed multiple assessment tools that were designed to test student understanding of the Vision and Change core concepts. Some of the assessment tools are intended be used in individual courses to measure learning gains pre–post, while others are designed to be used by departments to assess and provide benchmarks of student learning across an entire biology curriculum. Education researchers have collected evidence of reliability and validity of these assessment tools. We describe the assessment tools that are specifically aligned with the core concepts of Vision and Change below.

Biology Card Sorting Task (BCST)

The BCST was designed to probe how students organize biological ideas (Smith and Tanner 2010; Smith et al., 2013). Developed to complement other assessment tools that probe the presence or absence of particular biology knowledge, it challenges students to sort 16 cards into groups based on their perceptions of the fundamental principles of biology. Each card has the text of a biology question, chosen to represent both a single surface feature in biology—in this case the type of organism in the question (insect, human, plant, or microorganism)—and a single deep feature in biology—a Vision and Change core concept (evolution, structure and function, information flow, or pathways and transformations of energy and matter; Figure 5). The fifth core concept, systems, was omitted because of the difficulty of disambiguating this concept from the other four concepts (i.e., systems questions almost always address at least one additional core concept in biology.)

Students are initially tasked to sort the cards into any number of groups, labeled with their own language about the fundamental principle of biology being represented (unframed sort). Subsequently, students can be asked to sort the cards into four, predetermined groups labeled with the four deep features of biology represented in the card set (framed sort). Based on card-sorting behavior, the BCST can distinguish different populations, such as biology faculty versus non–biology majors (Smith et al., 2013). Non–biology majors often associated cards in ways that represented surface features (e.g., a group of cards titled “humans” would include all of the cards that specifically mentioned human beings regardless of biological concept developed in each question: N, L, O, P; Figure 5) whereas biology faculty almost exclusively sorted the cards based upon deep features (e.g., a group of cards titled “evolution” would include all of the cards that dealt with evolutionary concepts regardless of the specific organisms mentioned in each question: K, H, N, C; Figure 5; Smith et al., 2013). Additionally, the BCST has revealed that, while advanced biology majors may not sort differently from introductory biology majors in an unframed sort, they can sort based on deep features at a more expert level when given the four core concepts for a framed sort (Bissonnette et al., 2017).

The BCST was established as a tool that could address a variety of research and programmatic assessment questions ranging from analysis of within-course shifts in students’ organization of biology knowledge (Hoskinson et al., 2017) to a multiyear, departmental program assessment about changes in student thinking across an undergraduate curriculum (similar to Krieter et al., 2016). The BCST yields a rich collection of metrics about the characteristics of the card sorts produced, and the most challenging aspect of the BCST has been the time intensity required for data analysis. This challenge has been recently addressed with the development of CARDS: Collection and Analysis of Research Data for Sorting, an online card-sorting tool developed specifically for educational and research applications. CARDS enables stakeholders to easily administer and analyze card-sorting activities with larger populations and with automated analysis for most card-sorting metrics. Additionally, this platform makes it possible to collect high-fidelity card-sorting assessment data remotely without the need for printing physical cards, data entry, or even an in-person facilitator, in some cases. The BCST in combination with CARDS can efficiently and effectively reveal key insights into how individuals organize biological knowledge and develop biological conceptual expertise. Those interested in using CARDS may visit the website (atom.calpoly.edu/cardsort) or contact inventor Gregory Scott (gscott02@calpoly.edu) for access.

Biology Core Concept Instrument (BCCI)

The BCCI (Cary et al., 2019) was designed to teach and assess student understanding of individual core concepts, as well as student ability to make connections between core concepts. There are multiple BCCIs, each with a different scenario accompanied by a set of questions. Each BCCI is built using a template (Figure 6) consisting of a short narrative describing a biological phenomenon. Each narrative addresses at least three core concepts and is followed by a series of true–false/identify (TF/I) and open-ended questions. The identify question asks students to identify which one, or both, of two core concepts is being asked about in the T/F question. Four BCCI narratives with associated questions have been developed and tested thus far, spanning a wide range of subjects/topics: recombinant humulin; Galapagos finches; sloth, moth, algae symbiosis; and antibiotic resistance. These instruments and the open-ended question-scoring rubric are available as Supplemental Materials in Cary et al. (2019), and additional BCCIs are in development.

The TF/I questions are written to align with elements of the CE Framework, allowing for the development of questions that can target isomorphic learning objectives derived from specific conceptual elements. For example, two isomorphic learning objectives for Structure and Function conceptual element 2 (Individual structures can be arranged into organized units that enable more complex functions) are: 1) Students will be able to describe how [the arrangement of the myofilaments in the smooth muscle fibers of hollow organs] leads to [a reduction in the volume of the lumen during contraction]; and 2) Students will be able to describe how [the V-formation of flight in a flock of geese] leads to [more efficient, less energetically costly flight bouts compared with solo flight]. Responses to open-ended questions are efficiently graded using a rubric based on the conceptual elements. The questions assess student ability to identify the concepts represented in the biological narrative, to apply their understanding of those concepts to answer questions about the narrative, and to make connections between the concepts in the narrative. The BCCI component scoring system provides instructors with identify, apply, and connection scores, as well as individual concept and overall scores.

The BCCIs can be used in individual courses to assess student learning and as instructional tools to provide diagnostic information to students and instructors about gaps in student understanding. For example, data collected during BCCI field testing showed that students performed better on pathways and transformation of energy and matter assessment questions with ecological narratives compared with cellular narratives (Cary et al., 2019). An advantage of the template design is that performance can be compared using the same concepts but with narratives describing biological phenomena at different scales or narratives describing different core concepts at the same scale. Beyond individual classrooms, the BCCIs may also be used in multiple courses over time at the departmental level to track progression of student learning of concepts as they progress through a curriculum. Instructors interested in developing and testing new BCCIs are invited to email Janet Branchaw (branchaw@wisc.edu).

Biology—Measuring Achievement and Progression in Science (Bio-MAPS) Instruments

Another set of assessments, collectively called Biology—Measuring Achievement and Progression in Science (Bio-MAPS), were designed to assess biology majors’ understanding of the Vision and Change core concepts at the departmental level (Smith et al., 2019). Each of these assessment tools followed a similar set of design principles and is presented to students as restricted-response, multiple T/F, or likely–unlikely items designed to assess the conceptual understanding of a large number of students outside class. There are four separate instruments: general biology (GenBio-MAPS; Couch et al., 2019), molecular biology (Molecular Biology Capstone Assessment, MCBA; Couch et al., 2015), physiology (Phys-MAPS; Semsar et al., 2019), and ecology/evolution (EcoEvo-MAPS; Summers et al., 2018). All of the Bio-MAPS instruments are freely available and can be administered through an online web portal at http://cperl.lassp.cornell.edu/bio-maps (Smith et al., 2019).

The GenBio-MAPS assessment (Couch et al., 2019) was designed as a general biology departmental-level assessment aligned directly with the BioCore Guide (Brownell et al., 2014). It consists of 39 questions; each question consists of a scenario and four to five T/F statements to test student understanding of the core concepts for the three major subdisciplines of biology as articulated in the BioCore Guide (Brownell et al., 2014): molecular biology, physiology, and ecology/evolution (see example in Figure 7). Each student answers a random subset of 15 questions for a total of 60–75 T/F statements. This assessment design was used to maximize the number of questions students complete while minimizing student fatigue. Student data are analyzed in aggregate to allow departments to measure student progress at the population level for each of the Vision and Change core concepts.

The Bio-MAPS instruments are intended to be used by departments—not individual instructors or courses—to monitor student understanding of the core concepts at up to three time points at 4-year institutions: at the beginning of introductory biology, at the end of introductory biology, and at graduation. The instruments can also be used to measure progress at 2-year colleges by administering them at the beginning and end of an introductory biology series or at the beginning and end of a specialized set of courses (e.g., anatomy and physiology). Data suggest that students show greater understanding of the core concepts at more advanced levels compared with introductory levels (Summers et al., 2018; Couch et al., 2019; Semsar et al., 2019). Student performance on these assessments could be used to help departments identify the extent to which students learn the core concepts in introductory or upper-level courses and help departments assess their proficiency in teaching the core concepts over the duration of a degree. Student performance on individual questions on the instruments can also be used to inform development of additional course-level activities targeting particularly challenging concepts. Departments can administer these assessments individually or in combination to explore student thinking across a program.

A comparison of the three different sets of assessment tools described above is presented in Table 1. While there are now multiple Vision and Change–aligned assessment tools, there is a need for exploration of these tools in different contexts and for different purposes. How and when these tools are best used is still an empirical question, and we welcome biology education colleagues to investigate and report their findings on using these assessments in different contexts and for different purposes.

USING THE RESOURCES TO CATALYZE CURRICULAR REFORM

Individual instructors can use the resources presented in this essay to guide and document reform in their own classrooms. However, the transformation in undergraduate biology education called for in the Vision and Change report will require departmental engagement beyond individual champions. Transforming a curriculum is a long-term investment that requires leadership and trust to secure faculty buy-in (Olmstead et al., 2019; Reinholz et al., 2019). It takes time to assess student learning using “outside” assessment instruments in one’s course, honestly interpret the results of those assessments, and seriously consider changing long-time pedagogical practices.

We recommend departmental leaders begin by asking their faculty to map the current core concept coverage in their courses using either the BioCore Guide and/or the CE Framework. Once collected, individual course data can be aggregated to generate a departmental curricular map that leaders can use to complete the PULSE V&C Curriculum Rubric. This will document the breadth and depth of the department’s core concept teaching, ascertain what courses are focusing on which core concepts, and identify gaps that need to be addressed as far as course design and learning goals. Departments can gather direct evidence of student learning using the BCST, BCCI, and/or Bio-MAPS assessment instruments. Use of more than one assessment instrument can provide different, yet complementary information about when and where students are learning the core concepts over the course of a curriculum. Knowing when students are scoring well on core concept assessments provides information about which courses are successfully teaching the core concepts in the discipline and which are in need of improvement. Knowing whether students are able to transfer and apply core concept knowledge learned in one disciplinary course to another or in an interdisciplinary context provides information about the effectiveness of the curriculum as a whole.

Notably, the PULSE rubrics provide tools for departments to map core concept coverage in their curricula and to self-assess the progress of their reform efforts. However, the validity and reliability of self-assessment measurements made with these rubrics has not yet been established. Student learning assessment data, collected with tools like the BCST, BCCI, and Bio-MAPS can be used as evidence of validity for a department’s self-assessment rankings on the rubrics. Data generated from these assessments confirm whether and to what extent students are learning the core concepts and therefore can be used to confirm or correct the rubric self-assessment ratings and focus reform efforts on areas of need.

So which assessment should biology departments use to measure student understanding of the core concepts? We encourage departments to consider using all of these assessments. Although departments will want to be thoughtful of assessment fatigue, the BCST and the BCCIs can be easily integrated into regular course assessments (students won’t know the difference!) and the GenBio-MAPS assessment can be administered to students at three time points (outside class) over the degree program.

CONCLUSIONS AND NEXT STEPS

Now that we have conceptual frameworks and assessments aligned with those frameworks, an important next step in transforming undergraduate biology education is developing instructional activities specifically designed to teach the core concepts. Many such activities have already been developed and are available to the biology community (e.g., CourseSource: www.coursesource.org). The impact of instructional activities on student learning should be systematically assessed using instruments like those presented in this essay. For example, when new instructional activities are introduced, the BCCIs can be used in individual courses to collect evidence of student learning that complements the instructor-designed assessments and allows the comparison of different instructional activities that teach the same core concepts within and across courses.

In summary, this essay describes a “backward design” (Wiggins et al., 1998) process that begins with identifying clear learning objectives, followed by development of tools to assess student mastery of the learning objectives, and finally, design of instructional activities to support student achievement of the learning objectives. The BioCore Guide and CE Framework refined the broad learning objectives put forth in the Vision and Change report into specific, measurable components for each core concept that can be turned into learning goals by instructors (backward design step 1: define learning goals). The BCST, BCCI, and Bio-MAPS instruments provide tools to assess achievement of the learning objectives, and the PULSE V&C Rubrics provide department-level tools to track core concept teaching and learning across a curriculum (backward design step 2: assess learning goals). Biological sciences instructors who have the experience, expertise, and creativity needed to design learning activities to teach the core concepts will lead the final step in this process: developing and assessing the impact of instructional activities and learning experiences that align with and support student learning of the core concepts (backward design step 3: develop appropriate activities aligned with learning goals).

We invite instructors to use the specific core concept learning objectives and assessment resources presented here to align, assess the effectiveness of, and publish their learning activities that are targeting the core concepts of Vision and Change. The addition of evidence-based learning activities will complete the portfolio of resources that departments and individual instructors need to respond to Vision and Change’s call to action to focus on the core concepts of biology. We acknowledge that focusing on core concepts is only one recommendation of Vision and Change and that efforts are ongoing to address teaching core competencies and research practices (Brownell and Kloser, 2015; Corwin et al., 2015; Clemmons et al., 2019). It is our hope that these collective efforts will help transform undergraduate biology instruction to the vision outlined in Vision and Change.

ACKNOWLEDGMENTS

REFERENCES