Online assessment of students’ reasoning when solving example-eliciting tasks: using conjunction and disjunction to increase the power of examples

Abstract

We argue that examples can do more than serve the purpose of illustrating the truth of an existential statement or disconfirming the truth of a universal statement. Our argument is relevant to the use of technology in classroom assessment. A central challenge of computer-assisted assessment is to develop ways of collecting rich and complex data that can nevertheless be analyzed automatically. We report here on a study concerning a dedicated design pattern of a special kind of task for assessing student’s reasoning when establishing the validity of geometry statements that go beyond a single case, concerning the similarity of triangles. In each task students are given three relations that exist either in every triangle or in special types. Their task is to verify, by creating an example, claims that argue for logically compounded claims built out of the given relations. 50 students, aged 15–16, were asked to verify or disprove claims. Their submissions were automatically characterized along categories based on the correctness of the claims they chose and the examples they used to support the claims. We focus on characterizing the properties of the conjunction/disjunction design for automatically assessing conceptions related to examples generated by the learner with interactive diagrams. Our analysis shows that our automated scoring environment, which supports interactive example-eliciting-tasks, and the design principles of conjunction and disjunction of geometric relations, enable one to assess students’ exploration of the logic of universal claims, characterize successful and partial answers, and differentiate between students according to their work.