Abstract
Capturing the nature of students’ mental representations and how they change with learning is a primary goal in science education research. This can be challenging in spatially intense domains, such as geoscience, architecture, and engineering. In this research, we test whether sketching can be used to gauge level of expertise in geoscience, using new technology designed to facilitate this process. We asked participants with differing levels of geoscience experience to copy two kinds of geoscience images—photographs of rock formations and causal diagrams. To permit studying the process of sketching as well as the structure and content of the sketches, we used the CogSketch system (Forbus et al. 2011, Topics in Cognitive Science 3:648–666) to record the time course of sketching and analyze the sketches themselves. Relative to novices, geoscience students included more geological structures and relational symbols in their sketches of geoscience materials and were more likely to construct their sketches in a sequence consistent with the order of causal events. These differences appear to stem from differences in domain knowledge, because they did not show up in participants’ sketches of materials from other fields. The findings and methods of this research suggest new ways to promote and assess science learning, which are well suited to the visual–spatial demands of many domains.
Similar content being viewed by others
Notes
This accurately describes the interface used in these experiments; since then, the interface has been streamlined so that pressing a “start glyph” button is unnecessary, and users can merge or split their ink as they desire.
In our coding scheme, each part of the sketch that contained an arrow was coded as a relational part. Although there are five arrows in the geoscience student’s sketch, the participant grouped the arrows into three spatially separated parts; thus, only three relational parts were counted. (Coding also took into account the labels given to the parts, so additional relational parts may be present in Fig. 5 in addition to those indicated in red.)
The coefficient of agreement for mean ordering was lower than the median and modal values because one student sketched the events in perfect reverse order (Kendall’s W = 0). The median value of Kendall’s W was .75 and the mode was 1.0 for geoscience students who drew at least two causal events.
Experiments 2 and 3 both showed a similar statistical interaction, i.e., an advantage for geoscience students on the geoscience images, but no difference between the groups on the control images. However, whereas in experiment 2 both groups produced a high proportion of relations on the control images, in experiment 3 both groups produced a low proportion of relations on the control images.
We used CogSketch’s ordering of events but omitted events not considered to be key events.
This pattern has also been observed in biology. Louis Gomez (personal communication, May 2012) reports that students who understand the photosynthesis process tend to copy diagrams of photosynthesis in causal order.
CogSketch normally requires the user to categorize their glyphs as objects or relations, but to allow for more spontaneity, this feature was not used in this research.
References
Ainsworth, S., Prain, V., & Tytler, R. (2011). Drawing to learn in science. Science, 333, 1096–1097.
Black, A. A. (2005). Spatial ability and earth science conceptual understanding. Journal of Geoscience Education, 53(4), 402–414.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55–81.
Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.
Chi, M. T. H., Glaser, R., & Farr, M. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.
de Silva, R., Bischel, T. D., Lee, W., Peterson, E. J., Calfee, R. C. & Stahovich, T., 2007. Kirchhoff’s pen: A pen-based circuit analysis tutor. In Proceedings of the 4th Eurographics workshop on Sketch-based interfaces and modeling.
Dihoff, R. E., Brosvic, G. M., & Epstein, M. (2004). Provision of feedback during preparation for academic testing: learning is enhanced by immediate but not delayed feedback. Psychological Record, 54, 207–231.
Do, E. Y. (2005). Design sketches and sketch design tools. Knowledge Based Systems, 18, 383–405.
Duley, J. F., Wilkins, J., Hamby, S., Hopkins, D., Burwell, R., & Barry, N. (1993). Explicit scoring criteria for the Rey–Osterrieth and Taylor complex figures. The Clinical Neuropsychologist, 7, 29–38.
Forbus, K. (2011). Qualitative modeling. Wiley Interdisciplinary Reviews: Cognitive Science, 2(4), 374–391.
Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2008). CogSketch: open-domain sketch understanding for cognitive science research and for education. In the Proceedings of the Fifth Eurographics Workshop on Sketch-Based Interfaces and Modeling. Annecy, France
Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). CogSketch: sketch understanding for cognitive science research and for education. Topics in Cognitive Science, 3, 648–666.
Gentner, D., & Ratterman, M. J. (1991). Language and the career of similarity. In S. A. Gelman & J. P. Byrnes (Eds.), Perspectives on thought and language: interrelations in development (pp. 257–277). London: Cambridge University Press.
Gentner, D., & Stevens, A. L. (Eds.). (1983). Mental models. Hillsdale, NJ: Erlbaum.
Gobert, J. (2000). A typology of models for plate tectonics: inferential power and barriers to understanding. International Journal of Science Education, 22, 937–977.
Gobert, J., & Clement, J. J. (1999). Effects of student-generated diagrams versus student-generated summaries on conceptual understanding of causal and dynamic knowledge in plate tectonics. Journal of Research in Science Teaching, 36, 39–53.
Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C. H., Jones, G., Oliver, I., et al. (2001). Chunking mechanisms in human learning. Trends in Cognitive Science, 5, 236–243.
Goel, V. (1995). Sketches of thought. Cambridge: MA: MIT Press.
Hay, D. B., Williams, D., Stahl, D., & Wingate, R. J. (2013). Using drawings of the brain cell to exhibit expertise in neuroscience: exploring the boundaries of experimental culture. Science Education, 97(3), 468–491.
Heiser, J., & Tversky, B. (2006). Arrows in comprehending and producing mechanical diagrams. Cognitive Science, 30, 581–592.
Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 1, 127–138.
Holden, M., Curby, K., Newcombe, N., & Shipley, T. F. (2010). Spatial memory: hierarchical encoding of location in natural scenes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(3), 590–604.
Jee, B. D., & Wiley, J. (2007). How goals affect the organization and use of domain knowledge. Memory and Cognition, 35, 837–851.
Jee, B. D., Uttal, D. H., Gentner, D., Manduca, C., Shipley, T., Sageman, B., et al. (2010). Analogical thinking in geoscience education. Journal of Geoscience Education, 58, 2–13.
Jee, B. D., Uttal, D. H., Gentner, D., Manduca, C., & Shipley, T. (2013). Finding faults: analogical comparison supports spatial concept learning in geoscience. Cognitive Processing, 14(2), 175–187.
Kali, Y., & Orion, N. (1996). Spatial abilities of high school students in the perception of geologic structures. Journal of Research in Science Teaching, 33, 369–391.
Kastens, K. A., Agrawal, S., & Liben, L. (2009). How students and field geologists reason in integrating spatial observations from outcrops to visualize a 3-D geological structure. International Journal of Science Education, 31, 365–394.
Kindfield, A. (1992). Expert diagrammatic reasoning in biology. Paper presented at the AAAI Symposium on Reasoning with Diagrammatic Representations I, Stanford University.
Kulik, J. A., & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79–97.
Landy, D., & Goldstone, R. L. (2007). Formal notations are diagrams: evidence from a production task. Memory and Cognition, 35, 2033–2040.
Larkin, J., & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99.
Libarkin, J. C., & Brick, C. (2002). Research methodologies in science education: visualization and the geosciences. Journal of Geoscience Education, 50, 449–455.
Liben, L. S., & Titus, S. J. (2012). The importance of spatial thinking for geoscience education: Insights from the crossroads of geoscience and cognitive science. Geological Society of America Special Papers, 86, 51–70.
Lowe, R. K. (1993). Constructing a mental representation from an abstract technical diagram. Learning and Instruction, 3, 157–179.
Marshak, S. (2005). Earth: portrait of a planet. New York: W.W. Norton.
Miller, B. W., Cromley, J. G., Newcombe, N. S., Chang, M. D., & Forbus, K. D. (2014). Supporting student science learning through sketching with on-demand feedback. Manuscript in preparation.
Moss, J., Kotovsky, K., & Cagan, J. (2006). The role of functionality in the mental representations of engineering students: some differences in the early stages of expertise. Cognitive Science, 30(1), 65–93.
Pargas, R., Cooper, M., Williams, C., and Bryfczynski, S. (2007). Organicpad: A tablet pc based interactivity tool for organic chemistry. First International International Workshop on Pen-based Learning Technologies. (pp. 1-6). IEEE.
Piburn, M.D., van der Hoeven Kraft, K., and Pacheco, H. (2011). A new century for geoscience education research. Paper presented at the Second Committee Meeting on the Status, Contributions, and Future Directions of Discipline-Based Education Research. Available: http://www.nationalacademies.org/bose/DBER_Piburn_October_Paper.pdf.
Plimmer, B., & Freeman, I. (2007). A toolkit approach to sketched diagram recognition. Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI…but not as we know it, (Vol. 1, 205–213). Swinton, UK: Computer Society.
Plimmer, B., & Hammond, T. (2008). Getting started with sketch tools: a tutorial on sketch recognition tools. Lecture Notes in Computer Science, 5223, 9–12.
Proffitt, J. B., Coley, J. D., & Medin, D. L. (2000). Expertise and category-based induction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 811–828.
Schunn, C. D., & Anderson, J. R. (1999). The generality/specificity of expertise in scientific reasoning. Cognitive Science, 23, 337–370.
Schwamborn, A., Mayer, R. E., Thillmann, H., Leopold, C., & Leutner, D. (2010). Drawing as a generative activity and drawing as a prognostic activity. Journal of Educational Psychology, 102(4), 872–879.
Shea, D. L., Lubinski, D., & Benbow, C. P. (2001). Importance of assessing spatial ability in intellectually talented young adolescents: a 20-year longitudinal study. Journal of Educational Psychology, 93(3), 604–614.
Shin, M. S., Park, S. Y., Park, S. R., Seol, S. H., & Kwon, J. S. (2006). Clinical and empirical applications of the Rey–Osterrieth Complex Figure Test. Nature Protocols, 1, 892–899.
Suwa, M., & Tversky, B. (1997). What architects and students perceive in their sketches: a protocol analysis. Design Studies, 18, 385–403.
Taylor, H. A., & Tversky, B. (1992). Descriptions and depictions of environments. Memory and Cognition, 20, 483–496.
Tversky, B. (1981). Distortions in memory for maps. Cognitive Psychology, 13, 407–433.
Tversky, B. (1991). Spatial mental models. In G. H. Bower (Ed.), The psychology of learning and motivation: advances in research and theory (Vol. 27, pp. 109–145). N.Y.: Academic Press.
Tversky, B. (2002). What do sketches say about thinking? In the Proceedings of the AAAI Spring Symposium on Sketch Understanding, 148-151.
Uttal, D. H., & Cohen, C. A. (2012). Spatial thinking and STEM education: when, why and how. Psychology of Learning and Motivation, 57, 147–181.
Uttal, D. H., Friedman, A., Hand, L. L., & Warren, C. (2010). Learning fine-grained and category information in navigable real-world space. Memory and Cognition, 38, 1026–1040.
Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H., Li, W., Linsey J., & Hammond, T. (2012). Mechanix: A sketch-based tutoring system for statics courses. In Proceedings of AAAI.
Van Meter, P., & Garner, J. (2005). The promise and practice of learner-generated drawing: literature review and synthesis. Educational Psychology Review, 17(2), 285–325.
Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: a study of conceptual change in childhood. Cognitive Psychology, 24, 535–585.
Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817.
Wetzel, J., & Forbus, K. (2012). Teleological representations for multi-modal design explanations. Proceedings of the 26th International Workshop on Qualitative Reasoning. Los Angeles, California.
White, R. T., & Gunstone, R. F. (1992). Probing understanding. New York: Falmer Press.
Yin, P., Forbus, K., Usher, J., Sageman, B., & Jee, B. D. (2010). Sketch Worksheets: A sketch-based educational software system. Proceedings of the 22 nd Annual Conference on Innovative Applications of Artificial Intelligence.
Author information
Authors and Affiliations
Corresponding author
Additional information
Author Note
The authors thank Jack Butler, Nathaniel Goldin-Meadow, Katherine James, and Nadeeka Dias for assistance with data collection and coding, and Jeffrey Usher for technical assistance with CogSketch. This research was supported by NSF grant SBE-0541957, the Spatial Intelligence and Learning Center (SILC).
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Jee, B.D., Gentner, D., Uttal, D.H. et al. Drawing on Experience: How Domain Knowledge Is Reflected in Sketches of Scientific Structures and Processes. Res Sci Educ 44, 859–883 (2014). https://doi.org/10.1007/s11165-014-9405-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11165-014-9405-2