Exploring teachers' perceptions on different CSCL script editing tools

Highlights

• Teacher adoption of learning design technological support is still low.

• A study of teacher usage of two such tools shows no clear-cut tool preference.

• Immediate context/constraints seem to matter more than learning design approach.

• Evidence highlights other factors, e.g., connection to existing enactment platforms.

Abstract

Despite the apparent maturity of the learning design field, and the variety of tooling available to support it, adoption among the teacher community (one of its alleged main targets) is still low. There is a lack of research on teachers' perception and use of different technological learning design tools, as existing evaluations are often restricted to a single tool. In order to explore whether there are common factors hampering teacher adoption, and which tool features might appeal to different teachers, more studies involving multiple authoring tools are needed. This paper provides a first step in this direction, describing a mixed methods study performed around a professional development workshop with 18 university teachers from multiple disciplines. This workshop exposed teachers to two different authoring tools (WebCollage and EDIT2), as they learned to create computer-supported collaborative learning (CSCL) designs and implement them. The findings of our interpretive study (which included questionnaires, observations, or group discussion recordings) support the idea that there is no single tool or set of features that are globally perceived as better, although our evidence also highlights certain factors as important for participant teachers – amongst others, the integration of learning designs with the ICT platforms for enactment, as well as with other tools that they already use in their everyday practice.

Introduction

The planning and preparation of learning activities is a very common activity in any formal education setting, and it has been studied in depth in the fields of instructional design (Gagne & Briggs, 1974) and learning design (Conole, 2013, Koper and Tattersall, 2005). The roles and actors involved in such preparation vary from setting to setting: a large online university may have specialized roles and technical support staff while, in a smaller institution, the teacher may have to face most of the work involved. Even in research-driven scenarios (e.g., De Jong et al., 2010), teachers often play a role in adapting the provided plan to the context of their concrete classroom. Thus, it is commonly considered that teachers play a crucial role in the process of learning design (Casey et al., 2008).

Throughout the years, researchers have proposed computer-aided support to learning design (to make pedagogical decisions explicit) and computer-interpretable representations of the designs (to ease the creation of the technological environment to support learning). These proposals have varied in emphasis and audience: from specialist learning designer tools to others aimed specifically at teachers (who might not be expert in ICTs or modeling languages) as designers (Dalziel, 2003, Griffiths and Blat, 2005, Hernández-Leo et al., 2006, Laurillard et al., 2013). These researcher-driven tools are often developed in co-design efforts with a limited number of target users (e.g., teachers, see for example Laurillard et al., 2013). However, despite the variety of approaches and technical support proposed in this area (see Persico et al., 2013, Prieto et al., 2013, for a recent sample), the adoption of learning design tools in the teacher community is still low (Berggren et al., 2005, Griffiths et al., 2011, Mor and Mogilevsky, 2013, Neumann et al., 2010).

This problem is especially apparent in pedagogical approaches such as computer-supported collaborative learning (CSCL), which studies “how people can learn together with the help of computers” (Stahl, Koschmann, & Suthers, 2006). The added social complexity of this kind of approach has prompted the proposal of specific learning design approaches such as scripting (the structuring of the collaboration process to make such social learning more productive, see, e.g., Fischer et al., 2007, Kobbe et al., 2007), and their own range of supporting tools (e.g., Harrer et al., 2009, Hernández-Leo et al., 2006). Not surprisingly, in the complex case of CSCL we also find the same lack of adoption of research proposals in teacher practice (Chan, 2011, Looi et al., 2011).

There is not, however, a clear understanding of the reasons behind this lack of adoption or about the tool features that could appeal most to the teacher community. We could interpret the lack of adoption as the natural resistance (or negotiation) to the introduction of any new technology in the local teaching culture (Demetriadis et al., 2003). However, this lack of widely-accepted understanding might also be related with the fact that most learning design studies normally consider and evaluate only one approach or tool, thus making it difficult to accumulate knowledge in this regard (Dobozy, 2013).

There exist compilations of different technological approaches and tools for educational design (Botturi & Stubbs, 2008), as well as several comparative analyses of different learning design technologies (Prieto et al., 2013, Vignollet et al., 2008). However, all these studies take on a specialist/researcher (as opposed to a teacher) perspective. Masterman and others (Masterman, 2006, Masterman et al., 2009, Masterman and Manton, 2011, Masterman et al., 2013) have initiated studies of teachers' perceptions of learning design from the conceptual and technological points of view. However, studies in which teachers have the opportunity to experience and compare different tools within their local, authentic setting are still scarce. This kind of studies can help us understand which features teachers perceive as more useful when they design for learning (in contrast with evaluating whether teachers like the one approach/tool proposed by the evaluation team, which is the usual approach). Providing empirical evidence not centered on a single approach can thus be invaluable in helping design new technological learning design tools that could overcome the current adoption hurdles.

In this paper we explore this issue of practitioner perceptions of learning design technological tools, from an interpretive, naturalistic perspective (Orlikowski & Baroudi, 1991). Our mixed methods study involved 18 university teachers from a wide variety of disciplines, who used two different tools for authoring designs (concretely, for authoring CSCL scripts). The structure of the paper is as follows: first, we describe in more detail existing related work on the issue of teacher adoption and perception of learning design and CSCL script authoring tools; next, we describe the context, methodology and results of our exploratory mixed methods study; then, we discuss these results, including implications for authoring tool designers and also providing advice for similar future studies.