An empirical study on m-learning adaptation: Learning performance and learning contexts

Highlights

• Improving the learning performance in a graduate course using an adaptive system.

• We establish a positive correlation in the usage of the adaptive system.

• Adapting the learning contents to the learner's context and mobile device.

• We analyze the contexts specified by the students in the experiment.

Abstract

M-learning enables students to learn everywhere and at any time. But mobility also brings a new challenge. Students may now be constantly moving and the context from which they learn has to be adjusted dynamically. Therefore adaptation is becoming increasingly important when it comes to m-learning. This paper presents an empirical study to assess the learning performance and attitude of graduate students when they use an adaptive mobile system that tailors learning contents to their skills, their device and current context. Results suggest that mobile adaptation had a limited impact in learning performance of practical skills when compared to an e-learning approach. Information about the context of use of the mobile system was also collected and compared with traditional computer accesses. Results suggest that students learned in similar context independently of the way that they used to access learning contents. This may challenge current assumptions about the mobility of students.

Introduction

Adaptive learning systems aim to produce the most suitable sequence of learning units for each particular learner to work with. Intelligent tutoring systems (ITSs) were notably developed in the 1980s (Burns & Capps, 1988), but it was with the advent of the internet in the 1990s that came electronic learning (e-learning) and a new hope for time -and location-independent personalization. Adaptive hypermedia (Brusilovsky, 1999) presented a wide variety of techniques for adaptation that could take advantage of the nature of the new media and use information to personalize learning contents dynamically. Lately, standardization (Bohl, Scheuhase, Sengler, & Winand, 2002) introduced a set of specifications to create interoperable learning experiences.

Mobile learning (m-learning) comes more recently to stage boosted by the wide spread of mobile devices. It is an extension of e-learning in which mobile devices and wireless technologies are used to support the learning process. M-learning aims to further extend the e-learning paradigm, i.e., the ubiquity of learning, facilitating learning anytime and anywhere (Pu, Lin, Song, & Liu, 2011). It even increased the promises of the ‘learn at anytime & anywhere’ paradigm but it also posed new challenges to adaptation (Costabile et al., 2008, Smyth, 2004). Firstly, mobile devices present a huge level of fragmentation in terms of the amount of devices, technologies and media formats. A second challenge is mobility as learners now can move and the context in which they learn is now also constantly changing. Based on these challenges the related work is divided into two categories: (1) m-learning, with special focus on m-learning adaptation and (2) the concept of context in e-learning, especially in what concerns context-based adaptation.

There is much literature and contributions about mobile learning, several studies have focused in applying m-learning in different areas (Vavoula, Pachler, & Kukulska-Hulme, 2009). For instance, Vavoula et al. (Vavoula, Sharples, Rudman, Meek, & Lonsdale, 2009) presented a tool called Myartspace that offers a mobile learning service used when the children visit museums and they have to work later in classrooms. This tool tries to fill the gap between the activities in museums and classrooms. The tool was evaluated with different experiments and they concluded that the students who used the tool were more motivated and found the learning experience joyful. Pimmer et al. (Pimmer, Mateescu, Zahn, & Genewein, 2013) applied mobile learning in medicine. Authors presented an experiment with students of a medical Master degree. It was observed that the students who used the mobile devices with voice, pictures and images annotation benefited of a better learning experience than other students who only used voice or pictures. From a pedagogical perspective, other authors (Sharples, Arnedillo-Sánchez, Milrad, & Vavoula, 2009) showed how to design and evaluate mobile learning experiences. One of the aspects to highlight of this work is that it shall not be assumed that the context of use is fixed and well defined. In mobile learning, the context of use can vary significantly, for instance, in terms of ergonomics (user posture, lighting and background noise). On the other hand, several studies have focused on m-learning architectures (Motiwalla, 2007, Sharma and Kitchens, 2004) for supporting the learning process with mobile devices.

Our focus is on the research conducted in m-learning adaptation. The wide variety of existing mobile devices and their different technical characteristics is a usual concern (Zhao & Okamoto, 2008). Different architectures (Bomsdorf, 2005, Capuano et al., 2005, Trifonova and Ronchetti, 2004), frameworks (Chen et al., 2008, Motiwalla, 2007) and systems (Garcia-Cabot, 2013, Garcia-Cabot et al., 2011) have been proposed to deal with this ecosystem providing adapted contents to learners. Several of these systems are described below.

Trifonova and Ronchetti (Trifonova & Ronchetti, 2004) proposed an architecture for supporting mobile devices in LMSs (Learning Management Systems). Their learning environment should have at least three new functionalities: (1) Context Discovery, (2) Mobile Content Management and Presentation Adaptation and (3) Packaging and Synchronization. The architecture was tested on a real project (Trifonova, Knapp, Ronchetti, & Gamper, 2004), called ELDIT (Elektronisches Lernerwörterbuch Deutsch-Italienisch). An important difference in the level of connectivity of the learner was found when comparing the e-learning system with the m-learning counterpart. In the former, the learner is usually connected to the system during the whole learning session; but in an m-learning system, the student may not be always connected to the system and disconnection periods were observed for different reasons (connection cost, infrastructure problems, etc.). Gómez et al. (Gómez and Fabregat, 2010, Gómez et al., 2009) proposed a system that breaks down adaptation in two sub-processes: (1) adaptation process at design-time and (2) adaptation process at run-time. The first sub-process used IMS-LD (IMS-Learning Design) (IMS, 2003) which is a standard that provides a generic and flexible language to model and implement the learning design. Adaptation at run-time was performed during learner's interaction with the LMS system, tailoring contents according to the student's characteristics.

Other relevant aspect related to adaptation is the use of agents (Sklar & Richards, 2010). Literature discusses about integrating techniques of artificial intelligence and multi-agent systems in frameworks, architectures and systems for mobile learning adaptation. Andronico et al. (Andronico et al., 2003) proposed a multi-agent system which includes a module that provides recommendations to the learner based on the preferences of similar learners and on her profile. Al-Sakran (Al-Sakran, 2006) proposed an architecture of mobile agents that provides communication between students and facilitates individual and collaborative work. A relevant feature of this proposal lays in supporting the process of composing personalized content for individual users. De-Meo et al. (De Meo et al., 2007, De Meo et al., 2009) proposed the X-Learn system, which is a multi-agent system with three agents: (1) User-Device Agent, (2) Skill Manager Agent and (3) Learning Program Agent.

The second group of related works deals with context-awareness or context-based adaptation. The concept was firstly introduced in 1994 by Schilit and Theimer (Schilit & Theimer, 1994): “One challenge of mobile distributed computing is to exploit the changing environment with a new class of applications that are aware of the context in which they are run. Such context-aware software adapts according to the location of use, the collection of nearby people, hosts, and accessible devices, as well as to changes to such things over time.” A system or an application is also considered ‘context-aware’ if “it uses context to provide relevant information and/or services to the user, where relevancy depends on the user's task” (Dey, 2001).

The representation or categorization of contexts can be approached from different perspectives. Some authors (Martín, Carro, & Rodríguez, 2006) think of the context as the idle time of the user (e.g. time in which the user is waiting for a bus), while other authors (Gomez, Zervas, Sampson, & Fabregat, 2012) propose the use of contextual elements related to the place (location, zones, interactive space, etc.), time (task duration, task scheduled, etc.) or physical conditions (illumination, noise level, weather conditions, etc.), among them. An empirical study carried out by Kim et al. (Kim, Kim, & Lee, 2005) identified the relevant context used in mobile Internet. Context information was categorized into two types: personal and environmental. The former refers to information about the person using the mobile device like her emotional (e.g. joyful or depressed) and physical (e.g. moving or standing) conditions. The latter describes external circumstances like the user's location or the number of people who are physically close to the user. Based on these definitions and categorizations, there are different approaches and proposals of context-aware systems or architectures in the education field. Martin et al. (Martín et al., 2006) proposed a new system of context-based adaptation for m-learning, which considered the amount of idle time as the context. Adaptation consisted in selecting activities from a list based on the context of the learner. Gomez et al. (Gomez et al., 2012) proposed a tool for delivering adaptive and context-aware educational scenarios via mobile devices based on the IMS-LD specification.

Currently there are not many empirical studies about the educational benefits of adaptation and about the contexts in which m-learning is really used. Hsu et al. (Hsu, Hwang, & Chang, 2013) presented an approach for personalized recommendation-based mobile language learning. A mobile learning system was developed based on their approach. It provided a mechanism for recommendation of reading material. Experimental results showed that students using the new system outperformed the control group. This work focused on language learning therefore the system would need to be adapted to operate in other domains. Furthermore, the context and the diversity of mobile devices were not taken into account during adaptation. Gomez et al. (Gómez et al., 2013, Gómez et al., 2014) assessed the use of context-aware adaptive and personalized system. They analyzed the subjective effectiveness using a questionnaire. Results suggested that the attitude of students toward the adaptive system was good. However, they did not assess the pedagogical effectiveness of the system. Finally, Martin and Carro (Martín & Carro, 2009) presented an adaptive system that tailors contents to m-learning environments by means of suggesting collaborative activities in different contexts. They reported the outcomes of two case studies carried out with students based on the results and feedback obtained from students. Although the case studies offered feedback about the usefulness and quality of context-based recommendations, there is no indication about the impact in the learning performance.

This study sets out to overcome these limitations. We present a m-learning system that adapts the learning contents to the current skills, to the mobile device and to the context of the learner. The system is then empirically tested in a real case scenario (a graduate course in human–computer interaction) to assess its impact in terms of learning performance and of the attitude of students. We also collected data about the contexts in which students learn in order to appraise it and to compare the contexts of use of the mobile application with a traditional web-based application counterpart. The rest of the paper is structured as follows: Section 2 presents materials and methods. Results are presented in Section 3. Section 4 presents discussion and conclusions are presented in Section 5.