ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform

Highlights

• ALAS-KA is a tool that extends the learning analytics features of the Khan Academy platform.

• ALAS-KA includes new visualizations for the entire class and individual students of more than 20 new learning indicators.

• ALAS-KA helps teachers to make decision supported by the high level information provided.

• ALAS-KA enables students to gain awareness of their learning process for self-reflection.

• ALAS-KA can be used by the course instructors to detect class tendencies and learner models.

Abstract

The Khan Academy platform enables powerful on-line courses in which students can watch videos, solve exercises, or earn badges. This platform provides an advanced learning analytics module with useful visualizations. Nevertheless, it can be improved. In this paper, we describe ALAS-KA, which provides an extension of the learning analytics support for the Khan Academy platform. We herein present an overview of the architecture of ALAS-KA. In addition, we report the different types of visualizations and information provided by ALAS-KA, which have not been available previously in the Khan Academy platform. ALAS-KA includes new visualizations for the entire class and also for individual students. Individual visualizations can be used to check on the learning styles of students based on all the indicators available. ALAS-KA visualizations help teachers and students to make decisions in the learning process. The paper presents some guidelines and examples to help teachers make these decisions based on data from undergraduate courses, where ALAS-KA was installed. These courses (physics, chemistry, and mathematics) for freshmen were developed at Universidad Carlos III de Madrid (UC3M) and were taken by more than 300 students.

Introduction

Education is being boosted by new tendencies to improve the learning process, and learning analytics is one of the most promising tools. Although there is a debate about the definition of learning analytics, we consider the learning analytics term in a broad sense as introduced in the call for papers of the 1st LAK conference¹: “Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”

There are two main approaches for making decisions based on learning analytics techniques. On the one hand, work on visual analytics (Duval, 2010, Leony et al., 2012, Mazza and Dimitrova, 2004, Schmitz et al., 2009) aims at providing students with visualizations for self-reflection and at providing teachers with visual information so that they can interpret and make decisions taking into account the educational context. Therefore, teachers and students make final decisions with the help of the visual information. On the other hand, other works (Chen and Persen, 2009, D’Mello et al., 2011, Ya Tang and McCalla, 2005, Özyurt et al., 2013) aim to implement automatic actuators, such as recommenders or adaptive systems, which take into account different variables related to the learning process to carry out their actions. These automatic actuators do not require teacher or student intervention but are usually restricted to specific indicators (while visual analytics usually cover a wider range of possibilities); however, they might make more errors in their decisions than live people.

The use of MOOCs (Massive Online Open Courses) is emerging as a new paradigm. In this context, the use of learning analytics becomes more necessary because it requires instructors to analyze and to interpret students’ learning processes on a large scale (thousands of students in a course). Tools that provide insights about this learning process are required because a teacher cannot take care of so many students in detail in an efficient way without technological help.

The Khan Academy² platform is one of the pioneer systems for running MOOCs. The Khan Academy system provides an advanced learning analytics support (considering the previously introduced broad definition of learning analytics). Some of the included features are related to the skill progress, the exercise report, or the student activity report. Even though the Khan Academy system offers this analytical support through several visualizations, some interesting information is not included in its module. Therefore, an extension is required to achieve the goal of including this additional information.

In this paper, we present our implemented ALAS-KA module (Add-on of the Learning Analytics support of the Khan Academy) as a contribution to the visual analytics area, and specifically make the following contributions:

• We provide an overview of the implemented architecture of ALAS-KA for extending the Khan Academy learning analytics support (Section 4). This architecture enables teachers to process the huge amount of educational low level data (in the form of events) and to obtain higher level learning information that can be presented in the form of visualizations and recommendations.

• We describe new types of visualizations that were not previously present in the Khan Academy platform with new types of information (Section 5). On the one hand, these types of visualizations are novel because other visual analytics works have usually focused on direct indicators (such as number of accesses, number of posts, and correct number of exercises) but these visualizations show information about complex processes (e.g. taking into account pedagogical aspects such as for unthoughtful users, hint abusers, and affective information). On the other hand, the new visualizations imply an advancement with respect to the previously supported Khan Academy functionality.

• We give an analysis about how ALAS-KA visualizations can be used for making decisions about the learning process (Section 6), illustrating with different examples with real student data from pre-graduate courses at UC3M with more than 300 students. The ALAS-KA visualizations can give teachers a general view of different useful indicators of their classes so that they can make proper corrections, enable students’ self-reflection, or use a user model’s automatic definition based on learning styles and emotions.