Keywords

1 Introduction

The present research work analyses the role of classic background music in the students’ attention, emotional arousal, and performance in the e-Learning environment. The proposed approach is based on ECG readings with the aim of feeding a smart system able to perform analysis and propose recommendations for improved learning approaches.

In the context of Internet based teaching, E-learning is becoming an innovative approach for delivering well-designed, learner-centered, interactive methodology. In that context E-learning is providing a learning environment to anyone, anyplace, anytime by utilizing the attributes and resources of various digital technologies, and for that it is growing its importance in the market [1]. The near future of E-learning will, most probably, be supported by the integration of sensing, diagnose, describe and qualify a given situation. Such exercises will be related with the specification of its related Smart Systems. The aimed characteristic of that Smart Systems would be the potential to sense using biosensors able to measure various physiological parameters such as heart rate (HR), galvanic skin response (GSR), electroencephalogram (EEG) and electrocardiogram (ECG) [2, 3].

The hereby-presented research work follows the traditional research method and encloses the following research question: Does eLearning training courses with classic background music influence emotional states and will it have an impact in attention and performance?

Concerning the proposed research question, aforementioned, we argue, by hypothesis that if we define a prototype, able to conduct analysis of arousal and attention based on biosignals, then the process to determine the emotional arousal and attention of learning students could be seamlessly executed. Such development would be able to proof the usefulness of background music in eLearning materials.

This paper is organized as follows: next section is dedicated to Smart Systems and e-Learning. The state of the art, concerning background music and Emotional State and Attention detection using Electrocardiogram is presented in Sect. 3, followed by the experiment in Sect. 4. Section 5 reveals the results and finally Sect. 6 presents the conclusions of the study.

2 Smart Systems and E-Learning

Smart systems are understood as systems able to sense, diagnose, describe and qualify a given situation as well as be capable to mutually address and identify each other. They are also able to interface, interact and communicate with their environment and with other smart systems [4]. Sensors and actuators are part of the Smart Systems Ecosystem that will become, most of the times, its central component. Different approaches have been proposed in the context of E-learning, among those we highlight the development of systems based on context and emotional sensing.

Zarrad and Zaguia [5] propose a smart system with an architectural approach in which modalities are selected dynamically based on the operating context. The system considers context from different features including lighting, noise level, network connectivity, communication costs, communication bandwidth, and the social and medical situations.

Emotions have an important role in learning, and are significantly related with students’ motivation, learning strategies, cognitive resources, self-regulation and academic achievements [6]. Several contributions were developed based on the identification of a user’s emotional state with the goal of improving the eLearning experience. The Framework for Improving Learning Through Webcams And Microphones (FILTWAM) [7] was developed with the aim of interpreting the emotional state of users in an e-learning environment by using webcams and microphones. This study focused only on facial mood detection and presents a structured framework composed by the following layers: learner, device, application and data with the most complex application layer on top of it and including an affective computing tool. In that same direction, Santos et al. [8] conducted a research in computer assisted language learning, using an Arduino based infrastructure that could sense the learners’ affective state and deliver interactive recommendations. For that it was developed the Ambient Intelligence Context-aware Affective Recommender Platform (AICARP), a platform similar to the one that will be developed in the context of this study. A particularity of the platform is the sensor module that receives biosignals from the learner and information from the environment and a recommendation module that takes advantage of the intelligence of the environment using internal and/or external actuators to offer interactive personalized support to the learner. This overview brings the relevant smart systems related to the work developed in the scope of the proposed research question. In particular we look to design a smart system that analyses data from bodily-connected sensors, analyses such data and by reasoning over selected algorithms identifies a person’s emotional state. Such systems, while connected with other similar smart systems, can produce markers that give indications of common properties of different people while listening to music and performing a learning task (e.g. attending an E-learning course).

3 State of the Art

3.1 Background Music

The “Mozart effect”, i.e. the scientific finding that listening to Mozart’s piano sonata enhanced spatial reasoning [9], was supported by McCuntheon [10], while other studies found no support [11, 12]. Despite that, the study conducted to new research and subsequent hypothesis in the field. A study conducted by Thompson et al. [13] suggested that the “Mozart effect” performs a sort of manipulation of arousal and mood that conducted to enhanced performance. According to Day et al. [14], the effect of background music is rather specific and thus, needs to be tested on a selected target situation [14].

Concerning computer based environments, it was visible that, when reading news from a pocket computer in a distracting environment, the reading rate and efficiency was improved in the presence of fast music than slow music [5]. A study by Thompson et al. [13] revealed that background music disrupted the ability of writing fluently.

In order to study the effect of background music in virtual environments for learning Richards et al. [15], it was created a specific environment for a history lesson. Results revealed that listening to the soundtrack of ‘Oblivion’ conducted to an improvement in the number of facts remembered by the participants.

It is important to notice that the majority of the reported findings about background music are not representative of computer-based environments. Thus, considering the actual impact of technology in daily life it is crucial to continue progressing on that specific research field.

3.2 Emotional State and Attention Detection Using Electrocardiogram

Physiological measurements, along with face recognition, or other measurements (e.g. posture and body movements) are the indirect clues about a person’s emotional states since those cannot be assessed with some sort of direct measurements. Because of that, biosignals, such as electrocardiogram (ECG), electromyogram (EMG), electroencephalogram (EEG), galvanic skin response (GSR), blood volume pressure (BVP), heart rate (HR) or heart rate variability (HRV), temperature (T), respiration rate (RR), become the most important vital signals of the human body for an emotional assessment. Although the majority of the studies use different biosignals in conjunction with ECG in order to detect the emotional states [16,17,18], there are studies focused only in relating ECG with the expression of human emotions [3, 19].

Emotions and Autonomic Nervous System (ANS) are interrelated. ANS activity is viewed as the major component of the emotions in many recent theories about emotional assessment [20]. ANS is composed by the sympathetic and parasympathetic nervous system components [20]. The sympathetic nervous system is stimulated by changes in external environment, reacting mostly to any situation perceived as an emergency [20]. The primary function of parasympathetic nervous system is for example to coordinate activities related to the restoration and conservation of the body [21]. Both systems interact antagonistically causing the variations in interbeat intervals of consecutive heart beats [3]. Parasympathetic nervous system is more active in calm periods decreasing heart rate (HR) [17, 22]. However sympathetic activity is predominant when arousal increases conducting to the acceleration of HR [17, 22].

HRV is a measure of the continuous interplay between sympathetic and parasympathetic influences on heart rate that yields information about autonomic flexibility and thereby represents the capacity for regulated emotional response [3].

Several HRV parameters can be measured resulting from time domain analysis or frequency domain analysis. Among those parameter is the ratio between LF/HF, which is known as the active indicator of the sympathetic nervous system activity. High values of LF/HF correspond to the predominance of sympathetic activity and thus high arousal in opposition to parasympathetic activity for which the ratio LF/HF is lower, corresponding to a low arousal [23].

Progressing further towards the relevance of such parameters for the current study, it is important to value the physiological signals from ECG as a good source to analyze and predict the lack or presence of cognitive attention in individuals during task execution [3] that can achieve similar performance than EEG. Scientific evidence shows that there is a close relation between variability in heart rate and attention in adults [24], infants [25] and children [26]. It also has been shown that during tasks that require attention, HF and LF parameters of HRV have a decrease [27].

4 Experiment

The proposed experiment was designed to study emotional arousal and attention in the context of eLearning and was prepared as follows. Ten individuals where enrolled in the study (1 female and 9 males). Their ages were between 32 and 47. The participants have different educational backgrounds, all completed at least high school, and none was employed in the aquaculture business. This fact was not considered relevant since the aim was not the course contents but it was considered of relevance as they would be similar to an aquaculture candidate willing to have training in such matters.

The experiment was conducted in the scope of the AquaSmart project [28], which has the focus on data mining and big data technologies applied to the aquaculture sector. AquaSmart helps companies to transform raw data into knowledge via accurate business-driven analytical models in a seamless and efficient process. The AquaSmart project has a component of training and that leads to the relevance of such experiment in the scope of the project’s training framework.

The materials for the execution of the experiment were adopted from AquaSmart training program and adapted for the hereby described experimental setup. The study comprises two e-learning courses: (a) Introduction to AquaSmart (course 1); (b) History of aquaculture (course 2). The second course was delivered with an extract from the beginning of the first part of Bach’s Brandenburg Concerto Nº 2 as background music, which was rated as expressing happiness [29]. Both courses consisted of a learning component composed by 4 slides followed by a test with 5 questions about the learned materials. Previous studies already evaluated the impact of music in a person’s mood, in this case it is aimed a step further in evaluating music’s impact on attention [30].

During the sessions ECG measurements were executed using a Olimex SHIELD-EKG-EMG. In the end of the session a questionnaire was performed to quantify: (a) the attention of the participants in each training course; (b) the likability of each course; (c) the difficulty of each course.

The sample was divided into two groups (Fig. 1): (a) a group that performed the training course 1 with no music followed by the training course 2 with music in an automatic form, i.e., with a pre-determined time interval between slides; (b) a group that performed the training course 1 with no music followed by the training course 2 with music in an interactive form, i.e., requiring by the user to press the “Next” button of the interface to move to the next slide.

Fig. 1.
figure 1

Experimental design.

Full size image

The participants were informed about the procedures and while agreeing each signed a consent form. The procedural execution started by installing clinical electrodes to his/her left arm, right arm and right leg. Heart rate was measured 30 s before the start of each e-learning training course for baseline purposes and data was collected during the attendance to the e-learning courses. After the session, each subject rated the level of difficulty, attention and motivation towards each training course on a seven-point scale instantiated from “very little” (1) to “very much” (7).

Heart Rate Variability was measured, according to specification, as beat to beat variation in the heart rate (R-R intervals) of ECG. Such measurement quantifies the interplay between sympathetic and parasympathetic system of ANS, and it was acquired using Artifact software [2]. Special emphasis was given to LF and HF of HRV, related with attention, and the ratio LF/HF indicator of arousal (more details in Sect. 3.2) that were measured while resting and along the proposed learning tasks.

5 Results

As an example, the results of participant 1 (P1), concerning training course “AquaSmart” (course 1) with no music, revealed that LF/HF ratio increased during learning phase, as expected. Since LF and HF values, calculated in milliseconds squared, decreased; it can support that such participant was more attentive during the learning stage (Table 1). The performance achieved during the test phase for training course 1 in an automatic form was 80%.

Table 1. Training course 1 with no music for P1.
Full size table

The next tables present LF and HF measurements for Course 1 (no music) and Course 2 (with music) respectively for the same person designed as P1.

Participant P1 commented, after performing training course “History of Aquaculture” (course 2) with music, that the volume was excessively high and that he usually does not appreciate history related contents. Those remarks could explain the reason why HRV analysis revealed a decrease in the ratio LF/HF during learning phase (Table 2), i.e., the low level for emotional arousal indicator meaning that he didn’t like the contents of the course. Parameters related with attention (LF and HF absolute values) decreased (Table 2) revealing a high level of attention while comparing to the other course stages. The performance achieved by this participant during test phase in course 2, in an automatic form was 60%.

Table 2. Training course 2 with music for P1.
Full size table

Additionally, from the collected information of all questionnaires participants it was concluded that some of them like to study with music and others not. From the data collected using the ECG, it was possible to infer the same. This means that music helps in the learning and test phases only of a part of the participants. Such conclusion is in line to the study made by of Gião et al. in [30], which says that a same music can result in different emotional states to each person. This is also in line to the standard deviation of the graph of Fig. 2, where with music (Fig. 2 – right graph) it has a σ = 4.67 and without music (Fig. 2 – left graph) it has a σ = 2.48. This means that there is more variability of values in the first case (course with music) representing its tendency for amplified results, which implies also an amplified arousal (emotional intensity). That means that people feel the music reacting accordingly, independently if in a positive or negative way. In conclusion could be said that despite being in a positive or negative way, classic background music enhances intensity of emotion in eLearning experiences.

Fig. 2.
figure 2

(At Left) Average LF/HF ratio for course 1 with no music presented in both automatic and interactive way; (At Right) Average LF/HF ratio for course 2 with music presented also in both ways.

Full size image

When analyzed the engagement of students in the courses with music in relation to the one without can be verified that a higher commitment attention it is obtained in courses with music. That can be verified in Fig. 3 by lower average LF and HF values in absolute terms in courses with music.

Fig. 3.
figure 3

(At Left) Average LF (ms2) and HF (ms2) for course 1 with no music; (At Right) Average LF (ms2) and HF (ms2) for course 2 with music.

Full size image

Figure 4 presents the average LF and HF values of all the students, from which could be inferred that students were more attentive or engaged in the automatic course. This means, that since they had not the possibility of going back or forward to see the contents again. It indicates that in order do not fail in the test they tried to be more attentive to the contents. This is explained by the reducing in time of the LF and HF absolute values. The graph in the left of Fig. 4 shows that in relation to the “automatic” course. On the other hand, the graph in the right part of Fig. 4 shows the opposite.

Fig. 4.
figure 4

(At left) Average LF (ms2) and average HF (ms2), for courses presented in an automatic form; (At right) Average HF (ms2) and average LF (ms2) for courses presented in an interactive form.

Full size image

6 Conclusions and Future Work

The use of wearables able to obtain LF and HF values from a ECG potentiates the creation of smart systems. This can be applied to the area of education, which will allow the system to sense the students’ physiological signals enabling the possibility of inferring particular situations about the level of the attention of the student. Through this kind of systems could be implemented specific recommendation services that will suggest particular contents or topics to recover the students attention.

A smart system prototype able to conduct analysis of arousal and attention based on biosignals using wearables to measure ECG is proposed with incorporation of the analyzed HRV parameters. Thus, the process to determine the emotional arousal and attention of learning students can be facilitated and recommendations, such as a pause in the study, adjust the volume, presence or absence of the music could be suggested by the smart system.

As future work would be interesting to apply the study to a numerous population in order to have more reliable results since interesting results were found in the presented experimental setup.