Next Article in Journal
Rapidex: A Novel Tool to Estimate Origin–Destination Trips Using Pervasive Traffic Data
Previous Article in Journal
How Shall We Start? The Importance of General Indices for Circular Cities in Indonesia
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 20
10.3390/su132011169
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Emotionally Engaged Learners Are More Satisfied with Online Courses

by
Ruiqi Deng
Department of Educational Technology, Jing Hengyi School of Education, Hangzhou Normal University, Hangzhou 211121, China
Sustainability 2021, 13(20), 11169; https://doi.org/10.3390/su132011169
Submission received: 3 September 2021 / Revised: 30 September 2021 / Accepted: 5 October 2021 / Published: 10 October 2021
(This article belongs to the Section Sustainable Education and Approaches)
Browse Figure
Versions Notes

Abstract

:
Research on massive open online courses (MOOCs) has tended to focus on outcome indicators valued in traditional higher education settings, particularly achievement and completion. This study highlights the differences between MOOCs and credit-bearing university courses and shifts this focus to an alternative outcome indicator—learner satisfaction. In this study, engagement is identified as an important antecedent of learner satisfaction and is conceptualised and operationalised as a multidimensional construct. This study built three regression models to identify the relative importance of behavioural, cognitive, emotional, and social engagement for learner satisfaction after controlling for personal characteristics unrelated to the criteria of good teaching. The analysis showed that engagement explained approximately 20% of the variance in learner satisfaction with MOOCs. Emotional engagement was more influential for predicting learner satisfaction than cognitive engagement and behavioural engagement. Social engagement had no significant effect on learner satisfaction. Demographics (age, education level, and origin) and motivation were of limited utility in predicting learner satisfaction with MOOCs, accounting for 4% and 2% of variance, respectively. Based on research findings, the article presents the following propositions: (1) configure the MOOC teaching and learning environment in a way that enhances emotional engagement; (2) statistically adjust for age, education level, origin, and motivation when interpreting learner satisfaction results; and (3) monitor the level of emotional engagement and implement educational interventions to provide support for emotional disengagers.

1. Introduction

The disruptive potential of massive open online courses (MOOCs) has generated increasing scholarly interest in MOOC learning outcomes, particularly in achievement [1,2,3] and retention [4,5,6]. While these are important measures for evaluating the success of university students, it could be argued that these indicators are less relevant in the context of MOOCs. Compared with credit-bearing university courses, learners have the option to enter and exit the MOOC teaching and learning space without restrictions. They are not required to pay fees, except when they intend to obtain a (paid) certificate of completion. People display a combination of intrinsic, extrinsic, and social motivations when registering for a MOOC [7]. Many MOOC learners are more concerned about achieving their own learning goals rather than achieving instructor-determined learning goals such as high performance [8]. A high dropout rate, which is often considered as a threat in credit-bearing university courses, is not a breach of expectations in MOOCs and should not be simply viewed as indicating poor course quality [9]. Rather, it could be the natural result of an open learning space and a free registration process.
Recent research has tended to endorse the view that MOOC success should not be simply evaluated through traditional measures such as achievement and retention, but through learner-centred indicators such as satisfaction [9,10,11]. In addition, it is not surprising to observe that some MOOC practitioners are keen to find out whether the courses they produced and/or facilitated are successful or not [12]. For the above reasons, this study shifts the focus from traditional outcome measures to an alternative outcome indicator—learner satisfaction with MOOCs. Consistent with previous MOOC research [10], this study defines learner satisfaction as an individual’s overall assessment of his or her learning experience and operationalises learner satisfaction as an important outcome indicator.
The massiveness and openness of MOOCs affords people the opportunity to engage with course content, instructors, peers, and themselves in different ways, and this engagement process can determine the outcomes of learning [7,13]. However, there is a general lack of research relating to how engagement affects learner satisfaction with MOOCs. The apparent knowledge gap provides impetus for this study. Identifying the engagement–satisfaction relationship extends the theoretical understanding of the factors that can influence learner satisfaction with MOOCs. More importantly, determining the strength of the engagement–satisfaction relationship can provide important rationale for MOOC researchers and practitioners to constantly monitor the level of engagement and configure the teaching and learning environment that promotes engagement.
MOOC learners manifest their engagement in distinct ways. Not all learners persevere through the entire MOOC, strictly following the learning path predefined by MOOC instructors. Some learners perceive MOOCs as modularised resources. They study a portion of the MOOC to fill immediate needs and skip others [14]. Some learners capitalise social learning and professional networking opportunities in MOOCs [15]. Some learners consume MOOC video lectures as edutainment resources in their spare time [14]. The diversity in how learners engage in MOOCs has prompted researchers to adopt a more nuanced approach to operationalising engagement. This study conceptualises learner engagement in MOOCs as a process factor that comprises four discrete dimensions. The study aims at identifying the specific engagement dimensions that can predict learner satisfaction with MOOCs and estimate the relative importance of behavioural, cognitive, emotional, and social engagement for satisfaction. To determine if the engagement–satisfaction relationship is true, this study controls for personal characteristics unrelated to the criteria of good teaching when investigating the influence of engagement on learner satisfaction with MOOCs.
This article begins with an overview of learner satisfaction as an outcome indicator, followed by a more detailed literature review of the conceptualisation and operationalisation of learner engagement in the MOOC context. The literature review also considers personal characteristics which may confound the engagement–satisfaction relationship. The research methods section explains the steps taken to collect and analyse the data. The findings are discussed in the context of the MOOC and broader education research. The researchers then discuss the potential limitations of this study and provide avenues for future research. The conclusion presents the theoretical contributions and practical implications.

2. Literature Review

2.1. Learner Satisfaction

The measurement of learner satisfaction is typically achieved by asking students to rate the quality of their educational experience in a course [16]. Learner satisfaction is often used interchangeably with student rating [17], student evaluation [18], and the perceived quality of instruction [9]. The higher education literature shows that students are competent enough to rate the quality of their educational experience, and this metric positively correlates with outcome indicators such as student achievement, instructors’ self-evaluation, and the evaluation provided by trained observers [19]. Measuring learner satisfaction has important implications for both credit-bearing university courses and MOOCs. In higher education institutions, learner satisfaction is frequently used for purposes such as curriculum improvement, performance review of academic staff, allocation of funding, and strategic planning [20]. Similarly, learner satisfaction with MOOCs provides feedback to help construct or redesign a MOOC, assists policy makers and university administrators in making decisions about future MOOC production and investment, and helps prospective learners to select courses for study.
Early MOOC research on learner satisfaction was centred on learners’ perceptions of and attitudes towards MOOCs. For instance, Khalil and Ebner [21] reported that approximately 6% of learners were not satisfied with the level of interaction in MOOCs. These early studies tended to be descriptive in nature but nevertheless provided an overview of how learners were satisfied and dissatisfied with MOOCs. More recently, MOOC scholars have conducted correlational studies to identify the antecedents of learner satisfaction. Li [22], for example, found that personal characteristics such as education background and the number of previous online courses taken explained the variance in MOOC learners’ satisfaction. Rabin, Kalman, and Kalz [10] revealed that MOOC learners’ age indirectly predicts satisfaction. Hew, Hu, Qiao, and Tang [9] reported that environmental factors, such as the flexibility of the course schedule, significantly predict learner satisfaction with MOOCs.
The massiveness and openness of MOOCs prompt learners to engage with course content, instructors, peers, and themselves in different ways, and this engagement process can determine the outcomes of learning [7,13]. However, there is a general lack of knowledge relating to how engagement affects learner satisfaction with MOOCs. Identifying this relationship extends the theoretical understanding of the factors that can influence learner satisfaction with MOOCs and provide a theoretical basis for continually monitoring and improving learner engagement. The apparent knowledge gap leads to the research question: How does engagement influence learner satisfaction with MOOCs?

2.2. Conceptualisation of Learner Engagement

Learner engagement was originally conceptualised as a unidimensional construct, representing students’ time on task [23], sense of belonging to school [24], participation in academic activities [25], or attention and efforts invested in learning [26]. More recently, research has begun to conceptualise learner engagement as a multidimensional construct that incorporates dimensions such as behavioural engagement, cognitive engagement, emotional engagement, and social engagement [27]. Learner engagement has attracted considerable scholarly attention in the MOOC literature. This is because active learner engagement is often associated with desirable outcomes, such as improved academic performance and higher completion rates [28,29]. Learner engagement is sometimes used interchangeably with engagement [30], course engagement [31], learning engagement [29], and student engagement [32]. A recent review of 102 empirical studies showed that engagement is one of the most important topics in the MOOC literature [7]. However, synthesising research findings related to learner engagement is not an easy task because this notion is conceptualised very differently in MOOC research. The complexity is further exacerbated by the distinct approaches employed by MOOC scholars to measuring engagement.
The definition of learner engagement was fuzzy in early MOOC research—the word ‘engagement’ appeared in the titles of some publications but was not formally defined in the text [30,33]. The meaning of engagement was close to ‘participation’. Researchers investigated learner engagement in MOOCs for discrete activities, such as watching video lectures [34], attempting assessments [35], and spending time on academic tasks [36]. This type of engagement in MOOCs was later classified as behavioural engagement, representing learners’ observable actions and involvement and participation in academic-relevant activities [13,27]. Behavioural engagement is sometimes interwoven with social engagement, which is defined as learner–learner and learner–instructor interaction in MOOCs [27]. A recent review of 103 MOOC studies indicated that 19 articles addressed social aspects of learning in MOOCs [37]. Some scholars reported social engagement as a subdimension of behavioural engagement [38]. This is because interactions with peers and instructors are observable and form a part of participation in educational activities. Recent research shows that MOOC learners differentiate between behavioural and social engagement, and social engagement is a separate construct from behavioural engagement in MOOCs [27].
Some scholars have asserted that the meaning of engagement should be richer and more informative than its narrow interpretation in the MOOC literature. They argued that not all engagement domains can be directly observed and drew scholarly attention to domains that are less overt and more internal. A stream of scholars emphasised learners’ mental investment in the MOOC study process to comprehend complex ideas and master difficult skills [13,27]. This engagement domain is usually referred to as cognitive engagement. For example, Li and Baker [13] inferred cognitive engagement in MOOCs from video interaction events, such as slow watching, backward seeking, and pausing. Deng, Benckendorff, and Gannaway [27] defined cognitive engagement as motivated sets of behaviours in MOOCs, such as searching for further information and re-watching video lectures. Another less identifiable engagement dimension is emotional engagement, which is interpreted as the emotional connections learners make with instructors, peers, and MOOC content [27,39]. The presence of positive emotions [40] and the absence of negative emotions [41] are often considered as good signs of emotional engagement in MOOCs.

2.3. Operationalisation of Learner Engagement

The operationalisation of learner engagement varies across MOOC studies. Clickstream data, or log files, are sometimes used as a proxy for behavioural, social, and cognitive engagement in MOOCs. Clickstream data are relatively easy to obtain and can often objectively reflect the level of engagement in MOOCs [42]. However, caution must be exercised when interpreting such data because they are inferred rather than queried. For example, Lee [43] found that a simple count of learning activities was not an accurate proxy for behavioural engagement in MOOCs because it did not reflect the quality of learning. Moreover, it can be difficult to gauge emotional and cognitive engagement just based on clickstream data. Li and Baker [44], for example, initially proposed that individuals who never paused the videos and individuals who paused fewer and fewer times as the MOOC continued had low levels of cognitive engagement. Later on, Li and Baker [13] reported that pausing was not a conceptually clear measure of cognitive engagement in MOOCs because learners paused videos for reasons unrelated to learning. In addition, clickstream data only capture learners’ interactions with online materials [45]. MOOC participants frequently engage in learning opportunities beyond the platform [46], and such engagement should not be overlooked.
Self-report measures, particularly self-administered surveys, are also frequently employed to understand learner engagement in MOOCs. Self-report measures have the advantage of capturing engagement domains that are less observable, such as cognitive and emotional engagement. For this reason, a survey was adopted to measure learner engagement in this study. However, a major concern is that survey items might be worded too broadly so that engagement in specific learning environments is not captured [47]. This study addressed this potential issue by adopting an instrument designed specifically for measuring learner engagement in the MOOC context. Another concern is that people may not recall their MOOC engagement due to the lapse of time. To minimise the memory effect, the researchers only invited people who had participated in a MOOC in the last 12 months to complete the survey.
There is variation in the operationalisation of learner engagement when surveys are adopted. Early MOOC research tended to focus on behavioural or social engagement because they are evident and relatively easy to trace and interpret. Some MOOC researchers operationalised learner engagement as a multidimensional construct and investigated two or more engagement dimensions [39]. Some researchers acknowledged the multidimensionality of learner engagement but chose to measure engagement as the sum of multiple engagement dimensions [29]. Although this unified measurement approach greatly simplified data analysis and interpretation, the information about the driving factors and the effects of each engagement dimension was lost in the research process.
MOOC participants manifest engagement in different ways. They may find a MOOC interesting to learn (emotional), make notes when studying a MOOC (behavioural), use search engines to help solve difficult problems encountered in the learning process (cognitive), share course notes with the peers in the discussion board (social), and feel happy to see that their contributions are appreciated by other learners (emotional). The researchers of this study emphasised the importance of understanding the influence of discrete engagement domains on satisfaction and refined the research question: How does behavioural, cognitive, emotional, and social engagement influence learner satisfaction with MOOCs?

2.4. Personal Characteristics

The relationship between learner engagement and satisfaction can be affected by personal characteristics. Personal characteristics are attributes of a MOOC learner, such as gender, which can potentially affect the learning process and outcomes [48]. Some of the personal characteristics are irrelevant to the criteria of good teaching and should be considered when investigating the engagement–satisfaction relationship. Here, good teaching is defined as the adoption of effective instructional practices and the provision of appropriate teaching and learning environments that prompt learning processes and outcomes. The researchers of this study differentiated between two types of personal characteristics: demographics and MOOC-related background factors. Demographics are statistical data about the characteristics of a population (e.g., education level), and MOOC-related background factors represent individual factors directly related to MOOCs (e.g., prior MOOC experience).
Demographics can directly affect the MOOC learning process e.g., [31,49] and may confound the engagement–satisfaction relationship if they are not controlled for [50]. For instance, Shapiro, et al. [51] reported that individuals who had a Bachelor’s degree showed higher levels of positivity towards a MOOC. This finding was echoed by that of Li [22], who discovered that participants with higher degrees were more satisfied with the MOOC learning experience. Rabin, Kalman, and Kalz [10] found that age indirectly affected learner satisfaction through the number of video lectures accessed. In addition, the broader education literature suggests that female students display a higher level of satisfaction than their male counterparts in e-learning environments [52]. Comparative research also indicates that university students’ country of origin can affect their satisfaction with educational experience [53]. Based on a review of 102 MOOC studies, Deng, Benckendorff, and Gannaway [7] found that education background, country of origin, age, and gender were the most frequently investigated demographic factors. However, these demographic factors tended to be oversimplified and were often descriptively reported in MOOC research [7]. To find out if the engagement–satisfaction relationship truly exists, the present study controlled for age, gender, education level, and origin when investigating the influence of engagement on learner satisfaction with MOOCs.
MOOC-related background factors may also confound the engagement–satisfaction relationship [54]. In this study, MOOC-related background factors are conceptualised to comprise two factors—motivation and prior experience. One of the frequently investigated background factors is motivation. In the context of traditional higher education, if students have a prior interest in the subject matter, their satisfaction tends to be higher [16]. Similarly, MOOC learners with high motivations exhibited a higher level of satisfaction than individuals with low motivations [55]. Another background factor known to affect learner satisfaction is prior MOOC experience. Li [22] reported that learners who took more online courses tended to be less satisfied with a MOOC. To determine if the engagement–satisfaction relationship is spurious, the researchers controlled for motivation and prior MOOC experience when investigating the relationship between engagement and learner satisfaction. The researchers further refined the research question: How does behavioural, cognitive, emotional, and social engagement affect learner satisfaction after controlling for learners’ demographics and MOOC-related background factors?

2.5. Research Framework

To answer the research question, the researchers build three regression models to ascertain the degree to which learner satisfaction is influenced by (1) demographics, (2) MOOC-related background factors, and (3) behavioural, cognitive, emotional, and social engagement. The research framework is displayed in Figure 1.
Model 1 investigates the relationship between demographics and learner satisfaction, aiming to reveal the amount of variance in learner satisfaction solely explained by demographics. Model 2 is combined with Model 1 to identify the amount of variance in learner satisfaction explained by MOOC-related background factors. Model 3 is used in conjunction with Model 1 and 2 to investigate whether the relationship between engagement and learner satisfaction is spurious, that is, caused by personal characteristics (i.e., demographics and MOOC-related background factors) unrelated to the criteria of good teaching. If engagement explains a greater amount of variance in learner satisfaction than demographics and MOOC-related background factors, it can be concluded that engagement plays a more influential role in shaping learner satisfaction with MOOCs.

3. Research Methods

3.1. Participants

The researchers obtained ethical approval from the university human research ethics committee. A pilot test was undertaken prior to the administration of a full-scale survey. Twenty volunteers were invited to review the survey and provide constructive feedback. Based on the feedback, the researchers adjusted five questions to make them easier to answer. The results of the pilot study showed that respondents could fully comprehend the instructions and questionnaire items, and there were no errors in the survey.
The participants in this study were recruited from individuals who had previously participated in one or more MOOCs offered by a research-intensive university on edX. The researchers distributed the research purpose, consent form, and instructions on how to complete the survey through email newsletters. To capture learner engagement across a range of MOOCs and platforms, the researchers also recruited participants from two social media websites. The English survey was distributed in 2019 and remained open for four weeks.
A screening question was adopted to filter out respondents who reported that they had not participated in a MOOC in the previous 12 months. If a respondent had not participated in a MOOC in the previous 12 months, the survey would end immediately, and the respondent would be directed to the exit page. Only those who had participated in a MOOC in the previous 12 months would be given the instruction to fill out the survey. The instruction required the respondents to think about the most recent MOOC they studied and keep this MOOC in mind when answering all the questions in the survey.
MOOC platforms are usually unwilling to disclose the personal characteristics of their users for privacy reasons, making it difficult to adopt probability sampling to systematically access learners. Therefore, convenience sampling was adopted in the current study. Individuals who completed the survey and provided contact details were entered into a draw with a chance to win one of 10 gift tokens. A total of 1440 observations were retained for data analysis after deleting invalid surveys and outliers. This sample size satisfied the desired ratio for conducting multiple regression analysis, which is 15–20 observations for each predictor variable [56].

3.2. Measures

The researchers used four questions to obtain the demographics of MOOC learners. Past research showed that age, gender, education level, and origin were the most frequently investigated demographics in the MOOC literature [7]. These four demographic variables were measured and used as control variables in regression models 1, 2, and 3.
Motivation and prior MOOC experience were used as control variables when building regression models 2 and 3. The researchers designed two questions to obtain MOOC-related background factors. The motivation question was created by combining and consolidating the frequently occurring items in the past validated studies [51,57]. Motivation items were grouped into seven categories: personal interest, learning for study or work, finding relevant resources, obtaining a certificate of completion, enhancing one’s resume, the MOOC being free, and socialising. Prior MOOC experience was divided into five categories: no prior experience, 1–3 courses, 4–6 courses, 7–9 courses, and 10 and more courses.
The researchers adopted the MOOC engagement scale (MES) to capture learner engagement. Deng, Benckendorff, and Gannaway [27] developed the MES to measure four engagement dimensions in MOOCs—behavioural, cognitive, emotional, and social engagement. The 12 scale items were validated by two focus groups, an exploratory survey, an expert review, a pilot study, an item purification study, and a construct validation study. A six-point Likert scale ranging from strongly disagree (1) to strongly agree (6) was used. The MES demonstrates good reliability, face validity, construct validity, convergent validity, and discriminant validity [27]. In this study, the Cronbach’s alpha values for the behavioural, cognitive, emotional, and social engagement dimensions were 0.72, 0.70, 0.73, and 0.83, respectively.
The researchers employed a global item to measure learner satisfaction. Respondents were asked to indicate their satisfaction level on an 11-point scale ranging from 0 (dissatisfaction) to 10 (satisfaction). The research showed that a unidimensional approach can be as valid as a multidimensional approach in measuring learner satisfaction because certain dimensions are more important and explain a substantial amount of variance in satisfaction [58]. Consistent with previous work [9,10], this study defined and operationalised satisfaction as a learner-centred outcome measure. In the survey, the respondents were first asked to indicate their engagement during the study of a MOOC. After that, the respondents were instructed to report their level of satisfaction at the time when they exited the MOOC, not the fluid, developing satisfaction during the MOOC study process.

3.3. Data Analysis

IBM SPSS Statistics 26 was used for statistical analysis. The researchers first calculated and reported learners’ demographics, motivations, and prior MOOC experience. Next, the researchers conducted a principal component analysis (PCA) to (1) empirically validate the MES developed by Deng, Benckendorff, and Gannaway [27] and (2) single out variables that can be perceived as indicators of behavioural, cognitive, emotional, and social engagement. Cronbach’s alpha was computed to determine the internal consistency of the MES. Four summated scales ranging from 0 to 18 were created to substitute for the original 12 variables. The summated scale values were retained for use in multiple regression analysis.
The researchers then carried out a series of multiple regression analyses to determine the relationship between engagement and learner satisfaction with MOOCs. The four factors derived from the PCA were used as predictor variables to determine which of the four engagement dimensions were better predictors of higher levels of satisfaction. All the other predictor variables were treated as categorical indices.
There are two methods of dummy variable coding: indicator coding and effect coding. The key difference between the two methods is the interpretation of regression coefficients. While the coefficients in indicator coding represent differences for each group of respondents from the mean of the reference group, the coefficients in effect coding represent differences for each group of respondents from the mean of all groups. Indicator coding is more appropriate when a logical reference group is present [56]. As this study does not contain a logical reference group, the researchers opted for effect coding when configuring the regression models.
With effect coding, the reference group is assigned the value of −1 across all dummy variables, and the coefficients represent differences for each category from the overall mean of all categories. The researchers selected the category with the lowest number of observations as the reference group (Table 1). For example, there were only four individuals without any schooling experience in the sample. Therefore, ‘no schooling completed’ was selected as a reference group for the predictor variable ‘education level’.

4. Results

The demographics of the respondents are shown in Table 2. The demographics in this study are similar to the pattern reported in recent MOOC research [22]. MOOC-related background factors are displayed in Table 3.
Prior to conducting multiple regression analysis, a PCA with varimax orthogonal rotation was employed to establish the validity of the MES and determine the factor structure among the 12 items from the MES (Table 4). With factor loadings and communalities greater than 0.4, the best solution was a four-factor model accounting for 67.68% of the common variance. The factorial dimensions were consistent with Deng et al.’s [27] instrumentation of learner engagement in MOOCs, demonstrating sound reliability and validity of the MES. The Cronbach’s alpha values for the four engagement dimensions ranged from 0.70 to 0.83, demonstrating a high degree of internal consistency. The Kaiser–Meyer–Olkin measure of sampling adequacy (MSA) index of 0.83 and a significant chi-squared value for Bartlett’s test of sphericity, χ* (66) = 5808.73, p < 0.001, suggested that the factor model was appropriate for the data.
To reduce the reliance on any single variable and measurement error, this study followed Hair et al.’s [56] recommendation to incorporate summated scales into multiple regression by replacing the original predictor variables with the summated scale values. Based on the PCA results, the researchers calculated the summated scale values by computing the sum of the variables making up each dimension of the MES. The summated scale values were used as predictor variables in multiple regression analysis (Table 5).
The researchers selected learner satisfaction as the outcome variable. The relationship among the four predictor variables and the outcome variable was assumed to be statistical rather than functional. This is because the relationship examined involved people’s perceptions, and random components (e.g., measurement errors) are always present in such a relationship. The researchers built three regression models to evaluate the influence of engagement on learner satisfaction. Model 1 contained only MOOC learners’ demographic characteristics, including gender, age, education level, and origin. Model 2 added learners’ MOOC-related background factors, including motivation and prior MOOC experience. Summated scale values representing levels of behavioural, cognitive, emotional, and social engagement were added for Model 3. This modelling process allowed the researchers to identify engagement dimensions affecting learner satisfaction by separately controlling for learners’ demographics and MOOC-related background factors. The following equations represent multiple regression analysis conducted for Models 1, 2, and 3, where β0 is the constant, β1 to β10 are the slope coefficients for predictor variables, and εi is the residual.
Model 1.
Learner_Satisfactioni = β0+ β1Agei + β2Genderi + β3Education_leveli + β4Origini + εi
Model 2.
Learner_Satisfactioni = β0+ β1Agei + β2Genderi + β3Education_leveli + β4Origini + β5Motivationi + β6Prior_MOOC_Experiencei + εi
Model 3.
Learner_Satisfactioni = β0+ β1Agei + β2Genderi + β3Education_leveli + β4Origini + β5Motivationi + β6Prior_MOOC_Experiencei + β7Behavioural_Engagementi + β8Cognitive_Engagementi + β9Emotional_Engagementi + β10Social_Engagementi + εi
The researchers checked the model assumptions prior to conducting multiple regression analysis. To test the assumptions of linearity and homoscedasticity, the researchers examined the residual plot that depicted one axis for the standardised residuals and the other axis for the standardised predicted value. The results showed that the standardised residuals were scattered randomly around a horizontal line in a rectangular shape, indicating that the linearity was not violated. This finding also implied that the variance of residuals was similar at each point of the predictor variables across the model. Therefore, the assumption of homoscedasticity was met. To ensure there was no multicollinearity in the data, the researchers inspected the tolerance and variance inflation factor (VIF) statistics. The tolerance values were well below 10, and the VIF values were all above 0.1, indicating that the assumption was met [59]. To test the assumption of independence, the researchers calculated the value of the Durbin–Watson statistic. The Durbin–Watson statistic for the model was 2.01, indicating that the residuals between the actual value and the predicted value obtained through the regression equation were independent. To test the assumption of normality, the researchers further examined the normal probability plot of the standardised residuals. The plot revealed that the values fell along the 45° diagonal, indicating that the values of the residuals were normally distributed. To ensure that the model was not affected by any influential datapoints, the researchers also used Cook’s distance to test for influential cases. The results showed that no observation placed undue influence on the regression model, and model re-specification was not required.
Multiple regression analyses were carried out to investigate the influence of demographics (age, gender, education, level, origin), MOOC-related background factors (motivation, prior experience), and engagement (behavioural, cognitive, emotional, social engagement) on learner satisfaction. The researchers sequentially added demographics, MOOC-related background factors, and the composite scores representing four engagement dimensions into Models 1, 2, and 3. All the three prediction models were statistically significant: F(18, 1412) = 3.59, p < 0.001 (Model 1), F(28, 1402) = 3.32, p < 0.001 (Model 2), F(32, 1398) = 15.58, p < 0.001 (Model 3). That is to say, all three sets of predictor variables affected the outcome variable. Models 1, 2, and 3 explained approximately 4%, 2%, and 26% of the total variance in learner satisfaction with MOOCs, respectively. These results indicated that engagement had a much greater influence on satisfaction than learners’ demographics and MOOC-related background factors.
The results of the multiple regression analyses are displayed in Table 6. Model 1 only considered the relationship between demographics and learner satisfaction. The analyses indicated that age, education level, and origin were significant predictors of satisfaction while gender was not. Specifically, individuals aged below 25 (B = −0.39, p < 0.001) and individuals aged between 25 and 34 (B = −0.23, p < 0.01) tended to be less satisfied than their senior counterparts, ceteris paribus. In contrast, people aged between 45 and 54 (B = 0.29, p < 0.05), whose highest education level was high school (B = 0.43, p < 0.01) or who had a Latin American origin (B = 0.46, p < 0.001) showed higher levels of satisfaction. The effects of age, education level, and origin remained significant in Models 2 and 3.
After MOOC-related background factors were entered into Model 2, the effects of demographics on satisfaction remained stable, except for education level. Possessing a diploma had positive effects on satisfaction in Model 2, but this effect disappeared when learner engagement was entered into Model 3. Similarly, the effects of learning for study or work (motivation) and taking one to three courses (prior MOOC experience) on satisfaction were statistically significant in Model 2 but not in Model 3. These results suggested that the effects of education level, motivation, and prior MOOC experience on learner satisfaction may be mediated by engagement.
Model 3 indicated that learner satisfaction with MOOCs was positively associated with age between 45 and 54 (B = 0.30, p < 0.01), a high school education level (B = 0.27, p < 0.05), Latin American origin (B = 0.41, p < 0.001), and higher levels of behavioural (B = 0.03, p < 0.01), cognitive (B = 0.06, p < 0.001), and emotional engagement (B = 0.17, p < 0.001). Learner satisfaction was negatively linked to age less than 25 years (B = −0.27, p < 0.01), age between 25 and 34 (B = −0.18, p < 0.05), Arabic origin (B = −0.25, p < 0.05), and participating in a MOOC because the MOOC was free (B = −0.43, p < 0.01). A number of control variables were still significant in Model 3, indicating that they captured variance in learner satisfaction beyond that explained by learner engagement. The final model also showed that gender, prior MOOC experience, and social engagement had no influence on learner satisfaction.
After controlling for the variances explicable by demographics and MOOC-related background factors, the results of Model 3 demonstrated that behavioural, cognitive, and emotional engagement were significant predictors of satisfaction but not social engagement. The three engagement dimensions explained about 20% of the variance in learner satisfaction. Emotional engagement (B = 0.17, p < 0.001) was a better predictor of learner satisfaction than cognitive engagement (B = 0.06, p < 0.001) or behavioural engagement (B = 0.03, p < 0.01).

5. Discussion

This study revealed that engagement positively influenced learners’ satisfaction with MOOCs. This result is generally consistent with the engagement–satisfaction relationship reported in online [60] and technology-enhanced learning environments [61]. In addition, this study added to the existing body of knowledge that the strength of the engagement−satisfaction relationship differs across four engagement dimensions. The analysis showed that emotional engagement had the greatest impact on learner satisfaction, followed by cognitive and behavioural engagement. A meta-analysis investigating the engagement–achievement relationship suggested the reverse—behavioural engagement was the strongest predictor of academic performance, followed by cognitive engagement, with emotional engagement being the weakest predictor [50]. It may not be appropriate to make a direct comparison between Lei et al.’s [50] study and this research, because Lei, Cui and Zhou [50] selected achievement as the outcome variable and did not report the context of the meta-analysis. Future research should find out if the different results are attributable to the use of different outcome indicators (e.g., satisfaction, achievement) or different teaching contexts (e.g., face-to-face, technology-enabled).
The analysis demonstrated that emotional engagement played a more prominent role than other engagement domains in predicting learner satisfaction with MOOCs. This finding is consistent with Kucuk and Richardson’s [62] research showing that emotional engagement is one of the most important determining factors of student satisfaction in credit-bearing online courses. This study provides concrete evidence that maintaining learners’ emotional engagement is likely to contribute to a positive learning experience, and emotional engagement should be constantly monitored in the MOOC teaching and learning process. Up to now, few attempts have been made to investigate the antecedents of emotional engagement in MOOCs. Pireva, et al. [63] compared learners’ emotional engagement when performing academic tasks on two MOOC platforms. Their study only reported the descriptive statistics and was based on a small sample size (n = 11). Beirne, et al. [64] found that quizzes evoked more positive emotions than other learning tasks; however, they did not report if the difference was statistically significant. In the future, correlational and experimental research should be conducted to explore how design factors affect emotional engagement and how MOOC teaching and learning spaces can be (re)configured to promote emotional engagement.
Assessing emotional engagement can be more difficult than measuring other engagement types because emotional engagement is less observable and cannot be easily inferred based on the analysis of log files [13]. Emotions in MOOCs are often captured through quantitative self-report measures [27] and qualitative grounded theory approaches [41]. These methods are useful in describing discrete emotional states. However, learners’ emotions do not stay the same over the course of a MOOC; rather, they fluctuate at different points in time depending on the course content [65]. The fluctuation of emotions is often ignored in technology-enhanced learning research [66]. An important research direction is evaluating the feasibility of employing ubiquitous technologies, such as smartphones and web cameras, to measure learners’ photoplethysmography (PPG) signals and facial expressions [67] in a nonintrusive way, thereby supplementing self-reported, clickstream, and forum textual data to enhance emotional engagement. Another promising line of research is applying the techniques of natural learning processing and evaluating the feasibility of adopting a virtual assistant to capture emotional engagement throughout a MOOC [68]. The continuous emotional data can be used to provide just-in-time assistance to at-risk and disengaged learners.
Behavioural engagement is often associated with desirable learning outcomes in MOOCs, such as better academic performance and higher completion rates [7]. This study added to the existing literature that behavioural engagement is also linked to learner satisfaction with MOOCs. Recently, a number of empirical studies have explored the antecedents of behavioural engagement. For example, Sanz-Martínez, et al. [69] discovered that the application of homogeneous engagement criteria to group formation predicted a higher rate of task submission. Ortega-Arranz et al. [70] found that reward-based gamification strategies had no significant impact on behavioural engagement measured through the number of page views, task submissions, and time spent on tasks. However, it is unknown if the same strategies promote other types of engagement, such as emotional engagement. The researchers recommend that MOOC scholars operationalise learner engagement as a multidimensional construct and identify pedagogical elements that lead to improvement not just in behavioural engagement, but also in other types of engagement. A promising line of research is to adopt methodological triangulation and combine self-report measures and log files to triangulate findings, thereby enabling a richer and more objective insight into learner engagement in MOOCs and its relationship with other important teaching and learning variables.
This study identified a positive relationship between cognitive engagement and learner satisfaction with MOOCs. Cognitive engagement is not to be confused with a cognitive load. Higher levels of cognitive engagement are often positively associated with learning outcomes [50], whereas heavy cognitive loads are negatively correlated with desirable outcomes [71]. In the MOOC literature, cognitive engagement is an underexplored research area compared to other engagement domains [27]. The broader education literature has shown that learners tend to be less satisfied when a course is either too difficult or too easy and more satisfied when a course is ‘appropriately challenging’ [16]. Identifying the ‘sweet spot’ or ‘just right’ is a big challenge in MOOCs because of the diversity of user profiles [72,73]. An important research direction is to explore instructional design elements that enhance cognitive engagement without imposing excessive cognitive load on MOOC participants.
This analysis also revealed that social engagement was not predictive of learner satisfaction with MOOCs. This finding is consistent with recent research showing that interaction quality had no effect on MOOC satisfaction [9]. However, the nonsignificant result does not imply that social engagement plays a trivial role in the MOOC teaching and learning process. On the contrary, social engagement in MOOCs was found to be predictive of several other desirable outcomes [74]. Empirical research has shown that MOOC learners benefit from social engagement not only socially, but also cognitively [42]. There could be several possible explanations of why social engagement is not predictive of learner satisfaction. It is possible that MOOC learners do not expect the quality of social engagement to be exceptionally high as MOOCs are provided to the public for free. An alternative explanation is that MOOC learners may not expect instructors to frequently avail themselves to a massive number of participants [9]. For these reasons, people may have given less priority to the quality of social engagement when evaluating the performance of MOOCs.
It is also possible that the current affordances of MOOCs constrain learners from socially engaging with peers, instructors, and the broader social community, thereby suppressing the effects of social engagement. Deng, Benckendorff, and Gannaway [27], for example, observed that social engagement in MOOCs was less complex than social engagement in credit-bearing university courses, and MOOC participants did not distinguish learner–learner interactions from learner–instructor interactions. In contrast, university students differentiated between social engagement with peers and instructors [75]. Rather than concluding that social engagement is truly independent of satisfaction, the researchers maintain that it is more appropriate to emphasise the potential of social engagement in determining learner satisfaction with MOOCs. Future research could further examine the effects of social engagement on learner satisfaction in MOOC teaching and learning environments where social engagement opportunities are bolstered by innovative instructional designs and educational technologies.
This study revealed that age, education level, origin, and motivation account for approximately 6% of the variance in learner satisfaction with MOOCs. The results partially confirmed the previous findings of the higher education literature that gender was not related to satisfaction and study motivations were linked to satisfaction [16]. However, the findings of this study contradicted the previous findings indicating that age and education level are independent of course satisfaction [16]. These differences can most likely be explained by the greater variation in age and education level in MOOCs, reinforcing the researchers’ view that MOOC learners should not be treated the same as university students. Age, education level, origin, and motivation are individual characteristics and are not an accurate reflection of the quality of a MOOC. It is important that researchers and practitioners control for these factors when evaluating MOOC performance or investigating the engagement–outcome relationship. Some feasible methods are statistically adjusting satisfaction levels for learner characteristics or making learners aware of their potential biases when evaluating a course [76].
There are some potential limitations of this study and associated considerations for future research. First, this study adopted a convenience sampling approach. Although the demographic composition in this study was similar to the pattern reported in recent MOOC research [22], the participants in this study may not be representative of all MOOC learners. The researchers recommend opportunities for investigating this topic using other sampling methods, such as stratified random sampling. Second, this study defined and operationalised learner satisfaction as a product of the MOOC learning experience. Although it is a convention in MOOC research to treat satisfaction as an outcome indicator [9], it may also be reasonable to define and operationalise learner satisfaction as a process indicator. A promising avenue for future research is to understand how learner satisfaction changes during the study process and the effects of instructional design on changes in satisfaction. Third, this study requested learners to recall the most recent MOOC they had studied in the previous 12 months and complete the survey with that MOOC in mind. Although the survey items were easy to understand, the respondents may not have been able to recall every detail about the MOOC. Future research could overcome this potential limitation by recruiting respondents who had studied a MOOC in the previous six months or by asking learners to report their levels of engagement during the course of a MOOC.

6. Conclusions

The purpose of this study was to explore the influence of engagement on learner satisfaction with MOOCs. This objective was achieved by conceptualising and operationalising learner engagement as a multidimensional construct (behavioural, cognitive, emotional, social), controlling for learners’ demographics (age, gender, education level, origin) and MOOC-related background factors (motivation, prior experience), and regressing learner satisfaction on four discrete engagement dimensions. The results indicated that behavioural, cognitive, and emotional engagement reliably predicted learner satisfaction with MOOCs, accounting for 20% of the variance. However, learner satisfaction was not determined by the level of social engagement. In addition, demographic characteristics and MOOC-related background factors were of limited utility in predicting learner satisfaction with MOOCs, accounting for 4% and 2% of variance, respectively. This study demonstrates that learner engagement provides a unique perspective on learner satisfaction and its contributing factors. Future research could adopt a different theoretical model or research framework, such as Venkatesh et al.’s [77] unified theory of acceptance and use of technology and Moore’s [78] theory of transactional distance, to identify the key factors determining learner satisfaction with MOOCs.
This study made three important theoretical contributions to the body of knowledge. First, this study conceptualised learner engagement as a multidimensional construct comprising four domains when exploring the relationship between engagement and satisfaction. Past research attempting to establish such a relationship often adopted a unidimensional approach and overlooked the fact that engaged learners manifest their engagement in different ways. This study showed that not all engagement domains are linked to satisfaction with MOOCs, thereby contributing to a more nuanced understanding of the engagement–satisfaction nexus. Second, this study demonstrated that engagement explained approximately 20% of the variance in learner satisfaction with MOOCs after controlling for learners’ demographic characteristics and MOOC-related background factors. The final model predicted the change in the outcome variable significantly better than Models 1 and 2, indicating that engagement played a significant role in determining learner satisfaction. Third, this study revealed that emotional engagement was the most important determining factor of learner satisfaction with MOOCs, followed by cognitive and behavioural engagement. The findings highlight the necessity of maintaining emotional engagement in MOOCs and investigating the antecedents of behavioural, cognitive, and emotional engagement.
The results have practical implications for educators, MOOC designers, and higher education leaders and policymakers. First, this study showed that age, education level, origin, and motivation exert an influence (albeit small) on learner satisfaction with MOOCs. These influencing factors are irrelevant to the criteria of good teaching and are often beyond the control of MOOC practitioners. Failure to control for these factors may misrepresent the effectiveness of a MOOC. The findings should not dissuade practitioners from measuring learner satisfaction with MOOCs; rather, they emphasise the necessity of considering personal characteristics when evaluating the performance of any MOOC. MOOC instructors should statistically adjust for age, education level, origin, and motivation when interpreting learner satisfaction results and/or making learners aware of their bias when evaluating the MOOC performance.
Second, the study revealed that emotional engagement contributed to learner satisfaction more than any other engagement dimension. If a MOOC is developed to showcase the teaching talent of a university and attract potential learners for further on-campus study [79,80], learner satisfaction could be a more important outcome indicator for the host institution to monitor than traditional measures such as academic performance. To ensure favourable perceptions of MOOCs, instructors should configure the MOOC teaching and learning environment in a way that bolsters emotional engagement, such as designing the course content, materials, and video lectures so as to generate interest. Instructors should also constantly monitor the level of emotional engagement in a learner cohort and implement educational interventions to provide just-in-time support for emotional disengagers.

Funding

This research was funded by the Cultivation Project of the Provincial-Level Predominant and Characteristic Discipline, Hangzhou Normal University (grant number 20JYXK009).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of The University of Queensland.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author. The data are not publicly available due to participant privacy reasons.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Mittelmeier, J.; Rienties, B.; Tempelaar, D.; Hillaire, G.; Whitelock, D. The influence of internationalised versus local content on online intercultural collaboration in groups: A randomised control trial study in a statistics course. Comput. Educ. 2018, 118, 82–95. [Google Scholar] [CrossRef]
  2. Conijn, R.; Van den Beemt, A.; Cuijpers, P. Predicting student performance in a blended MOOC. J. Comput. Assist. Learn. 2018, 34, 615–628. [Google Scholar] [CrossRef] [Green Version]
  3. Moreno-Marcos, P.M.; Muñoz-Merino, P.J.; Alario-Hoyos, C.; Estévez-Ayres, I.; Delgado Kloos, C. Analysing the predictive power for anticipating assignment grades in a massive open online course. Behav. Inf. Technol. 2018, 37, 1021–1036. [Google Scholar] [CrossRef]
  4. Reparaz, C.; Aznárez-Sanado, M.; Mendoza, G. Self-regulation of learning and MOOC retention. Comput. Hum. Behav. 2020, 111, 106423. [Google Scholar] [CrossRef]
  5. Semenova, T. The role of learners’ motivation in MOOC completion. Open Learn. J. Open Distance e-Learn. 2020, 1–15. [Google Scholar] [CrossRef]
  6. Zhang, Q.; Bonafini, F.C.; Lockee, B.B.; Jablokow, K.W.; Hu, X. Exploring demographics and students’ motivation as predictors of completion of a Massive Open Online Course. Int. Rev. Res. Open Distrib. Learn. 2019, 20, 140–161. [Google Scholar] [CrossRef]
  7. Deng, R.; Benckendorff, P.; Gannaway, D. Progress and new directions for teaching and learning in MOOCs. Comput. Educ. 2019, 129, 48–60. [Google Scholar] [CrossRef]
  8. Jung, E.; Kim, D.; Yoon, M.; Park, S.; Oakley, B. The influence of instructional design on learner control, sense of achievement, and perceived effectiveness in a supersize MOOC course. Comput. Educ. 2019, 128, 377–388. [Google Scholar] [CrossRef]
  9. Hew, K.F.; Hu, X.; Qiao, C.; Tang, Y. What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Comput. Educ. 2020, 145, 103724. [Google Scholar] [CrossRef]
  10. Rabin, E.; Kalman, Y.M.; Kalz, M. An empirical investigation of the antecedents of learner-centered outcome measures in MOOCs. Int. J. Educ. Technol. High. Educ. 2019, 16, 14. [Google Scholar] [CrossRef]
  11. Deng, R.; Benckendorff, P. What are the key themes associated with the positive learning experience in MOOCs? An empirical investigation of learners’ ratings and reviews. Int. J. Educ. Technol. High. Educ. 2021, 18, 9. [Google Scholar] [CrossRef]
  12. Henderikx, M.A.; Kreijns, K.; Kalz, M. Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Educ. 2017, 38, 353–368. [Google Scholar] [CrossRef] [Green Version]
  13. Li, Q.; Baker, R. The different relationships between engagement and outcomes across participant subgroups in Massive Open Online Courses. Comput. Educ. 2018, 127, 41–65. [Google Scholar] [CrossRef]
  14. Zheng, S.; Rossen, M.B.; Shih, P.C.; Carroll, J.M. Understanding student motivation, behaviors, and perceptions in MOOCs. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada, 14–18 March 2015; pp. 1882–1895. [Google Scholar]
  15. Cross, S.; Whitelock, D. Similarity and difference in fee-paying and no-fee learner expectations, interaction and reaction to learning in a massive open online course. Interact. Learn. Environ. 2016, 25, 439–451. [Google Scholar] [CrossRef] [Green Version]
  16. Benton, S.L.; Cashin, W.E. Student ratings of instruction in college and university courses. In Higher Education: Handbook of Theory and Research; Paulsen, M.B., Ed.; Springer: Dordrecht, The Netherlands, 2014; pp. 279–326. [Google Scholar]
  17. Darwin, S. What contemporary work are student ratings actually doing in higher education? Stud. Educ. Eval. 2017, 54, 13–21. [Google Scholar] [CrossRef]
  18. Liu, O.L. Student evaluation of instruction: In the new paradigm of distance education. Res. High. Educ. 2012, 53, 471–486. [Google Scholar] [CrossRef]
  19. Joo, Y.J.; Lim, K.Y.; Kim, J. Locus of control, self-efficacy, and task value as predictors of learning outcome in an online university context. Comput. Educ. 2013, 62, 149–158. [Google Scholar] [CrossRef]
  20. Gannaway, D.; Green, T.; Mertova, P. So how big is big? Investigating the impact of class size on ratings in student evaluation. Assess. Eval. High. Educ. 2018, 43, 175–184. [Google Scholar] [CrossRef]
  21. Khalil, H.; Ebner, M. “How satisfied are you with your MOOC?”—A research study on interaction in huge online courses. In Proceedings of the EdMedia 2013—World Conference on Educational Media and Technology, Victoria, BC, Canada, 24 June 2013; Herrington, J., Couros, A., Irvine, V., Eds.; Association for the Advancement of Computing in Education: Victoria, BC, Canada, 2013; pp. 830–839. [Google Scholar]
  22. Li, K. MOOC learners’ demographics, self-regulated learning strategy, perceived learning and satisfaction: A structural equation modeling approach. Comput. Educ. 2019, 132, 16–30. [Google Scholar] [CrossRef]
  23. McIntyre, D.J.; Copenhaver, R.W.; Byrd, D.M.; Norris, W.R. A study of engaged student behavior within classroom activities during mathematics class. J. Educ. Res. 1983, 77, 55–59. [Google Scholar] [CrossRef]
  24. Goodenow, C. The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychol. Sch. 1993, 30, 79–90. [Google Scholar] [CrossRef]
  25. Finn, J.D. School Engagement and Students at Risk; National Center for Education Statistics: Washington, DC, USA, 1993. [Google Scholar]
  26. Marks, H.M. Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. Am. Educ. Res. J. 2000, 37, 153–184. [Google Scholar] [CrossRef]
  27. Deng, R.; Benckendorff, P.; Gannaway, D. Learner engagement in MOOCs: Scale development and validation. Br. J. Educ. Technol. 2020, 51, 245–262. [Google Scholar] [CrossRef]
  28. Sunar, A.S.; White, S.; Abdullah, N.A.; Davis, H.C. How learners’ interactions sustain engagement: A MOOC case study. IEEE Trans. Learn. Technol. 2017, 10, 475–487. [Google Scholar] [CrossRef] [Green Version]
  29. Jung, Y.; Lee, J. Learning engagement and persistence in Massive Open Online Courses (MOOCS). Comput. Educ. 2018, 122, 9–22. [Google Scholar] [CrossRef]
  30. de Freitas, S.I.; Morgan, J.; Gibson, D. Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. Br. J. Educ. Technol. 2015, 46, 455–471. [Google Scholar] [CrossRef] [Green Version]
  31. Williams, K.M.; Stafford, R.E.; Corliss, S.B.; Reilly, E.D. Examining student characteristics, goals, and engagement in Massive Open Online Courses. Comput. Educ. 2018, 126, 433–442. [Google Scholar] [CrossRef]
  32. Leal, G.; Brenda, E.; González, V.; Ricardo, J. Student engagement as a predictor of xMOOC completion: An analysis from five courses on energy sustainability. Online Learn. 2019, 23, 105–123. [Google Scholar]
  33. Cheung, E. Analyzing Student Engagement and Retention in Georgetown’s First MOOC: Globalization’s Winners and Losers: Challenges for Developed and Developing Countries. Master’s Thesis, Georgetown University, Washington, DC, USA, 2014. [Google Scholar]
  34. Kizilcec, R.F.; Piech, C.; Schneider, E. Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, 8–13 April 2013; ACM: New York, NY, USA, 2013. [Google Scholar]
  35. Perna, L.W.; Ruby, A.; Boruch, R.F.; Wang, N.; Scull, J.; Ahmad, S.; Evans, C. Moving through MOOCs: Understanding the progression of users in massive open online courses. Educ. Res. 2014, 43, 421–432. [Google Scholar] [CrossRef]
  36. Labarthe, H.; Bouchet, F.; Bachelet, R.; Yacef, K. Does a peer recommender foster students’ engagement in MOOCs? In Proceedings of the International Conference on Educational Data Mining, Raleigh, NC, USA, 29 June–2 July 2016; Barnes, T., Chi, M., Feng, M., Eds.; pp. 418–423. Available online: https://hal.archives-ouvertes.fr/hal-01376431 (accessed on 5 October 2021).
  37. Stracke, C.M.; Trisolini, G. A systematic literature review on the quality of MOOCs. Sustainability 2021, 13, 5817. [Google Scholar] [CrossRef]
  38. Maroco, J.; Maroco, A.L.; Campos, J.A.D.B.; Fredricks, J.A. University student’s engagement: Development of the University Student Engagement Inventory (USEI). Psicol. Reflexão E. Crítica. 2016, 29, 1–12. [Google Scholar] [CrossRef] [Green Version]
  39. Daniels, L.M.; Adams, C.; McCaffrey, A. Emotional and social engagement in a Massive Open Online Course: An examination of Dino 101. In Emotions, Technology, and Learning; McCreery, M.P., Ed.; Academic Press: San Diego, CA, USA, 2016; pp. 25–41. [Google Scholar]
  40. Cheng, C.Y. An exploratory study of emotional affordance of a Massive Open Online Course. Eur. J. Open Distance e-Learn. 2014, 17, 43–55. [Google Scholar] [CrossRef] [Green Version]
  41. Comer, D.K.; Baker, R.; Wang, Y. Negativity in Massive Online Open Courses: Impacts on learning and teaching and how instructional teams may be able to address it. InSight: A J. Sch. Teach. 2015, 10, 92–113. [Google Scholar] [CrossRef]
  42. Cohen, A.; Shimony, U.; Nachmias, R.; Soffer, T. Active learners’ characterization in MOOC forums and their generated knowledge. Br. J. Educ. Technol. 2019, 50, 177–198. [Google Scholar]
  43. Lee, Y. Effect of uninterrupted time-on-task on students’ success in Massive Open Online Courses (MOOCs). Comput. Hum. Behav. 2018, 86, 174–180. [Google Scholar] [CrossRef]
  44. Li, Q.; Baker, R. Understanding engagement in MOOCs. In Proceedings of the International Conference on Educational Data Mining, Raleigh, NC, USA, 29 June–2 July 2016; Barnes, T., Chi, M., Feng, M., Eds.; 2016; pp. 605–606. [Google Scholar]
  45. Baker, R.; Xu, D.; Park, J.; Yu, R.; Li, Q.; Cung, B.; Fischer, C.; Rodriguez, F.; Warschauer, M.; Smyth, P. The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: Opening the black box of learning processes. Int. J. Educ. Technol. High. Educ. 2020, 17, 13. [Google Scholar] [CrossRef]
  46. Bulger, M.; Bright, J.; Cobo, C. The real component of virtual learning: Motivations for face-to-face MOOC meetings in developing and industrialised countries. Inf. Commun. Soc. 2015, 18, 1200–1216. [Google Scholar] [CrossRef]
  47. Fredricks, J.A.; McColskey, W. The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Handbook of Research on Student Engagement; Christenson, S.L., Reschly, A.L., Wylie, C., Eds.; Springer: New York, NY, USA, 2012; pp. 319–339. [Google Scholar]
  48. Biggs, J.B.; Tang, C. Teaching for Quality Learning at University, 2nd ed.; Open University Press: Maidenhead, UK, 2003. [Google Scholar]
  49. Stöhr, C.; Stathakarou, N.; Mueller, F.; Nifakos, S.; McGrath, C. Videos as learning objects in MOOCs: A study of specialist and non-specialist participants’ video activity in MOOCs. Br. J. Educ. Technol. 2019, 50, 166–176. [Google Scholar] [CrossRef]
  50. Lei, H.; Cui, Y.; Zhou, W. Relationships between student engagement and academic achievement: A meta-analysis. Soc. Behav. Personal. 2018, 46, 517–528. [Google Scholar] [CrossRef]
  51. Shapiro, H.B.; Lee, C.H.; Wyman Roth, N.E.; Li, K.; Çetinkaya-Rundel, M.; Canelas, D.A. Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Comput. Educ. 2017, 110, 35–50. [Google Scholar] [CrossRef]
  52. González-Gómez, F.; Guardiola, J.; Martín Rodríguez, Ó.; Montero Alonso, M.Á. Gender differences in e-learning satisfaction. Comput. Educ. 2012, 58, 283–290. [Google Scholar] [CrossRef] [Green Version]
  53. Mai, L.-W. A comparative study between UK and US: The student satisfaction in higher education and its influential factors. J. Mark. Manag. 2005, 21, 859–878. [Google Scholar] [CrossRef]
  54. Kennedy, G.; Coffrin, C.; de Barba, P.; Corrin, L. Predicting success: How learners’ prior knowledge, skills and activities predict MOOC performance. In Proceedings of the 5th International Conference on Learning Analytics and Knowledge, New York, NY, USA, 16–20 March 2015; ACM: New York, NY, USA, 2015; pp. 136–140. [Google Scholar]
  55. Chen, Y.; Gao, Q.; Yuan, Q.; Tang, Y. Discovering MOOC learner motivation and its moderating role. Behav. Inf. Technol. 2020, 39, 1257–1275. [Google Scholar] [CrossRef]
  56. Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis: Pearson New International Edition; Pearson: London, UK, 2014. [Google Scholar]
  57. Luik, P.; Suviste, R.; Lepp, M.; Palts, T.; Tõnisson, E.; Säde, M.; Papli, K. What motivates enrolment in programming MOOCs? Br. J. Educ. Technol. 2017, 50, 153–165. [Google Scholar] [CrossRef] [Green Version]
  58. Apodaca, P.; Grad, H. The dimensionality of student ratings of teaching: Integration of uni- and multidimensional models. Stud. High. Educ. 2005, 30, 723–748. [Google Scholar] [CrossRef]
  59. Miles, J. Tolerance and Variance Inflation Factor; John Wiley & Sons: Hoboken, NJ, USA, 2014. [Google Scholar]
  60. Gray, J.A.; DiLoreto, M. The effects of student engagement, student satisfaction, and perceived learning in online learning environments. Int. J. Educ. Leadersh. Prep. 2016, 11, 1–20. [Google Scholar]
  61. Murillo-Zamorano, L.R.; López Sánchez, J.Á.; Godoy-Caballero, A.L. How the flipped classroom affects knowledge, skills, and engagement in higher education: Effects on students’ satisfaction. Comput. Educ. 2019, 141, 103608. [Google Scholar] [CrossRef]
  62. Kucuk, S.; Richardson, J.C. A structural equation model of predictors of online learners’ engagement and satisfaction. Online Learn. 2019, 23, 196–216. [Google Scholar] [CrossRef]
  63. Pireva, K.; Imran, A.S.; Dalipi, F. In User behaviour analysis on LMS and MOOC. In Proceedings of the IEEE Conference on e-Learning, e-Management and e-Services, Melaka, Malaysia, 24–26 August 2015; pp. 21–26. [Google Scholar]
  64. Beirne, E.; Nic Giolla Mhichíl, M.; Mac Lochlainn, C. Curiouser and curiouser: The wonderland of emotion in LMOOCs. In European MOOCs Stakeholders Summit; Springer: Cham, Switzerland, 2019; pp. 13–20. [Google Scholar]
  65. Dillon, J.; Bosch, N.; Chetlur, M.; Wanigasekara, N.; Ambrose, G.A.; Sengupta, B.; D’Mello, S.K. Student emotion, co-occurrence, and dropout in a MOOC context. In Proceedings of the International Conference on Educational Data Mining, Raleigh, NC, USA, 29 June–2 July 2016; pp. 353–357. [Google Scholar]
  66. Henritius, E.; Löfström, E.; Hannula, M.S. University students’ emotions in virtual learning: A review of empirical research in the 21st century. Br. J. Educ. Technol. 2019, 50, 80–100. [Google Scholar] [CrossRef]
  67. Pham, P.; Wang, J. Predicting learners’ emotions in mobile MOOC learning via a multimodal intelligent tutor. In International Conference on Intelligent Tutoring Systems, Montreal, QC, Canada, 11–15 June 2018; Springer: Cham, Switzerland, 2018; pp. 150–159. [Google Scholar]
  68. Lin, J.; Chen, Y.; Hung, C.W. In Enhancing an MOOC platform with emotional sensing and robot interaction features. In Proceedings of the IEEE International Conference on Consumer Electronics, Yilan, China, 20–22 May 2019; pp. 1–2. [Google Scholar]
  69. Sanz-Martínez, L.; Er, E.; Martínez-Monés, A.; Dimitriadis, Y.; Bote-Lorenzo, M.L. Creating collaborative groups in a MOOC: A homogeneous engagement grouping approach. Behav. Inf. Technol. 2019, 38, 1107–1121. [Google Scholar] [CrossRef] [Green Version]
  70. Ortega-Arranz, A.; Bote-Lorenzo, M.L.; Asensio-Pérez, J.I.; Martínez-Monés, A.; Gómez-Sánchez, E.; Dimitriadis, Y. To reward and beyond: Analyzing the effect of reward-based strategies in a MOOC. Comput. Educ. 2019, 142, 103639. [Google Scholar] [CrossRef] [Green Version]
  71. Küçük, S.; Kapakin, S.; Göktaş, Y. Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load. Anat. Sci. Educ. 2016, 9, 411–421. [Google Scholar] [CrossRef]
  72. Beemt, A.v.d.; Buijs, J.; Aalst, W.v.d. Analysing structured learning behaviour in Massive Open Online Courses (MOOCs): An approach based on process mining and clustering. Int. Rev. Res. Open Distrib. Learn. 2018, 19, 37–60. [Google Scholar]
  73. Poellhuber, B.; Roy, N.; Bouchoucha, I. Understanding participant’s behaviour in Massively Open Online Courses. Int. Rev. Res. Open Distrib. Learn. 2019, 20, 221–242. [Google Scholar] [CrossRef] [Green Version]
  74. Barak, M.; Watted, A.; Haick, H. Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Comput. Educ. 2016, 94, 49–60. [Google Scholar] [CrossRef] [Green Version]
  75. Zhoc, K.C.H.; Webster, B.J.; King, R.B.; Li, J.C.H.; Chung, T.S.H. Higher Education Student Engagement Scale (HESES): Development and psychometric evidence. Res. High. Educ. 2019, 60, 219–244. [Google Scholar] [CrossRef]
  76. Peterson, D.A.M.; Biederman, L.A.; Andersen, D.; Ditonto, T.M.; Roe, K. Mitigating gender bias in student evaluations of teaching. PLoS ONE 2019, 14, e0216241. [Google Scholar] [CrossRef] [PubMed]
  77. Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User acceptance of information technology: Toward a unified view. MIS Q. 2003, 27, 425–478. [Google Scholar] [CrossRef] [Green Version]
  78. Moore, M.G. Theory of transactional distance. In Theoretical Principles of Distance Education; Keegan, D., Ed.; Routledge: New York, NY, USA, 1997; pp. 22–38. [Google Scholar]
  79. Fujimoto, T.; Takahama, A.; Ara, Y.; Isshiki, Y.; Nakaya, K.; Yamauchi, Y. Designing a MOOC as an online community to encourage international students to study abroad. Educ. Media Int. 2018, 55, 333–346. [Google Scholar] [CrossRef]
  80. Deng, R.; Benckendorff, P. Technology enabled learning. In Handbook of E-Tourism; Xiang, Z., Fuchs, M., Gretzel, U., Höpken, W., Eds.; Springer: Cham, Switzerland, 2020; pp. 1–27. [Google Scholar]
Sustainability 13 11169 g001 550
Figure 1. Research framework.
Figure 1. Research framework.
Sustainability 13 11169 g001
Table
Table 1. Coding scheme for categorical variables.
Table 1. Coding scheme for categorical variables.
Variables0−1
AgeLess than 25, 25–34, 35–44, 45–54, 55–6465 and over
GenderMaleFemale
Education levelPrimary school, high school, Diploma, Bachelor’s degree, Master’s degree, Doctorate degreeNo schooling completed
OriginArab States, Asia, Europe, Latin America, North AmericaOceania
MotivationPersonal interest, learning for study or work, finding relevant resources, obtaining a certificate of completion, enhancing one’s resume, the MOOC being freeSocialising
Prior MOOC experienceNo experience, 1–3, 4–6, 10 and more7–9
Table
Table 2. Learners’ demographics.
Table 2. Learners’ demographics.
VariablesSubgroupsn%
AgeLess than 2548734.3
25–3449534.9
35–4422615.9
45–541067.5
55–64614.3
65 and over443.1
GenderFemale53037
Male90163
Education
Level		No schooling completed40.3
Primary school141
High school24517.1
Diploma1339.3
Bachelor’s degree57840.4
Master’s degree40828.5
Doctorate degree503.5
OriginAfrica21515.1
Arab States805.6
Asia50435.3
Europe19013.3
Latin America17312.1
North America19713.8
Oceania674.7
Table
Table 3. MOOC-related background factors.
Table 3. MOOC-related background factors.
VariablesSubgroupsn%
Motivation to study the MOOCPersonal interest54538.3
Learning for study or work57340.3
Finding relevant resources372.6
Obtaining a certificate of completion644.5
Enhancing one’s resume14310
MOOC being free513.6
Socialising100.7
Prior MOOC experienceNo prior experience17712.4
1–366046.4
4–630321.3
7–9856
10 and more19813.9
Table
Table 4. Summary of the PCA results.
Table 4. Summary of the PCA results.
MOOC Engagement Scale (MES) ItemsMeanSDFactor 1 (Social Engagement)Factor 2
(Emotional Engagement)		Factor 3 (Cognitive Engagement)Factor 4
(Behavioural Engagement)
I often responded to other learners’ questions.3.391.620.85
I contributed regularly to course discussions.3.511.570.85
I shared learning materials (e.g., notes, multimedia, links) with other classmates in the MOOC.3.021.740.81
I enjoyed watching video lectures in the MOOC.5.061.27 0.73
I found the MOOC interesting.5.340.98 0.80
I was inspired to expand my knowledge in the MOOC.5.231.07 0.78
When I had trouble understanding a concept or an example, I went over it again until I understood it.4.921.17 0.82
I often searched for further information when I encountered something in the MOOC that puzzled me.4.771.27 0.56
If I watched a video lecture that I did not understand at first, I would watch it again to make sure I understood the content.4.951.37 0.82
I set aside a regular time each week to work on the MOOC.4.421.34 0.44
I took notes while studying the MOOC.4.601.49 0.82
I revisited my notes when preparing for MOOC assessment tasks.4.381.57 0.82
Eigenvalue 2.352.041.881.85
% of variance 19.6016.9815.6715.42
Cronbach’s alpha 0.830.730.700.72
Table
Table 5. Descriptive statistics for composite scores.
Table 5. Descriptive statistics for composite scores.
MeanSDMinMax
Behavioural engagement13.403.52018
Cognitive engagement14.633.01018
Emotional engagement15.632.68018
Social engagement9.914.24018
Table
Table 6. Results of multiple regression analyses.
Table 6. Results of multiple regression analyses.
Model 1Model 2Model 3
Coef.SECoef.SECoef.SE
Demographic characteristics
 Age
   Less than 25−0.39 ***0.09−0.3 ***0.09−0.27 **0.08
   25–34−0.23 **0.08−0.19 *0.08−0.18 *0.07
   35–44−0.070.09−0.030.1−0.10.08
   45–540.29 * 0.120.27 *0.120.30 **0.11
   55–640.160.160.120.1600.14
   65 and over (reference group)
 Gender
   Male0.070.040.060.040.030.04
   Female (reference group)
 Education level
   Primary school0.60.340.640.340.430.3
   High school0.43 **0.140.42 **0.140.27 *0.13
   Diploma0.30.160.31 *0.160.180.14
   Bachelor’s degree0.20.130.210.130.170.11
   Master’s degree0.190.140.160.140.130.12
   Doctorate degree0.190.210.150.210.080.18
   No schooling (reference group)
 Origin
   Africa0.120.090.120.09−0.020.08
   Arab States−0.160.14−0.190.14−0.25 *0.12
   Asia0.110.070.110.070.030.07
   Europe−0.020.1−0.040.1−0.020.09
   Latin America0.46 ***0.10.46 ***0.10.41 ***0.09
   North America−0.060.1−0.040.10.060.09
   Oceania (reference group)
MOOC-related background factors
 Motivation
   Personal interest0.070.1−0.020.09
   Learning for study or work0.19 *0.0900.08
   Finding relevant resources−0.250.21−0.350.18
   Obtaining a completion certificate−0.140.17−0.150.15
   Enhancing one’s resume00.13−0.080.11
   MOOC being free−0.55 **0.18−0.43 **0.16
   Socialising (reference group)
 Prior MOOC experience
   No experience−0.050.090.080.08
   1–3−0.13 *0.06−0.070.06
   4–6−0.090.08−0.090.07
   10 and more0.130.090.050.08
   7–9 (reference group)
Learner engagement
 Behavioural engagement 0.03 **0.01
 Cognitive engagement0.06 ***0.01
 Emotional engagement0.17 ***0.02
 Social engagement0.010.01
Constant8.19 ***0.138.16 ***0.154.25 ***0.24
R20.040.060.26
Note: * p < 0.05, ** p < 0.01, *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Deng, R. Emotionally Engaged Learners Are More Satisfied with Online Courses. Sustainability 2021, 13, 11169. https://doi.org/10.3390/su132011169

AMA Style

Deng R. Emotionally Engaged Learners Are More Satisfied with Online Courses. Sustainability. 2021; 13(20):11169. https://doi.org/10.3390/su132011169

Chicago/Turabian Style

Deng, Ruiqi. 2021. "Emotionally Engaged Learners Are More Satisfied with Online Courses" Sustainability 13, no. 20: 11169. https://doi.org/10.3390/su132011169

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop