Satisfaction with remote teaching during the first semester of the COVID-19 crisis: Psychometric properties of a scale for health students

Cristhian Pérez-Villalobos; Juan Ventura-Ventura; Camila Spormann-Romeri; Roberto Melipillán; Catherine Jara-Reyes; Ximena Paredes-Villarroel; Marcos Rojas-Pino; Marjorie Baquedano-Rodríguez; Isidora Castillo-Rabanal; Paula Parra-Ponce; Nancy Bastías-Vega; Débora Alvarado-Figueroa; Olga Matus-Betancourt

doi:10.1371/journal.pone.0250739

Abstract

Introduction

Due to the health crisis caused by the COVID-19 pandemic, 220 million college students in the world had to halt face-to-face teaching and migrate to what has been called Emergency Remote Teaching, using virtual media, but without adequate preparation. The way this has impacted the student body and its satisfaction with the training process is unknown and there are no instruments backed by specific validity and reliability studies for this teaching context. This is why this study aims to analyze the psychometric properties of the Remote Teaching Satisfaction Scale applied to Chilean health sciences students.

Method

Quantitative study by means of surveys. We surveyed 1,006 health careers undergraduates chosen by convenience sampling. They came from six Chilean universities, located over a distance of 3,020 kilometers and followed 7 different careers. Women comprised the 78.53%. They answered the Remote Teaching Satisfaction Scale online to evaluate their perception of the first Emergency Remote Teaching term in 2020.

Results

A descriptive analysis of the items showed a moderate to positive evaluation of the teaching. The Confirmatory Factorial Analysis showed an adequate adjustment of the theoretical four factors model to the data obtained (CFI = 0.959; TLI = 0.953; RMSEA = 0.040). Correlations among factors oscillated from r = 0.21 to r = 0.69. The measurement invariance analysis supported the Configural, Metric and a partial Scalar model. Differences were found in three of the four factors when comparing the first-year students with those of later years. Finally, the Cronbach’s α and McDonald’s ω coefficients were over 0.70.

Discussion

The results display initial psychometric evidence supporting the validity and reliability of the Remote Teaching Satisfaction Scale to assess academic satisfaction in Chilean health careers students. Likewise, it is seen that first-year students show higher satisfaction levels about the implemented teaching.

Citation: Pérez-Villalobos C, Ventura-Ventura J, Spormann-Romeri C, Melipillán R, Jara-Reyes C, Paredes-Villarroel X, et al. (2021) Satisfaction with remote teaching during the first semester of the COVID-19 crisis: Psychometric properties of a scale for health students. PLoS ONE 16(4): e0250739. https://doi.org/10.1371/journal.pone.0250739

Editor: Oathokwa Nkomazana, University of Botswana Faculty of Medicine, BOTSWANA

Received: December 31, 2020; Accepted: April 12, 2021; Published: April 28, 2021

Copyright: © 2021 Pérez-Villalobos et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting information files.

Funding: This research was funded by the CONICYT-FONDECYT 1190931 grant of the Chilean Government.

Competing interests: The authors have declared that no competing interests exist.

Introduction

On March 11, 2020, the World Health Organization stated that COVID-19, a disease derived from the SARS-CoV-2 virus had become a pandemic [1]. Faced with this, most of the world’s countries established quarantine and lockdown measures [2], so that at the beginning of next month, 3.4 thousand million people of over 80 countries were confined [1], with a direct and radical impact on persons’ health, mobility, freedom, work, economy, social life and education, that is still difficult to measure [1, 3].

Effects on higher education

As part of those measures, many countries began to apply total or partial closures of their educational systems and, according to UNESCO statistics [4], the most complex moment was that of 2 April 2020, when over 1,484 million students of all education levels were affected (84.8% of the global total): 172 countries having totally closed their educational systems. In higher education, Bassett indicates that during that same month, 170 countries and communities had closed their higher education institutions, affecting over 220 million college students [3].

Since it was no longer possible to carry out face-to-face traditional teaching activities [5], universities were forced to migrate to remote teaching through online courses [1], even in those countries that had been successful in containing the pandemic such as Vietnam [6].

Emergency remote teaching

When we refer to e-learning, we refer to that where formation activities and the delivery of materials are made on line. This includes classes, homework and assessments [7]. When well designed, it has advantages, as it enables access to education to persons who have difficulties with face-to-face training [8] or who cannot pay transport to the city of the university, or the cost of living in it [9]. But, for online teaching to work well, a well-planned course is required, an adequate preparation of the teachers to dictate it, sufficient institutional support, the creation of learning communities [8] and to guarantee the access and management of the technologic equipment for students and teachers and the connectivity to Internet [10].

Unfortunately, what was obtained after the forced closing of the institutions was not the best face of online teaching. On the contrary, what was implemented has been called Emergency Remote Teaching, characterized as presenting a fast and transitory solution to the impossibility to keep up face-to-face work due to a passing problem, and hoping to return to the prior mode after that. Its purpose is to keep up formation going even if it could deplete quality, mainly because the abrupt change makes it quite probable that the institutional resources capacity, geared to face-to-face work, would be exceeded, including the support personnel for teachers and students, support systems and the control of the teachers themselves on the courses operation. The above, without even considering that, in an emergency context such as the COVID-19 pandemic, it is probable that the community itself (students, teachers, etc.) would be affected by the health emergency itself and would not have education as its immediate priority [11].

Moreover, in the health careers case, the traditional formation structure is heavily grounded in the Flexner model which promotes learning basic knowledge before mastering more complex abilities [12], such as clinics which usually take place at the patients’ bedside training. This structure was affected, since in many cases the students were removed from clinical spaces [13]. There were fewer clinical teachers to carry out teaching, classes continuity was lost [14], and the students also saw their direct contact with patients interrupted [15].

Effects on the teachers

A factor of no smaller importance was the effect that this had on the teachers, who were forced to migrate quickly to this new mode, without many of them having the necessary competences for online teaching or the use of technologies [1, 16]. In fact, according to a study by Watermeyer, the preparation and confidence declared by United Kingdom teachers was very heterogeneous [9].

And this becomes worse when one thinks that an online course requires six to nine months’ preparation [11]. However, in the context of the health crisis, teachers had to have their online courses ready in days or weeks; many fell back to trial and error [2] and other appealed to colleagues and social networks [9, 16]. Although it is to be expected that they will feel more comfortable the next time they give the course, it is also to be expected that the first time will be arduous, stressful and that they will be more centered in maintaining the educational service than in the students’ learning [11]. The stress of having to adapt oneself, not knowing how to do so, having to abandon old practices–even those that had been successful–and the pressure to innovate against time generate additional stressors [17].

Effects on the students

In the students’ case, they, like the teachers, reported to be worried about bad Internet connectivity and the constraints for access to the platforms used during the Emergency Remote Teaching [16, 18]. This was not surprising. Although online teaching can facilitate access to people with scarce available time or geographically remote [9, 11], in practice millions of students and teachers do not have the conditions to actively involve themselves in the online education platforms [10].

In the case of college students, many do not have the infrastructure nor adequate connectivity in their homes, although we believe that they are better prepared for online teaching than primary or high school students, there are no major differences with the access of the latter [3].

To this is added that the crisis victims are not equally distributed in the population [19]. The students of the more vulnerable sectors face greater problems of access to the equipment and necessary connectivity, but also show more difficulties to handle the technology, generating a breach that can paradoxically increase when their university implements new technological teaching resources [10].

And finally, since all activities take place at home, online teaching can interfere with the students’ private life [20].

Satisfaction with teaching

In spite of these adverse conditions, available studies to date document that the students report to be basically satisfied with the Emergency Remote Teaching implemented to face the COVID-19 crisis. In countries such as Vietnam [6] and Indonesia [21], students have shown more preference for this methodology than with face-to-face classes, especially in the case of first-year students. This could be due to the fact their courses are mainly theoretical and adapt better to remote teaching than higher courses [21]. While in China [22] and Poland [23] evaluation has been basically positive, although Polish students would prefer to return to face-to-face teaching.

However, some typical problems of this type of teaching are identified, such as difficulties to motivate the students, less interaction with the teachers, deficient teaching abilities, unsteadiness of the platforms [22], less social contact, loss of opportunities to train clinical abilities [5] and the difficulties of the Internet connection [21]. Besides, some medical students report fears about the pandemic affecting their specialty choice, their exploration of options or their access to letters of recommendation [24].

Purpose of this study

To date, there are no studies about the satisfaction of college students in Latin-American, nor in Chile, a country where the first academic term normally takes place between the months of March and July and where most of the universities did not have even one week of face-to-face classes during 2020 before they closed and were forced to migrate to Emergency Remote Teaching.

Likewise, although there are instruments to measure academic satisfaction, whose validity and reliability evidence has been previously assessed in Chilean university students [25, 26], none is specifically adapted to online teaching or aspects proper to Emergency Remote Teaching.

For this reason, this study proposes a new questionnaire created by the authors of this research: the Remote Teaching Satisfaction Scale, which seeks to assess the students’ experiences in this atypical formation scenario due to the COVID-19 health crisis, from the point of view of their satisfaction with training process.

Satisfaction has been chosen as variable, because it allows to carry out an assessment of the reaction of the participants in a formation experience and is the first level to be assessed before learnings, behavior changes and results achieved [27, 28]. Although it does not replace them and must be complemented with assessments of those other elements, it is a common basis of the formation activities assessments, as it permits to explore the educational experience [29, 30] and its adequate management allows to generate a closer relationship with the students [31].

This study aims to answer the following research questions: There is evidence to support the Remote Teaching Satisfaction Scale’s valid use to assess academic satisfaction in Chilean health sciences students in the COVID-19 crisis? Are its measures reliable? And, is there measurement invariance between this scale assessment for first-year and higher year’s students?

In order to solve these questions, the research objective is to analyze the psychometric properties of the Remote Teaching Satisfaction Scale applied to Chilean health sciences students, evaluating its factorial structure as initial evidence of its construct validity, its reliability and the measurement invariance between first-years’ students and higher years’ students. We decided to compare these groups because in Chile the first group has known the university only through its computer screens as Emergency Remote Teaching activities had already started before they entered. We performed a comparison of satisfaction between both groups after the assessment of the invariance measurement.

Method

We carried out a quantitative, non-experimental, transversal, analytic, and psychometric study.

Participants

Six Chilean universities participated in this study, permitting a representative geographical coverage of the country: there is a straight-line distance of 3,020 kilometers between the northernmost and southernmost participating universities. Also, these universities have diverse economic students’ profiles, enrolling between 0.7% and 32.6% of students from private secondary schools, where the country’s wealthiest population studies [32].

We obtained a sample of 1,006 health careers students using a non-probabilistic convenience sampling. Of these, 78.53% were women, a percentage similar to the 73.01% of the Chilean students who started health undergraduate studies in 2020 and were women [33]. Their average age was 21.11 years (SD = 3.53), and they had an average stay in the university of 2.99 years (SD = 1.92). The students came from seven different undergraduate programs of the six universities, identified in Table 1 by number only to preserve results privacy.

Download:

Table 1. Description of the sampled students.

https://doi.org/10.1371/journal.pone.0250739.t001

Data collection

The students answered the Remote Teaching Satisfaction Scale, prepared by the research group as a synthesis of previous experiences such as the Questionário de Satisfação Académica (QSA, Academic Satisfaction Questionnaire) of Soares and Almeida, translated and adapted to the Chilean population [25], the Curriculum Evaluation Scale developed by Glaría et al. [34] and the Lent Academic Satisfaction Scale, validated in Chile by Vergara-Morales et al. [26]. All these instruments were validated in Chile. However, they center on the formation process conditions [25, 34] or students’ well-being [26], but not in their contribution to the students’ learning. For this reason, the Remote Teaching Satisfaction Scale is centered on the impact attributed by the students to the formation conditions on the learnings achieved. It comprises 22 items (S1 and S2 Tables), representing affirmations on the impact of different conditions of online teaching on learning, involving four factors:

Contribution to learning: Based on the unidimensional satisfaction scales model previously validated in university students [25, 26], the perception of teaching, evaluation and planning of teaching was incorporated as a single factor, centering on the degree in which the student perceived that these activities had favored his learning. Elements proper to the Emergency Remote Teaching were incorporated into its wording.
Organized relationship: Considering that uncertainty has characterized the present health crisis and that many students report being worried about the lack of certainties and information [5], the scale incorporated that factor, assessing the degree in which the student perceives that the teacher established a respectful, transparent and organized relationship with the student body. It was expressed as the conceptual opposite of the unregulated exigencies, identified as a kind of mistreatment during health training, which therefore increase uncertainty and discomfort information [35, 36].
Context diagnosis: This was focused on the assessment of the students’ perception about the actions of the teacher to obtain information about the personal situation of the student and the access to the means and technologies to participate actively in remote teaching, it being understood that the access to them and their mastering are aspects unevenly distributed in the population and may affect the students’ educational results [10].
Students involvement: It is understood that, from a constructivist viewpoint, the students must be the center of good teaching [37] actively involving the students [38], it is necessary to incorporate measurements to allow that the students themselves should evaluate how involved they are in this process. This is necessary to go beyond the focus of academic satisfaction as a mere evaluation of the service received [26].

The instrument offered seven answer alternatives in Likert format, from: 0 = Fully disagree to 6 = Fully agree.

Once the document had been prepared and before its application, we submitted it for a judgment of experts: 12 specialists in medical teaching, higher education, educational psychology, and psychometrics evaluated the content validity of its items. Next, we piloted a previous version of the scale in a convenience sample of 21 health science students from one of the participating universities who responded to a retrospective cognitive interview after it. We used those outcomes to make up the scale’s final version.

Procedure

First, we obtained institutional approval of the six participant universities. Later on, the authorities of each of them were asked to virtually distribute the invitation to answer the anonymous online survey directly to the institutional e-mails of the students. Once the students received the e-mail, they were asked to voluntarily answer the survey by means of a hyperlink which led them directly to an informed consent form. Only those who consented went on to answer the survey. Those who refused ended the procedure at once.

The survey was applied between the third and fourth months of the first academic semester of 2020.

Ethics

The Ethical Science Committee of the Faculty of Medicine of Universidad de Concepción, Chile, approved this study as the institution responsible for the project (No. CEC 10 2020). The other universities accepted this approval.

Data analysis

First, a descriptive analysis of the Remote Teaching Satisfaction Scale items was performed.

In order to evaluate the factor structure of this scale, we used a confirmatory factor analysis (CFA). Robust maximum likelihood was used as the estimation method since Mardia’s kurtosis coefficient was 676.72.

The quality of the model fit was evaluated by using the following statistics: a) χ², b) CFI, c) TLI, d) RMSEA, and a 90% confidence interval. A non-significant statistic χ², values of 0.95 or higher for CFI and TLI, and a value of less than 0.08 for RMSEA [39, 40] were proposed as good fit criteria for the model.

To evaluate measurement invariance (MI) between first-year and higher level students, we analyzed a multi-group Confirmatory Factor Analysis sequence. Specifically, we compared three progressively restrictive models: Configural, Metric, and Scalar. For the Configural model, we specified the same pattern of free and fixed factor loadings for each group. We did not impose equality constraints on the intercepts, factor loadings, and residual variances across samples. For the Metric model, we constrained all factor loadings to be equal across groups. If factor loadings are not reasonably invariant over the groups, any comparison must be considered suspect because the latent variables themselves are not equivalent [41]. For the Scalar model, we constrained both intercepts and loadings to be equal across the two groups. A lack of invariance of item intercepts would suggest that differences over the groups cannot be only explained in terms of differences at the latent variable mean levels [41]. We should obtain Scalar invariance to ascertain that observed scores are the same across groups for identical factor scores.

To compare the fit for two nested models representing different levels in the sequence of measurement invariance, we used the Robust Likelihood Ratio Test, LRT (ΔMLR χ²). One important distinction between the Maximum Likelihood (ML) and MLR methods is that the traditional approach only takes into account of the difference in χ² values (ΔML χ²). Moreover, the degrees of freedom cannot be employed for nested models using MLR because this χ² difference has no χ² distribution. Instead, it is necessary to employ a rescaled χ²-test (ΔMLR χ²) proposed by Yuan and Bentler [42]. A non-significant result for the ΔMLR χ² test can be taken as evidence of measurement invariance. That is, the decrease in model fit associated with the more restricted model is not statistically significant.

As a second test of the measurement invariance, we analyzed the changes in the CFI and RMSEA indices. Following Chen’s recommendations for comparing two nested models, we used cut-off values of ΔCFI < 0.01 and ΔRMSEA < 0.015 for testing metric invariance and scalar invariance [43].

Finally, we evaluated the reliability of scales using Cronbach’s α. Additionally, we computed the McDonald’s ω reliability coefficient as it offers a more unbiased estimation of reliability [44].

All data analyses were conducted using Mplus 8.4.

Results

Descriptive data

Table 2 shows the descriptive analysis by item of the Remote Teaching Satisfaction Scale. To this end, the median was calculated, and a frequency analysis of each item was performed. To make reading of the latter easier–only for this analysis–the options were grouped as follows: 0 (Fully disagree), 1 (Mostly disagree) and 2 Disagree) within the “Disagree” category, the options 4 (Mostly Agree), 5 (Agree) and 6 (fully agree) within the “Agree” category, and option 3 (Neither agree nor disagree) in the “Neutral” category. Results show that all items have a median of 4.0 or over, this being the possible mean number of the scale, showing an assessment of the indicators that goes from moderate to positive. Individually, the better evaluated aspects were those related to the student’s involvement, where over 90% agreed that they had endeavored to understand the contents (item 18) and had fulfilled the tasks in due time (item 21). On the contrary, 12 was the worst evaluated item, where 45.8% of the students did not consider that the teachers asked about their personal situation, followed by 32.5% of students that did not believe that the times assigned for the activities were sufficient, a 30.3% of students disagree about the courses using diverse activities to achieve learning (item 3) and 30.0% did not believe that they were learning to apply course contents in an autonomous manner (item 1).

Download:

Table 2. Descriptive analysis of the Remote Teaching Satisfaction Scale Items.

https://doi.org/10.1371/journal.pone.0250739.t002

Construct validity: Confirmatory factor analysis

Then, a CFA was performed in order to evaluate the four-factor model proposed from authors for the 22 items. Model fit results corresponded to χ² (203) = 521.66, p < 0.001, CFI = 0.959, TLI = 0.953, RMSEA = 0.040 (90% CI: 0.035–0.044). As for the factor loadings, the absolute values in the range 0.583 to 0.918 were observed and all were statistically significant (p < 0.001). The correlations between the factors range from 0.22 to 0.72, Table 3. Taken together, these results provided satisfactory evidence regarding the adequacy of the four-factor model. The estimated parameters for this model are shown in Fig 1.

Download:

Fig 1. Confirmatory factor analysis of the Remote Teaching Satisfaction Scale.

https://doi.org/10.1371/journal.pone.0250739.g001

Download:

Table 3. Correlations between factors of Remote Teaching Satisfaction Scale.

https://doi.org/10.1371/journal.pone.0250739.t003

Measurement invariance

Once the factor structure for the Remote Teaching Satisfaction Scale was established, an analysis of the measurement invariance (MI) by student cohort was performed.

We first specified a four-factor configural model for freshmen and higher year students. The model fit indices of this model are shown in Table 4.

Download:

Table 4. Fit index of invariance models between first-year students and higher year students.

https://doi.org/10.1371/journal.pone.0250739.t004

In the next step, we estimated the metric model and compared it with the configural model using the Likelihood Ratio Test (LRT). This procedure estimates the differences in model fit between two nested models, where model fit indices are based on the robust χ². Results indicated support for metric invariance, indicating all factor loadings were invariant across the two groups.

Then, we estimated the scalar model and compared it with the metric model using the LRT. Results of this comparison showed the lack of scalar invariance, suggesting not all of the 22 intercepts tested were invariant across the two groups. We then identified a partial scalar invariant model. Based on the intercepts of the metric model and model indices of the scalar model, intercepts for items 1, 2, 3, 4, 5, 6, 13, 14, 15, 16, 17 and 18 were allowed to vary across the groups. Comparing this partial scalar model with the metric model showed partial scalar invariance. This suggests that intercepts for 8 of the 22 items were invariant between first-year and older students.

Comparing factor means across cohorts

Finally, we compared the factor means across cohort student groups. The results indicated that the groups showed statistically significant differences for three of the four factors. Specifically, it was observed that first-year students showed higher factor means for factor 1, 3 and 4 compared to higher level students, Table 5.

Download:

Table 5. Factor means comparison between first-year and higher level students.

https://doi.org/10.1371/journal.pone.0250739.t005

Reliability

Finally, reliability Cronbach’ α and the McDonald’ ω coefficients are shown in Table 6. Both coefficients obtained similar results for each of the four factors, all above 0.73.

Download:

Table 6. Reliability of Remote Teaching Satisfaction Scale.

https://doi.org/10.1371/journal.pone.0250739.t006

Discussion

Probably very few of us in 2019 would have expected to live a 2020 year such as this, in the midst of a global health crisis that has meant a severe interruption of our daily life, with impacts on different aspects of our life and implications still difficult to foresee [1, 3].

For this reason, a constant scientific examination of our situation during 2020 is a basic and indispensable tool to make decisions, adapt to changes and prepare for what is to come, above all in the education context which has been marked by uncertainty and lack of information during this year [5].

In the health professionals’ training, we know the restrictions that this has mean for teaching, but still know little of its effects, although there already are studies about this aspect in several countries of Europe [5, 13, 23, 45, 46], North America [24] and Asia [14, 18, 21, 47], there are no evidences in this respect in Latin-America. Nor is there psychometric evidence backing up the instruments that have been employed up to date, or their suitability for an atypical formation context, such as the Emergency Remote Teaching [11].

In this study, it has been decided to convey initial evidence, choosing the academic satisfaction construct, which allows to assess the subjective experience surrounding the formation process, it being understood that this evidence must be complemented by further evaluations of the achieved learning, behavior changes, results obtained [29], and incorporate objective elements [28]. However, an adequate measurement of satisfaction that includes several criteria which should be part of a formation process evaluation must be reliable, valid, acceptable and inexpensive [28].

When analyzing each item, it is possible to find relevant evidence of this first semester functioning, where the perception of the students fluctuates between moderate and positive. This agrees with studies of countries such as Indonesia [21], Vietnam [6], China [22] and Poland [23]. In spite of the above, there are critical points. Almost half the students do not believe that their teacher has asked about their personal situation, while only a third considers that he did so. This factor is complex in a pandemic situation, where face-to-face formation activities are suspended and access to online teaching means are heterogeneous, with students having problems of access to equipment and Internet connection and others with difficulties to manage new technologies [10]. This, without mentioning that the life situation of the students was affected by the social, economic and civil upheavals of the health crisis–as everyone was. Therefore, this crisis probably made it all the more relevant that teachers should make explicit efforts to know their students’ conditions. A factor that has always been important in teaching. But it is usual to think of teaching centered more in the diagnosis of aspects as prior knowledge, learning needs, and progress of the students [48, 49], than the conditions for learning.

The analysis by items revealed another factor receiving a deficient evaluation, the adequacy of time to carry out formation activities, the learning activities diversity and the achievement of learning that would allow an autonomous application of the contents. This reveals teaching difficulties which could have appeared even before the health crisis and are an evidence of historical deficiencies in university teaching. In effect, the time assignations and variety of methodologies are two aspects to be considered in teaching scheduling, and although Chilean academicians are aware that this is the tasks that takes them more time, some studies show that they tend to orient themselves less towards student centered teaching [50, 51]. Even if the teachers recognize that–after assuming the role of student–to experiment different ways to teach helps them to learn better, when teaching the students, they tend to resort to expository strategies [52].

However–beyond the analysis by items–this study provides evidence that supports its validity and reliability to learn to what degree the scores of the Remote Teaching Satisfaction Scale deliver an adequate measurement.

Validity refers to the quality of evidence and theory to support a test measurement interpretation. One source of evidence that can support a questionnaire construct validity is the appraisal of its internal structure, assessing how the relations among their items coincide with the instrument conceptual proposal [53].

This study, starts from the premise that an adequate formation process must contribute to learning (Contribution to learning), be based on a respectful and organized relationship with the teacher (Organized relationship), have a diagnosis of the student’s conditions to accede to the learning experiences (Context diagnosis) and an active participation of the involved students (Students’ involvement). The confirming factorial analysis empirically supported this conceptual proposal of four factors, bringing evidence of the validity of its structure [53].

Firstly, the instrument allows to differentiate the contribution to learning of the formative processes which have forcibly migrated to the Emergency Remote Teaching, abandoning traditional practices both within the university campus [1, 5], due to similar constraints in all areas of knowledge, as well as the clinical contexts [13–15]. To know if the students feel that they are learning in this forced transition context, is a keystone to identify how successful have been the measures taken by the universities.

A second factor refers to the Organized relationship, that is the degree in which the teacher established a transparent and organized and therefore expectable relationship with the student. It is a central aspect, considering that it has been widely documented that the health career students–especially medical students–have been historically submitted to academic mistreatment situations, where the most frequent source is the teachers themselves. One of its manifestations is to demand over-dimensioned academic tasks that surpass the student’s available resources and its formation-level [35, 54, 55]. These sources of mistreatment generate stress [56, 57] and affect the well-being of the student [58, 59]. This is especially complex in a context where the students lack certainties about how the formation processes will continue and at what time they will be able to complete them [5]. In this sense, a respectful and planned relationship lessens uncertainty of the formation processes, something that is valuable for the students’ well-being, even out of the pandemic context.

The third factor assesses Context diagnosis, an aspect that Shulman already highlighted as a key factor of the pedagogical content knowledge that every teacher should have [60]. In effect, good teaching, besides the subject taught and pedagogic strategies requires that the teacher must be aware of the context in which it takes place. This requires that teachers should pay attention to the type of technology that students have at home, the spaces they have in which to study and their connectivity [10], aspects that also worry the students [18] and which can affect their actual access to learning opportunities. This diagnostic must be especially painstaking for the students of more vulnerable sectors, who usually have more difficulties when institutions introduce new technologies [10], so that socio-economic differences should be used as control in future research that would apply this scale.

And the last factor, students’ involvement, does not usually appear in the instruments used to assess academic satisfaction [25, 26]. However, since education is not a product to be delivered to the student, but an opportunity in which he must be actively involved to achieve the expected learnings [37, 38], this study starts based on the premise that a teaching satisfaction scale should also foster the self-evaluation of the students’ involvement.

When evaluating correlations between factors, all showed significant correlations, which is to be expected under the assumption that they all constitute indicators of an adequate educational process. But they are all under 0.70, which shows that these factors are different from one another.

Additionally, the correlations allow us to obtain information about the behavior of the construct, evidencing that the contribution to learning showed moderate to intense correlations with the other three factors, the most intense being with organized relationship. In this sense, both this last factor as well as the contribution to learning, have as basis an adequate planning of teaching. And, to the contrary, the less intense was found between the students’ involvement and context diagnosis, which could be attributed that the latter should be used as an input for teaching activities (contained in the factors Contribution to Learning and Organized Relationship) and only then can it affect the degree of participation achieved by the student.

When assessing the invariance of the measurement between students of the first-year and upper levels, it was found that the full scalar model was not supported. Due to this result, any comparisons between the two groups of students, not accounting for the non-invariance in the intercepts as mentioned above, could be biased. These results suggest the importance of performing measurement invariance tests before conducting mean comparisons across groups [61].

However, in this study the comparison of both groups was made after the MI test finding that first-year students showed higher satisfaction with the contribution to learning, students’ involvement and organized relationship in comparison with upper levels. This coincides with a previous study carried out in Indonesia, where first-year students preferred online teaching. This could be explained because the contents of first-year courses have a more theoretical emphasis, which helps their online implementation, in contrast with the more practical or integrated learnings of upper courses [21]. However, in the Chilean case, this could also be due to the fulfilling of expectations: in the case of the upper levels students, they already had university experience and expectations about the type of formation that they would receive. Furthermore, these students may resent to a greater extent the reduced opportunities to develop clinical abilities [5]. However, in Chile, the first-year students of the universities participating in the study know the emergency remote university only, so they have their school experience alone for comparison where university formats such as laboratory work or clinical formation are not present. Their present situation makes them less pressured to develop clinical abilities than older students, and they have less antecedents to critically assess the acquisition of specific abilities, such as laboratory skills.

The absence of differences in the Context diagnosis factor may be due to the fact that this activity, carried out by the teacher, implied to know the conditions of the students to accede to an online remote teaching format much more heterogeneous that the remote teaching to which each formation level was used to. In this sense, the strategies and the teacher’s degree of care to know the students’ context did not have external factors to generate variation.

Finally, the instrument showed adequate reliability coefficients, both according to Cronbach’s α which has usually been used to assess reliability, as well as McDonald’ ω which would generate less biased estimations [44]. This shows an adequate precision of the delivered measurements [62].

Limitations

This research shows some limitations that we must consider. In the first place, although the sample represents six different universities with relevant geographic coverage of the country, we obtained them with a non-probabilistic sampling which can affect the representativity of the obtained results. In effect, the students with less access to the Internet and technologies, who can be more affected by the transition, and therefore less satisfied, may not have had access to the survey or fewer opportunities and willingness to answer it. In this sense, it represents a potential selection bias that makes it necessary to carry out future studies to deal specifically with the students’ lifestyle with less access to technologies and the adaptation experiences of those from more vulnerable sectors, ideally from a qualitative perspective.

On the other hand, we carried out this study during the first academic term of 2020, when initial efforts to migrate to the Emergency Remote Teaching had just started. This makes the study results especially valuable, as they document the first adaptation steps to this new teaching and learning way. It makes it desirable to carry out new research to assess if the student’s satisfaction improves over time. Also, to research if, with more planning time and better mastering of the technologies, the universities’ adaptation processes and the teachers become more successful than during the abrupt initial transition.

And a last limitation refers to the results, since no invariance was found in the measurements, which shows that some items are not behaving exactly in the same way in both groups. It would be important to examine this using a qualitative approach to understand what is the reason why these items are interpreted and are answered differently between the first-year students and those of higher years. This could be related to the first-year students’ participation, where they represented almost 40% of the sample with higher level students underrepresented. This was a consequence of the non-probabilistic sampling strategy we used and needs to be corrected in further studies.

Conclusions

The results show initial psychometric evidence supporting the validity and reliability of the Remote Teaching Satisfaction Scale measures to assess the academic satisfaction of the Chilean health career students.

It also shows that the four academic satisfaction factors assessed, although different, show direct correlations among themselves, and that first-year students reported to be more satisfied with the Emergency Remote Teaching of the first term of 2020, considering its contribution to learning, the relationship with the teacher and their own involvement.

These results, although thought from the viewpoint, and in order to understand the COVID-19 health crisis effects, must not be understood as an experience restricted to it. There is consensus that this health crisis will affect life after the pandemic, will imply radical changes and people will have to reinvent themselves [1, 3].

It may be that Emergency Remote Teaching could seem to us a unique and passing experience, but this is not the first time that changes of this kind must be implemented–at least temporarily–as shown by the experiences of countries such as Afghanistan or South Africa [11]. Likewise, it is highly probable that educational institutions might face the need to interrupt their face-to-face teaching in future due to new social, climate emergencies, etc. [11], therefore disaster planning must be incorporated to the strategic plans of university organizations [19].

In this sense, although the Remote Teaching Satisfaction Scale shows adequate functioning conditions to carry out diagnostics during the present crisis, it is a tool that can be useful to assess students’ satisfaction in any remote teaching experience–not necessarily emergencies–and can be viewed as an instrument to be employed in future crises.

Supporting information

S1 Table. Escala de satisfacción con la docencia a distancia (versión original en Español).

https://doi.org/10.1371/journal.pone.0250739.s001

(PDF)

S2 Table. Remote Teaching Satisfaction Scale (proposed English version).

https://doi.org/10.1371/journal.pone.0250739.s002

(PDF)

Acknowledgments

This research was possible by the support in the data collection process of Alejandra Ricouz, Ruben Inostroza, Lorena Mujica, Wilfredo Leyton, and Gloria Villarroel.

References

1. Marinoni G, van’t Land H. The Impact of the Coronavirus on Global Higher Education. International Higher Education. 2020; 102: 7–9.
- View Article
- Google Scholar
2. Alawamleh M. COVID-19 and higher education. Journal of Economic and Economic Education Research 2020; 21(2): 1–3.
- View Article
- Google Scholar
3. Bassett RM. Sustaining the Values of Tertiary Education during the COVID-19 Crisis. International Higher Education. 2020; 102: 5–7.
- View Article
- Google Scholar
4. United Nations Educational, Scientific, and Cultural Organization, UNESCO [Internet]. United States: UNESCO; c2019-2020 [cited 2020 Nov 24]. COVID-19. Education: From disruption to recovery; [about 3 screens]. https://en.unesco.org/covid19/educationresponse.
5. Loda T, Löffler T, Erschens R, Zipfel S, Herrmann-Werner A. Medical education in times of COVID-19: German students’ expectations—A cross-sectional study. PLOS ONE. 2020; 15(11): e0241660. pmid:33206678
- View Article
- PubMed/NCBI
- Google Scholar
6. Dinh LP, Nguyen T. Pandemic, social distancing, and social work education: students’ satisfaction with online education in Vietnam. Social Work Education. 2020; 00(00), 1–10.
- View Article
- Google Scholar
7. Asarta CJ, Schmidt JR. The effects of online and blended experience on outcomes in a blended learning environment. The Internet and Higher Education. 2020; 44: 1–10. 100708.
- View Article
- Google Scholar
8. Sun A, Chen X. Online education and its effective practice: A research review. Journal of Information Technology Education: Research. 2016; 15: 157–190.
- View Article
- Google Scholar
9. Watermeyer R, Crick T, Knight C, Goodall J. COVID-19 and digital disruption in UK universities: afflictions and affordances of emergency online migration. High Educ. 2020; 78. pmid:32836334
- View Article
- PubMed/NCBI
- Google Scholar
10. Aguilar SJ. Guidelines and tools for promoting digital equity. Information and Learning Science. 2020; 121(5–6): 285–299.
- View Article
- Google Scholar
11. Hodges C, Moore S, Lockee B, Trust T, Bond A. The Difference Between Emergency Remote Teaching and Online Learning. Educause Review. 2020; 27: 1–15. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning
- View Article
- Google Scholar
12. Finnerty E, Chauvin S, Bonaminio G, Andrews M, Carroll R, Pangaro L. Flexner Revisited: The Role and Value of the Basic Sciences in Medical Education, Acad Med. 2010; 85 (2) 349–355. pmid:20107367
- View Article
- PubMed/NCBI
- Google Scholar
13. De Ponti R, Marazzato J, Maresca AM, Rovera F, Carcano G, Ferrario MM. Pre-graduation medical training including virtual reality during COVID-19 pandemic: a report on students’ perception. BMC Med Edu. 2020; 20(1): 1–7. pmid:32977781
- View Article
- PubMed/NCBI
- Google Scholar
14. Cleland J, Tan ECP, Tham KY, Low-Beer N. How Covid-19 opened up questions of sociomateriality in healthcare education. Adv Health Sci Educ. 2020; 25(2): 479–482. pmid:32378152
- View Article
- PubMed/NCBI
- Google Scholar
15. Soled D, Goel S, Barry D, Erfani P, Joseph N, Kochis M, et al. Medical Student Mobilization during a Crisis: Lessons from a COVID-19 Medical Student Response Team. Acad Med. 2020; 95(9): 1384–1387. pmid:32282373
- View Article
- PubMed/NCBI
- Google Scholar
16. Trust T, Whalen J. Should Teachers Be Trained in Emergency Remote Teaching? Lessons Learned from the COVID-19 Pandemic. Journal of Technology and Teacher Education. 2020; 28(2): 189–199.
- View Article
- Google Scholar
17. Chemi T. It Is Impossible: The Teacher’s Creative Response to the Covid-19 Emergency and Digitalized Teaching Strategies. Qual Inq. 2020; 1–8.
- View Article
- Google Scholar
18. Chandrasinghe P, Siriwardana RC, Kumarage SK, Munasinghe BNL, Weerasuriya A, Tillakaratne S et al. A novel structure for online surgical undergraduate teaching during the Covid-19 pandemic. BMC Med Educ. 2020; 20(324): 1–7. pmid:32962691
- View Article
- PubMed/NCBI
- Google Scholar
19. Sklar DP. COVID-19: Lessons from the disaster that can improve health professions education. Acad Med. 2020; 95(11): 1631–1633. pmid:32544103
- View Article
- PubMed/NCBI
- Google Scholar
20. Manca S. Snapping, pinning, liking or texting: Investigating social media in higher education beyond Facebook. The Internet and Higher Education. 2020; 44. 100707.
- View Article
- Google Scholar
21. Amir LR, Tanti I, Maharani DA, Julia V, Sulijaya B, Puspitawati R. Student perspective of classroom and distance learning during COVID-19 pandemic in the undergraduate dental study program Universitas Indonesia. BMC Med Educ. 2020; 20: 392. pmid:33121488
- View Article
- PubMed/NCBI
- Google Scholar
22. Wang K, Zhang L, Ye L. A nationwide survey of online teaching strategies in dental education in China. J. Dent. Educ. 2020; 1–7. pmid:32954532
- View Article
- PubMed/NCBI
- Google Scholar
23. Rizun M, Strzelecki A. Students’ acceptance of the covid-19 impact on shifting higher education to distance learning in Poland. Int. J. Environ. Res. Public Health. 2020; 17(6468): 1–19. pmid:32899478
- View Article
- PubMed/NCBI
- Google Scholar
24. Byrnes YM, Civantos AM, Go BG, McWilliams TL, Rajasekaran K. Effect of the COVID-19 pandemic on medical student career perceptions: a national survey study. Med. Educ. Online. 2020; 25(1): 1798088. pmid:32706306
- View Article
- PubMed/NCBI
- Google Scholar
25. Inzunza B, Ortiz L, Pérez C, Torres G, McColl P, Meyer A et al. [Factorial Structure and Reliability of the Academic Satisfaction Questionnaire on Chilean Medical Students]. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2015; 40(1): 73–82.
- View Article
- Google Scholar
26. Vergara-Morales J, Del Valle M, Díaz A, Pérez MV. [Adaptation of the Academic Satisfaction Scale in Chilean university students]. Psicología Educativa. 2018; 24: 99–106.
- View Article
- Google Scholar
27. Kirkpatrick DL, Kirkpatrick JD. The transfer of learning to behavior: Using the four levels to improve performance. California, United States: Berrett-Koehler Publication. 2005.
28. Abdulghani HM, Shaik SA, Khamis N, Al-Drees AA, Irshad M, Khalil MS et al. Research methodology workshops evaluation using the Kirkpatrick’s model: translating theory into practice. Med Teach. 2014; 36(sup1): S24–S29. pmid:24617780
- View Article
- PubMed/NCBI
- Google Scholar
29. Aluko FR, Shonubi OK. Going beyond Kirkpatrick’s Training Evaluation Model: The role of workplace factors in distance learning transfer. Africa Education Review. 2014; 11(4): 638–657.
- View Article
- Google Scholar
30. Sutarto S, Usman H, Jaedun A. Impact-Based Training Evaluation Model (IBTEM) for School Supervisors in Indonesia. International Education Studies. 2016; 9 (2), 149–165.
- View Article
- Google Scholar
31. Torres E, Araya-Castillo L. [Constructing a Scale to Measure the Quality of University Services: Application to the Chilean Context]. Revista de Ciencias Sociales 2010; 16(1): 54–67.
- View Article
- Google Scholar
32. CNED.cl [Internet]. Santiago, Chile: [National Conseil of Education of Chile] Consejo Nacional de Educación de Chile. c2020 [cited 2020 Dec 25]. https://www.cned.cl/indices/contexto-institucional-anos-2007-2019
33. Mifuturo.cl [Internet]. Santiago, Chile: [Ministry of Education in Chile] Ministerio de Educación de Chile. c2020 [cited 2020 Dec 25]. https://www.mifuturo.cl/buscador-de-estadisticas-por-carrera/
34. Glaría R, Carmona L, Pérez C, Parra P. [Factorial Structure and Reliability of the Scale of Assessment of the Curriculum of University Programs in Chilean Speech and Language Therapist Students]. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2016; 1(41): 80–89.
- View Article
- Google Scholar
35. Mavis B, Sousa A, Lipscomb W, et al. Learning about medical student mistreatment from responses to the medical school graduation questionnaire. Acad Med 2014; 89(5):705–711. pmid:24667505
- View Article
- PubMed/NCBI
- Google Scholar
36. Hoosen I, Callaghan R, A survey of workplace bullying of psychiatric trainees in the West Midlands. BJPsych Bulletin 2004; 28(6): 225–227.
- View Article
- Google Scholar
37. Dennick R. Constructivism: reflections on twenty five years teaching the constructivist approach in medical education. Int. J. Med. Educ. 2016; 7: 200–215. pmid:27344115
- View Article
- PubMed/NCBI
- Google Scholar
38. Kim KJ Hwang JY. Characteristics of medical teachers using student-centered teaching methods. Korean J Med Educ. 2017; 29(3): 187. pmid:28870021
- View Article
- PubMed/NCBI
- Google Scholar
39. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal. 1999; 6(1): 1–55.
- View Article
- Google Scholar
40. West SG, Taylor AB, Wu W. Model fit and model selection in structural equation modeling. Handbook of structural equation modeling. United States; Guilford press: 2012.
41. Marsh HW, Nagengast B, Morin AJS. Measurement invariance of Big-Five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and La Dolce Vita effects. Dev. Psychol. 2013; 49: 1194–1218. pmid:22250996
- View Article
- PubMed/NCBI
- Google Scholar
42. Yuan K, Bentler P. Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociol. Methodol. 2000; 30: 165–200.
- View Article
- Google Scholar
43. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling. 2007; 14: 464–504.
- View Article
- Google Scholar
44. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014; 105(3): 399–412. pmid:24844115
- View Article
- PubMed/NCBI
- Google Scholar
45. Gonzalez T, de la Rubia MA, Hincz KP, Comas-Lopez M, Subirats L, Fort S et al. Influence of COVID-19 confinement on students’ performance in higher education. PloS One. 2020; 15(10): e0239490. pmid:33035228
- View Article
- PubMed/NCBI
- Google Scholar
46. Kang M, Choi Y, Kim J, Lee KO, Lee S, Park IK et al. COVID-19 impact on city and region: what’s next after lockdown? Urban Sci. 2020; 24(3): 297–315.
- View Article
- Google Scholar
47. Al-Balas M, Al-Balas HI, Jaber HM, Obeidat K, Al-Balas H, Aborajooh EA et al. Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. 2020; 20(1): 1–7.
- View Article
- Google Scholar
48. Harden R, Crosby J. The good teacher is more than a lecturer–the twelve roles of the teacher. AMEE Medical Education Guide No 20. Med Teach. 2000; 22(4): 334–347.
- View Article
- Google Scholar
49. Srinivasan M, Li STT, Meyers FJ, Pratt DD, Collins JB, Braddock C et al. Teaching as a competency: competencies for medical educators. Acad Med. 2011; 86(10): 1211–1220. pmid:21869655
- View Article
- PubMed/NCBI
- Google Scholar
50. Pérez C, Vaccarezza G, Aguilar C, Coloma K, Salgado H, Baquedano M et al. [Factor analysis and internal consistency of pedagogical practices questionnaire among health care teachers]. Rev Med Chile. 2016; 144: 795–805. pmid:27598500
- View Article
- PubMed/NCBI
- Google Scholar
51. Troncoso D, Pérez C, Vaccarezza G, Aguilar C, Muñoz N. [The influence of pedagogic and discipline training on the teaching quality of university professors]. Rev Med Chile. 2017; 145: 610–8. pmid:28898337
- View Article
- PubMed/NCBI
- Google Scholar
52. Pérez C, Fasce E, Coloma K, Vaccarezza G, Ortega J. [Perceptions about continuous training of Chilean health care teachers]. Rev Med Chile. 2013; 141: 787–92. pmid:24121583
- View Article
- PubMed/NCBI
- Google Scholar
53. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC, United States: APA; 2014. 194 p.
54. Al-Hussain SM, Al-Haidari MS, Kouri NA, El-Migdadi F, Al-Safar RS, Mohammad MA: Prevalence of mistreatment and justice of grading system in five health related faculties in Jordan University of Science and Technology. Med Teach. 2008; 30(3):e82–6. pmid:18484446
- View Article
- PubMed/NCBI
- Google Scholar
55. Chadaga AR, Villines D, Krikorian A. Bullying in the American Graduate Medical Education System: A National Cross-Sectional Survey. PLOS ONE. 2016, 11(3): e0150246. pmid:26982705
- View Article
- PubMed/NCBI
- Google Scholar
56. Rautio A, Sunnari V, Nuutinen M, Laitala M. Mistreatment of university students most common during medical studies. BMC Med Educ. 2005; 5: 36. pmid:16232310
- View Article
- PubMed/NCBI
- Google Scholar
57. Daugherty SR, Baldwin DC, Rowley BD. Learning, satisfaction, and mistreatment during medical internship: a national survey of working conditions. JAMA. 1998; 279(15): 1194–9. pmid:9555759
- View Article
- PubMed/NCBI
- Google Scholar
58. Lind KT, Osborne CM, Badesch B, Blood A, Lowenstein SR. Ending student mistreatment: early successes and continuing challenges. Med Educ Online. 2020; 25(1): 1690846. pmid:31787035
- View Article
- PubMed/NCBI
- Google Scholar
59. Chung M, Thang C, Vermillion M, Fried J & Uijtdehaage S Exploring medical students’ barriers to reporting mistreatment during clerkships: a qualitative study, Med Educ Online. 2018; 23(1): 1478170. pmid:29848223
- View Article
- PubMed/NCBI
- Google Scholar
60. Shulman L, Shulman J. How and what teachers learn: a shifting perspective, Journal of Curriculum Studies. 2004; 36:2: 257–271.
- View Article
- Google Scholar
61. Melipillán ER, Hu M. Measurement Invariance Across Groups. SAGE Research Methods Foundations; London, United Kingdom; Sage: 2020.
62. Martínez R, Hernández MJ, Hernández MV. [Psychometrics]. Madrid, Spain: Alianza; 2006.

[ref1] 1. Marinoni G, van’t Land H. The Impact of the Coronavirus on Global Higher Education. International Higher Education. 2020; 102: 7–9.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Alawamleh M. COVID-19 and higher education. Journal of Economic and Economic Education Research 2020; 21(2): 1–3.
View Article
Google Scholar

[5] View Article

[6] Google Scholar

[ref3] 3. Bassett RM. Sustaining the Values of Tertiary Education during the COVID-19 Crisis. International Higher Education. 2020; 102: 5–7.
View Article
Google Scholar

[8] View Article

[9] Google Scholar

[ref4] 4. United Nations Educational, Scientific, and Cultural Organization, UNESCO [Internet]. United States: UNESCO; c2019-2020 [cited 2020 Nov 24]. COVID-19. Education: From disruption to recovery; [about 3 screens]. https://en.unesco.org/covid19/educationresponse.

[ref5] 5. Loda T, Löffler T, Erschens R, Zipfel S, Herrmann-Werner A. Medical education in times of COVID-19: German students’ expectations—A cross-sectional study. PLOS ONE. 2020; 15(11): e0241660. pmid:33206678
View Article
PubMed/NCBI
Google Scholar

[12] View Article

[13] PubMed/NCBI

[14] Google Scholar

[ref6] 6. Dinh LP, Nguyen T. Pandemic, social distancing, and social work education: students’ satisfaction with online education in Vietnam. Social Work Education. 2020; 00(00), 1–10.
View Article
Google Scholar

[16] View Article

[17] Google Scholar

[ref7] 7. Asarta CJ, Schmidt JR. The effects of online and blended experience on outcomes in a blended learning environment. The Internet and Higher Education. 2020; 44: 1–10. 100708.
View Article
Google Scholar

[19] View Article

[20] Google Scholar

[ref8] 8. Sun A, Chen X. Online education and its effective practice: A research review. Journal of Information Technology Education: Research. 2016; 15: 157–190.
View Article
Google Scholar

[22] View Article

[23] Google Scholar

[ref9] 9. Watermeyer R, Crick T, Knight C, Goodall J. COVID-19 and digital disruption in UK universities: afflictions and affordances of emergency online migration. High Educ. 2020; 78. pmid:32836334
View Article
PubMed/NCBI
Google Scholar

[25] View Article

[26] PubMed/NCBI

[27] Google Scholar

[ref10] 10. Aguilar SJ. Guidelines and tools for promoting digital equity. Information and Learning Science. 2020; 121(5–6): 285–299.
View Article
Google Scholar

[29] View Article

[30] Google Scholar

[ref11] 11. Hodges C, Moore S, Lockee B, Trust T, Bond A. The Difference Between Emergency Remote Teaching and Online Learning. Educause Review. 2020; 27: 1–15. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning
View Article
Google Scholar

[32] View Article

[33] Google Scholar

[ref12] 12. Finnerty E, Chauvin S, Bonaminio G, Andrews M, Carroll R, Pangaro L. Flexner Revisited: The Role and Value of the Basic Sciences in Medical Education, Acad Med. 2010; 85 (2) 349–355. pmid:20107367
View Article
PubMed/NCBI
Google Scholar

[35] View Article

[36] PubMed/NCBI

[37] Google Scholar

[ref13] 13. De Ponti R, Marazzato J, Maresca AM, Rovera F, Carcano G, Ferrario MM. Pre-graduation medical training including virtual reality during COVID-19 pandemic: a report on students’ perception. BMC Med Edu. 2020; 20(1): 1–7. pmid:32977781
View Article
PubMed/NCBI
Google Scholar

[39] View Article

[40] PubMed/NCBI

[41] Google Scholar

[ref14] 14. Cleland J, Tan ECP, Tham KY, Low-Beer N. How Covid-19 opened up questions of sociomateriality in healthcare education. Adv Health Sci Educ. 2020; 25(2): 479–482. pmid:32378152
View Article
PubMed/NCBI
Google Scholar

[43] View Article

[44] PubMed/NCBI

[45] Google Scholar

[ref15] 15. Soled D, Goel S, Barry D, Erfani P, Joseph N, Kochis M, et al. Medical Student Mobilization during a Crisis: Lessons from a COVID-19 Medical Student Response Team. Acad Med. 2020; 95(9): 1384–1387. pmid:32282373
View Article
PubMed/NCBI
Google Scholar

[47] View Article

[48] PubMed/NCBI

[49] Google Scholar

[ref16] 16. Trust T, Whalen J. Should Teachers Be Trained in Emergency Remote Teaching? Lessons Learned from the COVID-19 Pandemic. Journal of Technology and Teacher Education. 2020; 28(2): 189–199.
View Article
Google Scholar

[51] View Article

[52] Google Scholar

[ref17] 17. Chemi T. It Is Impossible: The Teacher’s Creative Response to the Covid-19 Emergency and Digitalized Teaching Strategies. Qual Inq. 2020; 1–8.
View Article
Google Scholar

[54] View Article

[55] Google Scholar

[ref18] 18. Chandrasinghe P, Siriwardana RC, Kumarage SK, Munasinghe BNL, Weerasuriya A, Tillakaratne S et al. A novel structure for online surgical undergraduate teaching during the Covid-19 pandemic. BMC Med Educ. 2020; 20(324): 1–7. pmid:32962691
View Article
PubMed/NCBI
Google Scholar

[57] View Article

[58] PubMed/NCBI

[59] Google Scholar

[ref19] 19. Sklar DP. COVID-19: Lessons from the disaster that can improve health professions education. Acad Med. 2020; 95(11): 1631–1633. pmid:32544103
View Article
PubMed/NCBI
Google Scholar

[61] View Article

[62] PubMed/NCBI

[63] Google Scholar

[ref20] 20. Manca S. Snapping, pinning, liking or texting: Investigating social media in higher education beyond Facebook. The Internet and Higher Education. 2020; 44. 100707.
View Article
Google Scholar

[65] View Article

[66] Google Scholar

[ref21] 21. Amir LR, Tanti I, Maharani DA, Julia V, Sulijaya B, Puspitawati R. Student perspective of classroom and distance learning during COVID-19 pandemic in the undergraduate dental study program Universitas Indonesia. BMC Med Educ. 2020; 20: 392. pmid:33121488
View Article
PubMed/NCBI
Google Scholar

[68] View Article

[69] PubMed/NCBI

[70] Google Scholar

[ref22] 22. Wang K, Zhang L, Ye L. A nationwide survey of online teaching strategies in dental education in China. J. Dent. Educ. 2020; 1–7. pmid:32954532
View Article
PubMed/NCBI
Google Scholar

[72] View Article

[73] PubMed/NCBI

[74] Google Scholar

[ref23] 23. Rizun M, Strzelecki A. Students’ acceptance of the covid-19 impact on shifting higher education to distance learning in Poland. Int. J. Environ. Res. Public Health. 2020; 17(6468): 1–19. pmid:32899478
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref24] 24. Byrnes YM, Civantos AM, Go BG, McWilliams TL, Rajasekaran K. Effect of the COVID-19 pandemic on medical student career perceptions: a national survey study. Med. Educ. Online. 2020; 25(1): 1798088. pmid:32706306
View Article
PubMed/NCBI
Google Scholar

[80] View Article

[81] PubMed/NCBI

[82] Google Scholar

[ref25] 25. Inzunza B, Ortiz L, Pérez C, Torres G, McColl P, Meyer A et al. [Factorial Structure and Reliability of the Academic Satisfaction Questionnaire on Chilean Medical Students]. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2015; 40(1): 73–82.
View Article
Google Scholar

[84] View Article

[85] Google Scholar

[ref26] 26. Vergara-Morales J, Del Valle M, Díaz A, Pérez MV. [Adaptation of the Academic Satisfaction Scale in Chilean university students]. Psicología Educativa. 2018; 24: 99–106.
View Article
Google Scholar

[87] View Article

[88] Google Scholar

[ref27] 27. Kirkpatrick DL, Kirkpatrick JD. The transfer of learning to behavior: Using the four levels to improve performance. California, United States: Berrett-Koehler Publication. 2005.

[ref28] 28. Abdulghani HM, Shaik SA, Khamis N, Al-Drees AA, Irshad M, Khalil MS et al. Research methodology workshops evaluation using the Kirkpatrick’s model: translating theory into practice. Med Teach. 2014; 36(sup1): S24–S29. pmid:24617780
View Article
PubMed/NCBI
Google Scholar

[91] View Article

[92] PubMed/NCBI

[93] Google Scholar

[ref29] 29. Aluko FR, Shonubi OK. Going beyond Kirkpatrick’s Training Evaluation Model: The role of workplace factors in distance learning transfer. Africa Education Review. 2014; 11(4): 638–657.
View Article
Google Scholar

[95] View Article

[96] Google Scholar

[ref30] 30. Sutarto S, Usman H, Jaedun A. Impact-Based Training Evaluation Model (IBTEM) for School Supervisors in Indonesia. International Education Studies. 2016; 9 (2), 149–165.
View Article
Google Scholar

[98] View Article

[99] Google Scholar

[ref31] 31. Torres E, Araya-Castillo L. [Constructing a Scale to Measure the Quality of University Services: Application to the Chilean Context]. Revista de Ciencias Sociales 2010; 16(1): 54–67.
View Article
Google Scholar

[101] View Article

[102] Google Scholar

[ref32] 32. CNED.cl [Internet]. Santiago, Chile: [National Conseil of Education of Chile] Consejo Nacional de Educación de Chile. c2020 [cited 2020 Dec 25]. https://www.cned.cl/indices/contexto-institucional-anos-2007-2019

[ref33] 33. Mifuturo.cl [Internet]. Santiago, Chile: [Ministry of Education in Chile] Ministerio de Educación de Chile. c2020 [cited 2020 Dec 25]. https://www.mifuturo.cl/buscador-de-estadisticas-por-carrera/

[ref34] 34. Glaría R, Carmona L, Pérez C, Parra P. [Factorial Structure and Reliability of the Scale of Assessment of the Curriculum of University Programs in Chilean Speech and Language Therapist Students]. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2016; 1(41): 80–89.
View Article
Google Scholar

[106] View Article

[107] Google Scholar

[ref35] 35. Mavis B, Sousa A, Lipscomb W, et al. Learning about medical student mistreatment from responses to the medical school graduation questionnaire. Acad Med 2014; 89(5):705–711. pmid:24667505
View Article
PubMed/NCBI
Google Scholar

[109] View Article

[110] PubMed/NCBI

[111] Google Scholar

[ref36] 36. Hoosen I, Callaghan R, A survey of workplace bullying of psychiatric trainees in the West Midlands. BJPsych Bulletin 2004; 28(6): 225–227.
View Article
Google Scholar

[113] View Article

[114] Google Scholar

[ref37] 37. Dennick R. Constructivism: reflections on twenty five years teaching the constructivist approach in medical education. Int. J. Med. Educ. 2016; 7: 200–215. pmid:27344115
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref38] 38. Kim KJ Hwang JY. Characteristics of medical teachers using student-centered teaching methods. Korean J Med Educ. 2017; 29(3): 187. pmid:28870021
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

[ref39] 39. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal. 1999; 6(1): 1–55.
View Article
Google Scholar

[124] View Article

[125] Google Scholar

[ref40] 40. West SG, Taylor AB, Wu W. Model fit and model selection in structural equation modeling. Handbook of structural equation modeling. United States; Guilford press: 2012.

[ref41] 41. Marsh HW, Nagengast B, Morin AJS. Measurement invariance of Big-Five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and La Dolce Vita effects. Dev. Psychol. 2013; 49: 1194–1218. pmid:22250996
View Article
PubMed/NCBI
Google Scholar

[128] View Article

[129] PubMed/NCBI

[130] Google Scholar

[ref42] 42. Yuan K, Bentler P. Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociol. Methodol. 2000; 30: 165–200.
View Article
Google Scholar

[132] View Article

[133] Google Scholar

[ref43] 43. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling. 2007; 14: 464–504.
View Article
Google Scholar

[135] View Article

[136] Google Scholar

[ref44] 44. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014; 105(3): 399–412. pmid:24844115
View Article
PubMed/NCBI
Google Scholar

[138] View Article

[139] PubMed/NCBI

[140] Google Scholar

[ref45] 45. Gonzalez T, de la Rubia MA, Hincz KP, Comas-Lopez M, Subirats L, Fort S et al. Influence of COVID-19 confinement on students’ performance in higher education. PloS One. 2020; 15(10): e0239490. pmid:33035228
View Article
PubMed/NCBI
Google Scholar

[142] View Article

[143] PubMed/NCBI

[144] Google Scholar

[ref46] 46. Kang M, Choi Y, Kim J, Lee KO, Lee S, Park IK et al. COVID-19 impact on city and region: what’s next after lockdown? Urban Sci. 2020; 24(3): 297–315.
View Article
Google Scholar

[146] View Article

[147] Google Scholar

[ref47] 47. Al-Balas M, Al-Balas HI, Jaber HM, Obeidat K, Al-Balas H, Aborajooh EA et al. Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. 2020; 20(1): 1–7.
View Article
Google Scholar

[149] View Article

[150] Google Scholar

[ref48] 48. Harden R, Crosby J. The good teacher is more than a lecturer–the twelve roles of the teacher. AMEE Medical Education Guide No 20. Med Teach. 2000; 22(4): 334–347.
View Article
Google Scholar

[152] View Article

[153] Google Scholar

[ref49] 49. Srinivasan M, Li STT, Meyers FJ, Pratt DD, Collins JB, Braddock C et al. Teaching as a competency: competencies for medical educators. Acad Med. 2011; 86(10): 1211–1220. pmid:21869655
View Article
PubMed/NCBI
Google Scholar

[155] View Article

[156] PubMed/NCBI

[157] Google Scholar

[ref50] 50. Pérez C, Vaccarezza G, Aguilar C, Coloma K, Salgado H, Baquedano M et al. [Factor analysis and internal consistency of pedagogical practices questionnaire among health care teachers]. Rev Med Chile. 2016; 144: 795–805. pmid:27598500
View Article
PubMed/NCBI
Google Scholar

[159] View Article

[160] PubMed/NCBI

[161] Google Scholar

[ref51] 51. Troncoso D, Pérez C, Vaccarezza G, Aguilar C, Muñoz N. [The influence of pedagogic and discipline training on the teaching quality of university professors]. Rev Med Chile. 2017; 145: 610–8. pmid:28898337
View Article
PubMed/NCBI
Google Scholar

[163] View Article

[164] PubMed/NCBI

[165] Google Scholar

[ref52] 52. Pérez C, Fasce E, Coloma K, Vaccarezza G, Ortega J. [Perceptions about continuous training of Chilean health care teachers]. Rev Med Chile. 2013; 141: 787–92. pmid:24121583
View Article
PubMed/NCBI
Google Scholar

[167] View Article

[168] PubMed/NCBI

[169] Google Scholar

[ref53] 53. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC, United States: APA; 2014. 194 p.

[ref54] 54. Al-Hussain SM, Al-Haidari MS, Kouri NA, El-Migdadi F, Al-Safar RS, Mohammad MA: Prevalence of mistreatment and justice of grading system in five health related faculties in Jordan University of Science and Technology. Med Teach. 2008; 30(3):e82–6. pmid:18484446
View Article
PubMed/NCBI
Google Scholar

[172] View Article

[173] PubMed/NCBI

[174] Google Scholar

[ref55] 55. Chadaga AR, Villines D, Krikorian A. Bullying in the American Graduate Medical Education System: A National Cross-Sectional Survey. PLOS ONE. 2016, 11(3): e0150246. pmid:26982705
View Article
PubMed/NCBI
Google Scholar

[176] View Article

[177] PubMed/NCBI

[178] Google Scholar

[ref56] 56. Rautio A, Sunnari V, Nuutinen M, Laitala M. Mistreatment of university students most common during medical studies. BMC Med Educ. 2005; 5: 36. pmid:16232310
View Article
PubMed/NCBI
Google Scholar

[180] View Article

[181] PubMed/NCBI

[182] Google Scholar

[ref57] 57. Daugherty SR, Baldwin DC, Rowley BD. Learning, satisfaction, and mistreatment during medical internship: a national survey of working conditions. JAMA. 1998; 279(15): 1194–9. pmid:9555759
View Article
PubMed/NCBI
Google Scholar

[184] View Article

[185] PubMed/NCBI

[186] Google Scholar

[ref58] 58. Lind KT, Osborne CM, Badesch B, Blood A, Lowenstein SR. Ending student mistreatment: early successes and continuing challenges. Med Educ Online. 2020; 25(1): 1690846. pmid:31787035
View Article
PubMed/NCBI
Google Scholar

[188] View Article

[189] PubMed/NCBI

[190] Google Scholar

[ref59] 59. Chung M, Thang C, Vermillion M, Fried J & Uijtdehaage S Exploring medical students’ barriers to reporting mistreatment during clerkships: a qualitative study, Med Educ Online. 2018; 23(1): 1478170. pmid:29848223
View Article
PubMed/NCBI
Google Scholar

[192] View Article

[193] PubMed/NCBI

[194] Google Scholar

[ref60] 60. Shulman L, Shulman J. How and what teachers learn: a shifting perspective, Journal of Curriculum Studies. 2004; 36:2: 257–271.
View Article
Google Scholar

[196] View Article

[197] Google Scholar

[ref61] 61. Melipillán ER, Hu M. Measurement Invariance Across Groups. SAGE Research Methods Foundations; London, United Kingdom; Sage: 2020.

[ref62] 62. Martínez R, Hernández MJ, Hernández MV. [Psychometrics]. Madrid, Spain: Alianza; 2006.

Figures

Abstract

Introduction

Method

Results

Discussion

Introduction

Effects on higher education

Emergency remote teaching

Effects on the teachers

Effects on the students

Satisfaction with teaching

Purpose of this study

Method

Participants

Data collection

Procedure

Ethics

Data analysis

Results

Descriptive data

Construct validity: Confirmatory factor analysis

Measurement invariance

Comparing factor means across cohorts

Reliability

Discussion

Limitations

Conclusions

Supporting information

S1 Table. Escala de satisfacción con la docencia a distancia (versión original en Español).

S2 Table. Remote Teaching Satisfaction Scale (proposed English version).

Acknowledgments

References