Abstract
The concepts and implementation of flipped education have been prevailing globally. However, most studies focus on learning performance with few analyses on moderator variables. The study conducted a meta-analysis to analyze 39 studies published over the 2012–2018 period. Among the 39 studies involving 4662 participants, 33 of the studies analyzed the dependent variable of learning performance, and 6 of the studies analyzed the other dependent variable of higher-order thinking. To further examine the effects, four moderators—education level, scaffolding, subject area, and country, were analyzed to investigate the various effects on learning performance. Overall, flipped education demonstrated a low effect size of 0.31 for learning performance, and a moderate effect size of 0.53 for higher-order thinking. The in-depth results and syntheses of the four moderators and two dependent variables and their implications are discussed.
Similar content being viewed by others
References
Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14.
Ajuwon, P. M., Sarraj, H., Griffin-Shirley, N., Lechtenberger, D., & Zhou, L. (2015). Including students who are visually impaired in the classroom: Attitudes of preservice teachers. Journal of Visual Impairment & Blindness, 109(2), 131–140.
Anderson, N., Potočnik, K., & Zhou, J. (2014). Innovation and creativity in organizations: A state-of-the-science review, prospective commentary, and guiding framework. Journal of Management, 40(5), 1297–1333.
Alsowat, H. (2016). An EFL flipped classroom teaching model: Effects on English language higher-order thinking skills, student engagement and satisfaction. Journal of Education and Practice, 7(9), 108–121.
Ahmed, H. O. K. (2016). Flipped learning as a new educational paradigm: An analytical critical study. European Scientific Journal, 12(10).
Balakrishnan, A., Puthean, S., Satheesh, G., & MK, U., Rashid, M., Nair, S., & Thunga, G. (2021). Effectiveness of blended learning in pharmacy education: A systematic review and meta-analysis. PLoS ONE, 16(6), 1–17.
Beatty, K. (2013). Teaching & researching: Computer-assisted language learning. Routledge.
Bergmann, J., & Sams, A. (2012a). Flip your classroom: Talk to every student in every class every day. International Society for Technology in Education. URL http://books.google.com/books?id=nBi2pwAACAAJ
Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York, NY: McKay, 20–24.
Cagiltay, K. (2006). Scaffolding strategies in electronic performance support systems: Types and challenges. Innovations in Education and Teaching International, 43(1), 93–103.
Calimeris, L., & Sauer, K. M. (2015). Flipping out about the flip: All hype or is there hope? International Review of Economics Education, 20, 13–28.
Chen, K. S., Monrouxe, L., Lu, Y. H., Jenq, C. C., Chang, Y. J., Chang, Y. C., & Chai, P. Y. C. (2018a). Academic outcomes of flipped classroom learning: A meta-analysis. Medical Education, 52, 910–924.
Chen, T. L., Hsiao, T. C., Kang, T. C., Wu, T. Y., & Chen, C. C. (2020). Learning programming language in higher education for sustainable development: Point-earning bidding method. Sustainability, 12(11), 4489.
Chien, C. F., & Chiang-Hsieh, L. H. C. (2018). Exploring University Students’ Achievement, Motivation, and Receptivity of Flipped Learning in an Engineering Mathematics Course. International Journal of Online Pedagogy and Course Design (IJOPCD), 8(4), 22–37.
Chua, J. S. M., & Lateef, F. A. (2014). The flipped classroom: Viewpoints in Asian Universities. Education in Medicine Journal, 6(4), 20–26.
Collins, R. (2014). Skills for the 21st Century: teaching higher-order thinking. Curriculum & Leadership Journal, 12(14). Retrieved from http://www.curriculum.edu.au/leader/teaching_higher_order_thinking,37431.html?issueID=12910
Cookson, P. W. (2009). What would Socrates say. Educational Leadership, 67(1), 8–14.
Cronin, J. G., McMahon, J. P., & Waldron, M. (2009). Critical survey of information technology use in higher education: Blended classrooms. In C. R. Payne (Ed.). Information technology and constructivism in higher education: Progressive learning frameworks (pp. 203–215). Hershey, PA: IGI Global.
Davis, E. A. (1996). Metacognitive scaffolding to foster scientific explanations. Paper presented at AERA, New York, NY.
Doucouliagos, H., & Stanley, T. D. (2009). Publication selection bias in minimum-wage research? A meta-regression analysis. British Journal of Industrial Relations, 47(2), 406–428.
Field, A. P., & Gillett, R. (2010). How to do a meta-analysis? British Journal of Mathematical and Statistical Psychology, 63(3), 665–694.
Fisher, R. A. (1935). The design of experiments. Oliver & Boyd.
Flumerfelt, S., & Green, G. (2013). Using lean in the flipped classroom for at risk students. Journal of Educational Technology & Society, 16(1), 356.
Fulton, K. (2012). Upside down and inside out: Flip your classroom to improve student learning. Learning & Leading with Technology, 39(8), 12–17.
Gillette, C., Rudolph, M., Kimble, C., Rockich-Winston, N., Smith, L., & Broedel-Zaugg, K. (2018). A meta-analysis of outcomes comparing flipped classroom and lecture. American Journal of Pharmaceutical Education, 82(5), 433–440.
Haidich, A. B. (2010). Meta-analysis in medical research. Hippokratia, 14(Suppl 1), 29–37.
Hannafin, M., Land, S., & Oliver, K. (1999). Open learning environments: Foundations, methods, and models. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, 2 (pp. 115–140). Mahwah, NK. Lawrence Erlbaum Associates, Inc., Publishers.
Hew, K. F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., Jia, C., & Thankrit, K. (2021). On the use of flipped classroom across various disciplines: Insights from a second-order meta-analysis. Australasian Journal of Educational Technology, 37(2), 132–151.
Hwang, G. J., Lai, C. L., & Wang, S. Y. (2015). Seamless flipped learning: A mobile technology-enhanced flipped classroom with effective learning strategies. Journal of Computers in Education, 2(4), 449–473.
Fautch, J. M. (2015). The flipped classroom for teaching organic chemistry in small classes: Is it effective? Chemistry Education Research and Practice, 16(1), 179–186.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-Analysis. Academy Press, Inc.
Jamaludin, R., & Osman, S. (2014). The use of a flipped classroom to enhance engagement and promote active learning. Journal of Education and Practice, 5(2), 124–131.
Južnič, P., Pečlin, S., Žaucer, M., Mandelj, T., Pušnik, M., & Demšar, F. (2010). Scientometric indicators: Peer-review, bibliometric methods and conflict of interests. Scientometrics, 85(2), 429–441.
Khodaei, S., Hasanvand, S., Gholami, M., Mokhayeri, Y., & Amini, M. (2022). The effect of the online flipped classroom on self-directed learning readiness and metacognitive awareness in nursing students during the COVID-19 pandemic. BMC Nursing, 21(1), 1–10.
Kim, B. (2001). Social constructivism. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and technology. Retrieved from http://epltt.coe.uga.edu/index.php?title=Social_Constructivism
Kim, M., Kim, S., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. Internet and Higher Education, 22, 37–50.
King, W. R., & He, J. (2006). A meta-analysis of the technology acceptance model. Information & Management, 43(6), 740–755.
Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy. Computers & Education, 78, 160–173.
Lai, C. L., & Hwang, G. J. (2016). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education, 100, 126–140.
Li, X., Liu, B., Duan, N., Xiong, Y., Zhang, Y., Zhang, C., Lu, C., & Li, L. (2022). Clinical efficacy and safety of chinese patent medicine combined with oseltamivir for the treatment of adult influenza: A systematic review and meta-analysis. The American Journal of Chinese Medicine, 50(4), 1–19.
Makarem, S. C. (2015). Using online video lectures to enrich traditional face-to-face courses. International Journal of Instruction, 8(2), 155–164.
Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Transactions on Education, 56(4), 430–435.
McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., & Mumper, R. J. (2014). The flipped classroom: A course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236–243.
Milman, N. B. (2012). The flipped classroom strategy: What is it and how can it best be used? Distance Learning, 9(3), 85–87.
Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education, 52(10), 597–599.
Moran, K., & Milsom, A. (2015). The flipped classroom in counselor education. Counselor Education and Supervision, 54(1), 32–43.
Nelson, J. P. (2011). Alcohol marketing, adolescent drinking and publication bias in longitudinal studies: A critical survey using meta-analysis. Journal of Economic Surveys, 25(2), 191–232.
O’Scanaill, M. (2020). Flipped classroom vs. flipped learning: Know the difference. Dyknow. https://www.dyknow.com/blog/flipped-classroom-vs-flipped-learning-know-the-difference/
Paulus, P. B. (2002). Different ponds for different fish: A contrasting perspective on team innovation. Applied Psychology, 51, 394–399.
Perry, W. G. (1999). Forms of ethical and intellectual development in the college years: A scheme. Jossey-Bass.
Rank, J., Pace, V. L., & Frese, M. (2004). Three avenues for future research on creativity, innovation, and initiative. Applied Psychology, 53, 518–528.
Rosiek, J. (2003). Emotional scaffolding: An exploration of the teacher knowledge at the intersection of student emotion and the subject matter. Journal of Teacher Education, 54(5), 399–412.
Rothstein, H. R., & Bushman, B. J. (2012). Publication bias in psychological science: Comment on Ferguson and Brannick. Psychological Methods, 17(1), 129–136.
Shahnama, M., Ghonsooly, B., & Shirvan, M. E. (2021). A meta-analysis of relative effectiveness of flipped learning in English as second/foreign language research. Educational Technology Research and Development, 69(3), 1355–1386.
Shi, Y., Ma, Y., MacLeod, J., & Yang, H. H. (2020). College students’ cognitive learning outcomes in flipped classroom instruction: A meta-analysis of the empirical literature. Journal of Computers in Education, 7(1), 79–103.
Shyr, W. J., & Chen, C. H. (2018). Designing a technology-enhanced flipped learning system to facilitate students’ self-regulation and performance. Journal of Computer Assisted Learning, 34(1), 53–62.
Sletten, S. R. (2017). Investigating flipped learning: Student self-regulated learning, perceptions, and achievement in an introductory biology course. Journal of Science Education and Technology, 3(26), 347–358.
Strelan, P., Osborn, A., & Palmer, E. (2020). The flipped classroom: A meta-analysis of effects on student performance across disciplines and education levels. Educational Research Review, 30, 1–22.
Sun, J. C. Y., & Wu, Y. T. (2016). Analysis of learning performance and teacher–student interactions in flipped and conventional classrooms. The International Review of Research in Open and Distributed Learning, 17(1), 79–99.
Thi Van Pham, A., & Thi Thao Ho, N. (2021). Perceptions of Lecturers and Students on Challenges of the Flipped Classroom [Conference]. 2021 3rd International Conference on Modern Educational Technology, Jakarta, Indonesia.
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, pp 52–58.
Yilmaz, R. M., & Baydas, O. (2017). An examination of undergraduates’ metacognitive strategies in pre-class asynchronous activity in a flipped classroom. Educational Technology Research and Development, 65(6), 1547–1567.
Zainuddin, Z., Haruna, H., Li, X., Zhang, Y., & Chu, S. K. W. (2019). A systematic review of flipped classroom empirical evidence from different fields: What are the gaps and future trends? On the Horizon, 12, 1–15. https://doi.org/10.1108/OTH-09-2018-0027
Meta-analysis review articles
Al-Zahrani, A. M. (2015). From passive to active: The impact of the flipped classroom through social learning platforms on higher education students’ creative thinking. British Journal of Educational Technology, 46(6), 1133–1148.
Albert, M., & Beatty, B. J. (2014). Flipping the classroom applications to curriculum redesign for an introduction to management course: Impact on grades. Journal of Education for Business, 89(8), 419–424.
Aşıksoy, G., & Özdamlı, F. (2016). Flipped Classroom adapted to the ARCS model of motivation and applied to a physics course. Eurasia Journal of Mathematics, Science & Technology Education, 12(6), 1589–1603.
Bergmann, J., & Sams, A. (2012b). Flip your classroom: Reach every student in every class every day. International Society for Technology. in Education.
Bhagat, K. K., Chang, C. N., & Chang, C. Y. (2016). The impact of the flipped classroom on mathematics concept learning in high school. Educational Technology & Society, 19(3), 134–142.
Chao, C. Y., Chen, Y. T., & Chuang, K. Y. (2015). Exploring students’ learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education. Computer Applications in Engineering Education, 23(4), 514–526.
Chen, K. S., Monrouxe, L., Lu, Y. H., Jenq, C. C., Chang, Y. J., Chang, Y. C., & Chai, P. Y. C. (2018b). Academic outcomes of flipped classroom learning: A meta-analysis. Medical Education, 52(9), 910–924.
Chen, H. H., Pan, H. H., Hsu, Y. D., Cheng, C. Y., Wu, C. H., Lin, W. S., & Lin, C. S. (2016). Flipped Classroom as an effective model to teach electrocardiography to undergraduate medical students. Journal of Medical Education, 20(4), 276–284.
Chen Hsieh, J. S., Wu, W. C. V., & Marek, M. W. (2017). Using the flipped classroom to enhance EFL learning. Computer Assisted Language Learning, 30(1), 1–21.
Cohen, M. A. (1988). Some new evidence on the seriousness of crime. Criminology, 26(2), 343–353.
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61(4), 563–580.
Dove, A., & Dove, E. (2015). Examining the influence of a flipped mathematics course on preservice elementary teachers’ mathematics anxiety and achievement. Electronic Journal of Mathematics & Technology, 9(2), 166–179.
Entezari, M., & Javdan, M. (2016). Active learning and flipped classroom, hand in hand approach to improve students learning in human anatomy and physiology. International Journal of Higher Education, 5(4), 222–231.
Hibbard, L., Sung, S., & Wells, B. (2015). Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence. Journal of Chemical Education, 93(1), 24–30.
Huang, Y. N., & Hong, Z. R. (2016). The effects of a flipped English classroom intervention on students’ information and communication technology and English reading comprehension. Educational Technology Research and Development, 64(2), 175–193.
Hung, H. T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81–96.
Hwang, G. J., & Lai, C. L. (2017). Facilitating and bridging out-of-class and in-class learning: An interactive e-book-based flipped learning approach for math courses. Educational Technology & Society, 20(1), 184–197.
Jong, M. S. Y. (2017). Empowering students in the process of social inquiry learning through flipping the classroom. Educational Technology & Society, 20(1), 306–322.
Jensen, J. L., Kummer, T. A., & Godoy, P. D. D. M. (2015). Improvements from a flipped classroom may simply be the fruits of active learning. CBE—Life Sciences Education, 14(1), 1–12.
Kirvan, R., Rakes, C. R., & Zamora, R. (2015). Flipping an algebra classroom: Analyzing, modeling, and solving systems of linear equations. Computers in the Schools, 32(3), 201–223.
Kostaris, C., Stylianos, S., Sampson, D. G., Giannakos, M., & Pelliccione, L. (2017). Investigating the potential of the flipped classroom model in K-12 ICT teaching and learning: An action research study. Educational Technology and Society, 20(1), 261–273.
Kurt, G. (2017). Implementing the flipped classroom in teacher education: Evidence from Turkey. Educational Technology & Society, 20(1), 211–221.
Lai, C. L., & Hwang, G. J. (2016). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education, 100, 126–140.
Leo, J., & Puzio, K. (2016). Flipped instruction in a high school science classroom. Journal of Science Education and Technology, 25(5), 775–781.
Liou, W. K., Bhagat, K. K., & Chang, C. Y. (2016). Beyond the flipped classroom: A highly interactive cloud-classroom (HIC) embedded into basic materials science courses. Journal of Science Education and Technology, 25(3), 460–473.
Liu, D. (2016). The reform and innovation of English course: A coherent whole of MOOC, flipped classroom and ESP. Procedia-Social and Behavioral Sciences, 232, 280–286.
Mohanty, A., & Parida, D. (2016). Exploring the efficacy & suitability of flipped cassroom instruction at school level in India: A pilot study. Creative Education, 7(5), 768–776.
Mooring, S. R., Mitchell, C. E., & Burrows, N. L. (2016). Evaluation of a flipped, large-enrollment organic chemistry course on student attitude and achievement. Journal of Chemical Education, 93(12), 1972–1983.
Mori, Y., Omori, M., & Sato, K. (2016). The impact of flipped online Kanji instruction on written vocabulary learning for introductory and intermediate Japanese language students. Foreign Language Annals, 49(4), 729–749.
Mortensen, C. J., & Nicholson, A. M. (2015). The flipped classroom stimulates greater learning and is a modern 21st century approach to teaching today’s undergraduates. Journal of Animal Science, 93(7), 3722–3731.
Munson, A., & Pierce, R. (2015). Flipping content to improve student examination performance in a pharmacogenomics course. American Journal of Pharmaceutical Education, 79(7), 1–7.
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95.
Prashar, A. (2015). Assessing the flipped classroom in operations management: A pilot study. Journal of Education for Business, 90(3), 126–138.
Robert, J., Lewis, S. E., Oueini, R., & Mapugay, A. (2016). Coordinated implementation and evaluation of flipped classes and peer-led team learning in general chemistry. Journal of Chemical Education, 93(12), 1993–1998.
Salimi, A., & Yousefzadeh, M. (2015). The effect of flipped learning (Revised Learning) on Iranian students’ learning outcomes. Advances in Language and Literary Studies, 6(5), 209–213.
Şengel, E. (2016). To FLIP or not to FLIP: Comparative case study in higher education in Turkey. Computers in Human Behavior, 64, 547–555.
Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193.
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38.
Winquist, J. R., & Carlson, K. A. (2014). Flipped statistics class results: Better performance than lecture over one year later. Journal of Statistics Education, 22(3), 1–10.
Wong, T. H., Ip, E. J., Lopes, I., & Rajagopalan, V. (2014). Pharmacy students’ performance and perceptions in a flipped teaching pilot on cardiac arrhythmias. American Journal of Pharmaceutical Education, 78(10), 1–6.
Zhang, H., Du, X., Yuan, X., & Zhang, L. (2016). The effectiveness of the flipped classroom mode on the English pronunciation course. Creative Education, 7(9), 1340–1346.
Zhonggen, Y., & Guifang, W. (2016). Academic achievements and satisfaction of the clicker-aided flipped business English writing class. Educational Technology & Society, 19(2), 298–312.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Standards
This manuscript does not involve human or animal experiment. All data are journal articles, which are fully ethical.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chien, CF. A Meta-Analysis of Learning and Moderator Variables in Flipped Education. Tech Know Learn 28, 517–538 (2023). https://doi.org/10.1007/s10758-022-09618-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-022-09618-6