Design Matters: Development and Validation of the Online Course Design Elements (OCDE) Instrument

Authors

DOI:

https://doi.org/10.19173/irrodl.v22i2.5187

Keywords:

online course design, design elements, instrument validation, confirmatory factor analysis, structural equation model

Abstract

Course design is critical to online student engagement and retention. This study focused on the development and validation of an online course design elements (OCDE) instrument with 38 Likert-type scale items in five subscales: (a) overview, (b) content presentation, (c) interaction and communication, (d) assessment and evaluation, and (e) learner support. The validation process included implementation with 222 online instructors and instructional designers in higher education. Three models were evaluated which included a one-factor model, five-factor model, and higher-order model. The five-factor and higher-order models aligned with the development of the OCDE. The frequency of use of OCDE items was rated above the mean 4.0 except for two items on collaboration and self-assessment. The overall OCDE score was related to self-reported levels of expertise but not with years of experience. The findings have implications for the use of this instrument with online instructors and instructional designers in the design of online courses.

Author Biographies

Florence Martin, University of North Carolina Charlotte

Florence Martin is a Professor in Learning, Design and Technology at University of North Carolina Charlotte. She received her Doctorate and Master's degrees in Educational Technology from Arizona State University. She teaches 100% online in asynchronous, synchronous and Bichronous modalities. Dr. Martin engages in research to create transformative learning experiences through effective design and integration of digital teaching and learning innovations. She has conducted several studies focusing on designing and integrating online learning environments to improve learner achievement and engagement. Dr. Martin served as the President of Multimedia Production Division in 2012-2013 and as the President of the Division of Distance Learning in 2017-2018 for Association for Educational Communications and Technology. She also served as the Vice President for Marketing and Communications with ISPI Charlotte and is currently serving as a Director for the International Board of Standards for Training, Performance and Instruction. For more details visit, https://www.florencemartin.net

Doris U. Bolliger, Old Dominion University

Doris U. Bolliger is a contributing faculty member at Walden University. She received her doctorate degree from the University of West Florida. Her primary research interests are faculty and student satisfaction, communication, community, engagement, and interaction in the online environment. Her secondary research interests include the utilization of mobile technologies in informal learning environments.

Claudia Flowers, University of North Carolina Charlotte

Dr. Claudia Flowers is a professor of research, statistics, and measurement at the University of North Carolina at Charlotte in the Department of Educational Leadership. She has over 100 publications in the areas of assessment, measurement, and applied research methods. Her current research interests include exploring evidence of technical quality of assessments for students with significant cognitive disabilities and testing accommodations.

References

Allen, I. E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment report 2017. https://onlinelearningsurvey.com/reports/digtiallearningcompassenrollment2017.pdf

Anderson, K. (2017). Have we reached an inflection point in online collaboration? From e-mail to social networks, online collaboration has evolved fast—as have users. Research Information, 92, 24.

Baldwin, S., Ching, Y-H., & Hsu, Y-C. (2018). Online course design in higher education: A review of national and statewide evaluation instruments. TechTrends, 62(1), 46–57. https://doi.org/10.1007/s11528-017-0215-z

Blackboard. (2012). Blackboard exemplary course program rubric. https://www.blackboard.com/resources/are-your-courses-exemplary

Bozarth, J., Chapman, D. D., & LaMonica, L. (2004). Preparing for distance learning: Designing an online student orientation course. Journal of Educational Technology & Society, 7(1), 87–106. https://www.jstor.org/stable/jeductechsoci.7.1.87

Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005

California Community Colleges Online Education Initiative. (2016). Course design rubric. http://cvc.edu/wp-content/uploads/2016/11/OEI_CourseDesignRubric Nov2016-3.pdf

California State University. (2015). QOLT evaluation rubric. https://cal.sdsu.edu/_resources/docs/QOLT%20Instrument.pdfCapdeferro, N., & Romero, M. (2012). Are online learners frustrated with collaborative learning experiences? International Review of Research in Open and Distributed Learning, 13(2), 26–44. https://doi.org/10.19173/irrodl.v13i2.1127

Castle, S. R., & McGuire, C. J. (2010). An analysis of student self-assessment of online, blended, and face-to-face learning environments: Implications for sustainable education delivery. International Education Studies, 3(3), 36–40. https://doi.org/10.5539/ies.v3n3p36

Chen, S.-J. (2007). Instructional design strategies for intensive online courses: An objectivist-constructivist blended approach. Journal of Interactive Online Learning, 6(1), 72–86. http://www.ncolr.org/jiol/issues/pdf/6.1.6.pdf

Czerkawski, B. C., & Lyman, E. W. III. (2016). An instructional design framework for fostering student engagement in online learning environments. TechTrends, 60(6), 532–539. https://doi.org/10.1007/s11528-016-0110-z

Dell, C. A., Dell, T. F., & Blackwell, T. L. (2015). Applying universal design for learning in online courses: Pedagogical and practical considerations. Journal of Educators Online, 12(2), 166–192. https://doi.org/10.9743/jeo.2015.2.1

Dick, W. (1996). The Dick and Carey model: Will it survive the decade? Educational Technology Research and Development, 44(3), 55–63. https://doi.org/10.1007/BF02300425

Dietz-Uhler, B., Fisher, A., & Han, A. (2007). Designing online courses to promote student retention. Journal of Educational Technology Systems, 36(1), 105–112. https://doi.org/10.2190/ET.36.1.g

Fathema, N., Shannon, D., & Ross, M. (2015). Expanding the technology acceptance model (TAM) to examine faculty use of learning management systems (LMSs) in higher education institutions. Journal of Online Learning & Teaching, 11(2), 210–232. https://jolt.merlot.org/Vol11no2/Fathema_0615.pdf

Gaytan, J., & McEwen, B. C. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21(3), 117–132. https://doi.org/10.1080/08923640701341653

Graf, S., Liu, T.-C., & Kinshuk. (2010). Analysis of learners’ navigational behaviour and their learning styles in an online course. Journal of Computer Assisted Learning, 26(2), 116–131. https://doi.org/10.1111/j.1365-2729.2009.00336.x

Han, I., & Shin, W. S. (2016). The use of a mobile learning management system and academic achievement of online students. Computers & Education, 102, 79–89. https://doi.org/10.1016/j.compedu.2016.07.003

Illinois Online Network. (2015). Quality online course initiative (QOCI) rubric. University of Illinois. https://www.uis.edu/ion/resources/qoci/

Jaggars, S. S., & Xu, D. (2016). How do online course design features influence student performance? Computers & Education, 95, 270–284. https://doi.org/10.1016/j.compedu.2016.01.014

Jones, K. R. (2013). Developing and implementing a mandatory online student orientation. Journal of Asynchronous Learning Networks, 17(1), 43–45. https://doi.org/10.24059/olj.v17i1.312

Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.

Ko, S., & Rossen, S. (2017). Teaching online: A practical guide (4th ed.). Routledge.

Kumar, S., Martin, F., Budhrani, K., & Ritzhaupt, A. (2019). Award-winning faculty online teaching practices: Elements of award-winning courses. Online Learning, 23(4), 160– http://dx.doi.org/10.24059/olj.v23i4.2077

Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E.A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15–30. https://doi.org/10.1111/j.1365-2729.2011.00458.x

Le Maistre, C. (1998). What is an expert instructional designer? Evidence of expert performance during formative evaluation. Educational Technology Research and Development, 46(3), 21–36. https://doi.org/10.1007/BF02299759

Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7

Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202.

Liu, X., Magjuka, R. J., Bonk, C. J., & Lee, S.-H. (2007). Does sense of community matter? An examination of participants’ perceptions of building learning communities in online courses. Quarterly Review of Distance Education, 8(1), 9–24.

Luo, N., Zhang, M., & Qi, D. (2017). Effects of different interactions on students’ sense of community in e-learning environment. Computers & Education, 115, 153–160. https://doi.org/10.1016/j.compedu.2017.08.006

Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205–222. http://dx.doi.org/10.24059/olj.v22i1.1092

Martin, F., Wang, C., & Sadaf, A. (2018). Student perception of helpfulness of facilitation strategies that enhance instructor presence, connectedness, engagement and learning in online courses. The Internet and Higher Education, 37, 52–65. https://doi.org/10.1016/j.iheduc.2018.01.003

Martin, F., Ritzhaupt, A., Kumar, S., & Budhrani, K. (2019). Award-winning faculty online teaching practices: Course design, assessment and evaluation, and facilitation. The Internet and Higher Education, 42, 34–43. https://doi.org/10.1016/j.iheduc.2019.04.001

Moore, M. G. (1989). Three types of interaction [Editorial]. American Journal of Distance Education, 3(2), 1–7. https://doi.org/10.1080/08923648909526659

Muthén, L. K., & Muthén, B. O. (2012). Mplus (Version 7.11) [Computer software]. Mplus. https://www.statmodel.com/

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

Online Learning Consortium. (2016). OSCQR course design review. https://s3.amazonaws.com/scorecard-private-uploads/OSCQR+version+3.1.pdf

Perez, R. S., Johnson, J. F., & Emery, C. D. (1995). Instructional design expertise: A cognitive model of design. Instructional Science, 23(5–6), 321–349. https://doi.org/10.1007/BF00896877

Pimmer, C., Mateescu, M., & Grohbiel, U. (2016). Mobile and ubiquitous learning in higher education settings. A systematic review of empirical studies. Computers in Human Behavior, 63, 490–501. https://doi.org/10.1016/j.chb.2016.05.057

Price, J. M., Whitlatch, J., Maier, C. J., Burdi, M., & Peacock, J. (2016). Improving online teaching by using established best classroom teaching practices. Journal of Continuing Education in Nursing, 47(5), 222–227. https://doi.org/10.3928/00220124-20160419-08

Quality Matters. (2020). Specific review standards from the QM Higher Education Rubric (6th ed.). https://www.qualitymatters.org/sites/default/files/PDFs/StandardsfromtheQMHigherEducationRubric.pdf

Rennstich, J. K. (2019). Creative online collaboration: A special challenge for co-creation. Education and Information Technologies, 24(2), 1835–1836. https://doi.org/10.1007/s10639-019-09875-6

Salmon, G. (2013). E-tivities: The key to active online learning. Routledge.

Shackelford, J. L., & Maxwell, M. (2012). Sense of community in graduate online education: Contribution of learner to learner interaction. The International Review of Research in Open and Distributed Learning, 13(4), 228–249. https://doi.org/10.19173/irrodl.v13i4.1339

Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53(2), 252–266. https://doi.org/10.1016/0749-5978(92)90064-E

Ssekakubo, G., Suleman, H., & Marsden, G. (2013). Designing mobile LMS interfaces: Learners’ expectations and experiences. Interactive Technology and Smart Education, 10(2), 147–167. https://doi.org/10.1108/ITSE-12-2012-0031

Stavredes, T., & Herder, T. (2014). A guide to online course design: Strategies for student success. Jossey Bass.

Stevens, D. D., & Levi, A. J. (2013). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning (2nd ed.). Stylus.

Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education, 22(2), 306–331. https://doi.org/10.1080/0158791010220208

Vai, M., & Sosulski, K. (2016). Essentials of online course design: A standards-based guide (2nd ed.). Routledge.

Waterhouse, S., & Rogers, R. O. (2004). The importance of policies in e-learning instruction. EDUCAUSE Quarterly, 27(3), 28–39.

Young, S. (2006). Student views of effective online teaching in higher education. American Journal of Distance Education, 20(2), 65–77. https://doi.org/10.1207/s15389286ajde2002_2

Downloads

Published

2021-01-18

How to Cite

Martin, F., Bolliger, D. U., & Flowers, C. (2021). Design Matters: Development and Validation of the Online Course Design Elements (OCDE) Instrument. The International Review of Research in Open and Distributed Learning, 22(2), 46–71. https://doi.org/10.19173/irrodl.v22i2.5187

Issue

Vol. 22 No. 2 (2021)

Section

Research Articles

License

This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.

This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.

You are free to

  • Share — copy and redistribute the material in any medium or format
  • Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms below:

  • No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.