This study examined the experience of participants from eight LMICs in one of the 2020 TDR MOOC on IR. Using the data from the pre-and post-assessment surveys, the self-reported D&I competencies were analyzed as well as barriers and facilitators to completing the course, which provide recommendations and implications for future MOOCs. Although there was a low retention rate in the MOOC, participants completing the post survey showed improvements in their D&I competencies. Participants reported low access to D&I mentors, limited access to general research mentors, and low self-reported competency for manuscript and scientific writing.
The course completion rate was likely impacted by a couple of variables. First, internet access was a major barrier for retention in this study, which has been shown in similar studies.(10) MOOCs, by design, enroll large groups of students, including both active and passive participants. Reconceptualizing retention to only include participants who substantively engaged with the course (19) might provide a more accurate picture of program metrics. Third, the timeline in which we started the MOOC was challenging. Enrollment took place in April 2020 with a course start date in May 2020, right before several of the participating countries started the lock down to prevent further spread of the SARS-CoV-2 virus. Fourth, as soon as the cohort started the training, the MOOC website moved to be hosted by another company and the transition posed some issues with access to the videos. With the larger movement towards online courses and trainings, future guardrails to develop and maintain websites for online learning will be important.(20–22)
Due to the global uncertainty of the COVID-19 pandemic coupled with TDR platform issues, the first module was extended three months until the end of July 2020. The remaining modules adhered to the original timelines, with a spacing of two weeks between each module. During this period, 53% of enrolled participants no longer engaged in course activities. When asked about the barriers for participation in the post-assessment survey, participants shared that the lack of stable internet, other work commitments and responsibilities, and needing more time to complete the course were key barriers that affected their participation in this course. These barriers have been reported by participants from previous MOOCs,(10, 13, 17) but they were likely intensified by the COVID-19 pandemic and associated lockdowns in this past year.
The results indicate strong evidence of improvement with self-reported D&I competencies similar to previous courses.(10) The subscale that had the largest change was theory and approaches and the subscale with the least change was definitions, background and rationale and practice-based considerations. These results differ from other D&I trainings where a majority of participants reported the largest change in the definitions, background, and rationale subscale.(8, 10) The difference in results may be related to the composition of participants, where 33% of participants in this study had previous D&I training before the course. The general research competency scores in manuscript writing, proposal writing, and giving scholarly presentations did not change in a meaningful way from pre-to-post test. These findings suggest that general research capacities, not specific to D&I, should be targeted by future capacity-building activities, particularly grant writing. Accordingly, the TDR WHO has developed a flexible and interactive D&I toolkit to support capacity building and proposal writing.(23)
The need for increased mentoring and guidance was a prominent theme in the recommendations submitted by the participants, as only 14% reported having a mentor in D&I research. Even though every country has a D&I consultant, the limited access to D&I mentors may be a reflection of very few researchers being trained in the emerging field of implementation science in LMICs. Evaluation of other D&I trainings in high resource settings have shown the importance of networking and mentoring, as well as time, for the development of academic outcomes,(8, 24) and previous MOOCs with added support beyond mentorship (i.e., meetings for participants to discuss modules, Q&A sessions, discussion forums) demonstrated an increase in participant engagement.(10, 25) Future D&I capacity trainings in LMICs should include greater mentorship and support throughout the course as it could contribute to higher course completion and improved overall D&I competency reporting.(7, 26)
Limitations
The major limitation of this paper is the small number of participants that completed the course. Additionally, evaluation data was comprised of only self-report data and, therefore subjected to bias and social desirability. Lastly, we did not follow up with participants to ask whether they were able to apply what they learned. The lack of opportunity to practice what they learned has been a challenge described by participants in previous MOOCs.(24, 27)
Future Directions and Implications
Despite the challenges and limitations, partnering with the Special Programme for Research and Training in TDR MOOC is a feasible and scalable strategy to increase D&I training in LMICs. The use of D&I competency metrics allows for further evaluation on how to design training in D&I. In the future, research partners may add specific modules, such as HTN care and D&I grant writing. Some of this is already being done as part of capacity building initiatives.(17, 28) In moving forward, setting up and strengthening a collaborative practice whereby mentoring and peer collaboration across countries could be beneficial to all in enhancing the capacity for D&I research training.(28)