Introduction

Throughout the world, efforts are underway to expand K-12 student participation in computer science (CS). In the U.S., 51% of high schools offer CS while across 19 states, 30% of K-8 schools offer foundational CS courses (Code.org et al., 2021). Although progress has been made, data suggests that opportunity gaps continue to exist in relation to female and minoritized student participation in computing (Code.org et al.). For example, minoritized students are less likely to attend a school that offers CS courses while females are less likely to be enrolled in foundational CS courses (Code.org et al.). At the same time, global events such as the Covid-19 pandemic have further highlighted the critical role of CS knowledge in helping individuals understand data and computer models used to explain the pandemic itself, the economy, and the well-being of citizens (Lee & Campbell, 2020). Further, with existing inequalities exacerbated during the pandemic, understanding of how to use CS to identify the impact of complex societal problems on one’s communities has made computing more personally relevant to students (Lee & Campbell, 2020). In turn, these opportunities necessitate that teachers engage all students with CS in ways that help them address pressing societal issues.

A major challenge in engaging all students with CS is building teacher capacity for equitable, engaging, and rigorous CS instruction. Towards this end, culturally relevant pedagogy (CRP) frameworks which help teachers integrate knowledge relevant to student identities and communities with CS activities are key for enabling effective, meaningful, and equitable learning (Codding et al., 2021; Gay, 2018). Equity-oriented PD prepares teachers to integrate CRP into their teaching by engaging them in self-reflection that requires acknowledging their biases and positionality (Borrero et al., 2018). Yet compared to other subjects (e.g., mathematics), there are fewer opportunities as well as research documenting the design, implementation, and outcomes of equity-oriented CS PD programs (e.g., Codding et al., 2021; Menekse, 2015; Mouza et al., 2018). Importantly, the Covid-19 pandemic necessitated changes in the design and delivery of PD programs, further complicating current efforts to investigate the characteristics of effective CS PD and its associated impacts on teacher learning and practice. In this work, we present the transition from a face-to-face program to a virtual PD institute necessitated during the Covid-19 pandemic. We describe the topics and design principles guiding the virtual pivot and investigate the impact of the virtual PD institute by answering the following questions:

  1. 1.

    How did participation in a virtual PD institute influence teachers’ knowledge and confidence in teaching CS skills?

  2. 2.

    How did participating teachers apply CS content and pedagogy learned during the virtual PD institute in conceptual lesson designs?

  3. 3.

    In what ways, if at all, did participating teachers use or plan to use new resources and strategies from the virtual PD institute in their instruction?

  4. 4.

    What elements of the virtual PD institute were most valued by participating teachers?

Prior Literature: Professional Development in Computer Science Education

The current demand for K-12 CS instruction has given rise to a number of PD initiatives both in the U.S. and abroad. PD is essential for helping teachers build their knowledge of CS content along with high quality pedagogical strategies that will help meet the needs of all students (Goode et al., 2014; Fancsali et al., 2020). Further, PD is important for positively influencing teachers’ confidence or self-efficacy -- their beliefs in their ability to both learn and teach CS (Bandura, 1977; Rich et al., 2021). A growing body of research indicates that teachers’ self-efficacy has implications for teaching effectiveness, instructional practices, and student achievement (Klassen & Tze, 2014). Therefore, efforts to support the teaching of CS should focus on teachers’ knowledge and confidence as well as their ability to apply what they learn in practice.

Findings from Face-to-Face PD Efforts

Existing research indicates that participation in PD driven by a focus on content, pedagogy, active learning, coherence, and sustained duration could facilitate changes in teachers’ knowledge, self-efficacy, and practice in a variety of K-12 settings (e.g., Desimone, 2009; Goode et al., 2014; Rich et al., 2021). For instance, in prior work we have found that participation in a PD program which included an intensive face-to-face week-long institute and follow-up support during the academic year, helped teachers acquire CS content knowledge and a variety of pedagogical strategies for teaching CS (Mouza et al., 2016; Pollock et al., 2017). Further, interview data following participation in PD indicated that teachers implemented content and strategies from the PD in their classrooms, reaching a broad population of students (Pollock et al., 2017).

Findings from PD efforts associated with the popular Exploring Computer Science (ECS) high-school curriculum indicated that participation in PD strengthened teachers’ practice by building their capacity to foster equity, inquiry, and development of CS concepts. In turn, teachers’ practice had a direct impact on students’ attitudes and their engagement with CS (McGee et al., 2019). Similarly, Rich et al. (2021) documented positive changes in teachers’ learning as a result of participation in continuous PD associated with the BootUp K-8 program (https://bootuppd.org/professional-development/). The BootUp PD program extended over the period of one year and focused on coding and computational thinking, both of which are key aspects of CS. Results indicated that participation in the program increased confidence, self-efficacy, and comfort level with coding and computational thinking.

Findings from Virtual PD Efforts

Although these results are promising, the Covid-19 pandemic has forced a number of well-established face-to-face PD efforts to transition online. While there is an extensive body of literature focusing on online PD in other content areas (e.g., Parsons et al., 2019), such work is only beginning to emerge in the field of CS. For instance, Goode et al. (2020), transitioned the well-established ECS PD sessions from face-to-face to online focusing on a learning community model to support continuous participation while addressing barriers related to travel, time, geography, and cost. Findings indicated that the learning community approach can engage participants in learning that supports inquiry and equity-based CS instruction while breaking down the isolation of CS teachers (Goode et al., 2020).

Other efforts to leverage online PD are also reported from scholars outside the U.S. (e.g., Ireland, Australia). For instance, McGarr et al. (in press) describe a PD framework focusing on a community of practice approach to support nationwide CS efforts in Ireland. This approach included a combination of face-to-face and virtual just-in-time support delivered through collaborative platforms (e.g., SLACK), as well as online MOOCs focused on the development of fundamental CS skills. Similarly, Falkner et al. (2018) describe an ecosystem approach to supporting teacher PD in Australia, which includes a collection of MOOCs providing self-paced content and pedagogical learning, development of lead teachers, face-to-face PD events, and a lending library of CS technologies to facilitate integration. Evaluation results indicated that an ecosystem approach can support teacher learning and engagement with CS while strengthening the broader CS community goals.

Findings from Emergency Virtual PD Efforts

While the above studies examined the effectiveness of PD deliberately designed and delivered online, the Covid-19 pandemic necessitated emergency remote instruction, which is meaningfully different (Hodges et al., 2020). Ferdig et al. (2020) provided numerous examples of how educators working with both pre-service and in-service teachers pivoted to online instruction as a result of the pandemic, but few of these studies focused on CS PD specifically (e.g., Albert et al., 2020).

Examining their transition to virtual PD specifically in response to Covid-19, Jocius et al. (2021) describe how they shifted an established PD program to online focusing on the integration of computational thinking with English Language Arts (ELA). This program included face-to-face summer workshops and follow-up academic year activities modeled after the 3C approach (code, connect, and create). The online transition used a combination of synchronous and asynchronous activities delivered through various technologies (e.g., video conference, chat), online materials mirroring the 3C approach, opportunities for collaborative activities that supported engagement with content, and focus on practice and reflection. Follow-up activities were provided through monthly webinars focusing on CS content. Findings indicated that teachers infused CT into existing ELA curricula, albeit they differed in the manner in which they appropriated and adapted pedagogical tools for CT infusion.

In this work, we describe how we transitioned our face-to-face PD program to online delivery at a time where both the emotional strain and workload of teachers made it challenging to focus on CS in light of pressing educational challenges (Code.org et al., 2021; Jocius et al., 2021). We subsequently examine the impact of the virtual PD institute on teachers’ knowledge and confidence in teaching CS, application of CS in lesson designs, and PD features most valued by teachers.

Methods

Context of this Work

This work is situated in a larger effort to improve the teaching of computing in the U.S. through effective PD. Our PD program incorporates a three-tiered approach, which typically includes: (a) an annual face-to-face week-long summer institute, (b) a college field-experience course in which undergraduate students with background in CS assist teachers in developing and implementing CS lessons back in their classrooms, and (c) sustainable partnerships with K-12 schools (Pollock et al., 2015). The week-long summer institute focuses on preparing teachers in grades 5-12 to teach standalone CS courses or integrate foundational CS principles into existing content area instruction. These principles include creativity, abstraction, data, algorithms, programming, Internet, and impacts of computing (College Board, 2017).

The design of the institute is grounded in the framework of technological pedagogical content knowledge (TPACK), which describes the knowledge needed for the effective integration of technology in teaching (Mishra & Koehler, 2006). It includes explicit attention on CS resources and tools (technology), CS principles (content), pedagogy for teaching CS (pedagogy), and culturally responsive strategies for broadening participation in computing (Codding et al., 2021; Scott et al., 2010).

In spring 2020, as Covid-19 forced school closures, we quickly realized that we would not be able to offer our annual face-to-face summer institute. Following discussions with our local chapter of the Computer Science Teachers’ Association (CSTA), we decided to shift our summer institute to an institute series of bi-weekly 2-hour virtual workshops, delivered synchronously via Zoom, in fall 2020. A total of five workshops were offered between October - December for a total of 10 instructional hours. Similarly, as summer 2021 approached it again became clear that a face-to-face institute would not be possible. Therefore, we pivoted to a virtual summer institute offered over a period of four days. The design of the summer institute was informed by feedback on the content, duration, and instructional activities provided by fall participants. Each day, teachers met synchronously via Zoom from 9 a.m. – 12 p.m. for a total of 12 instructional hours. All sessions were delivered by members of our research team or teachers who had developed leadership and expertise through their participation in our prior PD efforts.

In addition, teachers participating in both the fall and summer virtual institutes devoted independent time for examining resources and tools and designing lessons that integrated CS with core curriculum content. This approach allowed us to capitalize on a combination of synchronous activities focused on new content and peer interactions with asynchronous activities focused on practice and reflection (Jocius et al., 2021). Further, technical assistance was available throughout each day of the summer institute in a separate Zoom room, which allowed teachers to receive one-to-one support or ask questions at any time. Such assistance was provided on-demand in the fall.

Table 1 provides a snapshot of the content, delivery, and rationale of our virtual institute in fall 2020 and summer 2021. Unlike our face-to-face PD program, no follow-up classroom support was provided. Due to our university classes also moving online, we were unable to offer the field-experience course.

Table 1 Content, delivery, and rationale of transition to online PD
Full size table

Participants

Participants included 15 teachers who attended the fall virtual institute and 27 teachers who attended the summer virtual institute, for a total of 42 teachers. Of those, 21 (50%) teachers had previously attended our face-to-face PD programs. Those teachers taught in a variety of grade levels and schools throughout a U.S. state. Table 2 presents demographic information on participating teachers. ParticipantsFootnote 1 were fairly seasoned teachers, with the majority of them having 16+ years of experience. However, as shown on Table 2, their experience in CS was much shorter. Only five teachers had 6+ years of CS experience. Further, only thirteen participants taught CS as their primary discipline while the rest were interested in integrating CS across other areas. All teachers voluntarily participated and were compensated for their time.

Table 2 PD participant demographics
Full size table

Design and Data Collection

The study used a convergent parallel mixed methods design (Cresswell & Plano Clark, 2017). Data for each research question were collected from different sources including pre and post surveys, lesson plans submitted at the end of each institute, and interviews. Qualitative and quantitative collection and analysis proceeded separately, with integration happening at the point of analysis.

Surveys

Data were collected using both quantitative and qualitative procedures. Participants completed electronic surveys, administered through Qualtrics, prior to the first PD session and after the last session. These surveys were designed to gauge participants’ level of knowledge and confidence in teaching CS before and after each institute. The post-survey also included both closed- and open-ended questions to measure participant satisfaction and perceptions of the PD, as well as what they had learned in it. Survey items were based on research discussed above about effective PD (e.g., “This PD workshop was tailored to my needs as a learner” or “Sufficient time was provided for guided practice and tasks”). There were 15 responses to the pre- survey and 8 to the post- survey in fall 2020. There were 25 responses to the pre- survey and 22 to the post- survey in summer 2021.

Lesson plans

As a culminating activity in both the fall 2020 and summer 2021 institutes, participants developed conceptual lesson plans integrating CS with content area instruction (N=22). Conceptual lesson plans refer to lessons that teachers planned to use in their own classrooms following their participation in PD (Codding et al., 2021). Participants developed the lesson plans independently (N = 18) or with a partner (N = 4). These plans were used as artifacts to demonstrate participant learning and application of PD concepts. Specifically, teachers were asked to indicate the target audience (e.g., grade level), content area, lesson goals, CS standards, required technologies, and learning assessment. Teachers designed lesson plans that could be used for multiple content areas including standalone CS courses (N=15), world languages (N=3), mathematics (N=2), social studies (N=1), and ELA (N=1). Lesson plans covered a wide range of CS topics ranging from block-based coding to create stories on Scratch to writing algorithms for playing rock, paper, scissors as an unplugged activity.

Interviews

Interview data helped obtain in-depth information on teachers’ experiences. All teachers who attended the fall 2020 institute were invited to an interview. Of those, 8 participants (53.3% response rate) responded and were interviewed in spring 2021. Of the teachers attending the summer 2021 institute, a purposeful sample of 14 new and returning participants from diverse school settings was invited for an interview after the PD. Out of those invited, 9 responded (64.3% response rate) and were interviewed at the end of the institute. Therefore, a total of 17 interviews were conducted. The interview protocol explored how teachers responded to the PD, what they had learned in it, and whether and how they had implemented or planned to implement strategies, practices, or resources shared in the PD. Interviews took place on Zoom and were recorded and transcribed. All interviews were conducted by an external evaluator associated with a research center responsible for the overall PD evaluation. This allowed participants to freely express their perspectives of the PD without trying to please PD providers.

Data Analysis

Both qualitative and quantitative analysis were conducted depending on each data source.

Surveys

Frequencies were calculated for all closed-ended items and means and standard deviations were also calculated for Likert scale ratings. Paired sample t-tests were conducted on the 21 items that appeared on both the pre- and post-PD surveys. There were a total of 28 matched pairs (i.e., where the individual completed both pre- and post-PD surveys) used in the analysis. Open-ended survey responses were coded by question, using an inductive, grounded theory approach (Saldaña, 2021; Corbin & Strauss, 2008).

Lesson Plans

Conceptual lesson plans were analyzed to examine the ways CS content and pedagogical practices were infused in the conceptual lesson plans after participating in the virtual PD. For this analysis, we developed a codebook drawing from two sources (CS for all, n.d.; Lytle et al., 2019). Codes were grouped into two categories that addressed CS principles and CS pedagogy, including specific scaffolding activities modeled after the use-modify-create framework (Lytle et al., 2019; see Table 3).

Table 3 Codebook for lesson plan analysis
Full size table

Interviews

Interview transcripts were coded in Dedoose (Socio-Cultural Research Consultants, 2021), using a mixture of a priori codes based on previous work (see Codding et al., 2021) and codes that emerged inductively. The goal was to identify themes salient across participants.

Findings

Findings indicated that participation in a virtual PD institute positively influenced teachers’ knowledge and confidence as well as application of learning into practice. Further, teachers valued many aspects of the virtual PD.

Impact of Virtual PD on Teacher Knowledge and Confidence

Survey data provide insight into teachers’ knowledge, skills, and confidence in CS. An item on the post survey asked participants to rate their knowledge and skill in CS both before and after participation in the virtual PD institute (Table 4). Results indicated self-reports of growth. The mean rating before the PD was 3.03 (SD = 0.82) compared to 3.79 after the PD institute (SD = 0.56). A paired samples t-test indicates that this difference is significant, t (28) = -5.93, p(two-tailed)<0.001. After participation in the virtual PD institute, every participant rated themselves at least average with the largest group (65.5%) selecting “above average.”

Table 4 Ratings of knowledge and skills before and after the workshop
Full size table

Post-survey responses also indicated higher confidence teaching CS principles (Table 5). Specifically, participants’ ratings of their confidence in teaching CS skills overall, as well as teaching skills related to each CS principle, were higher on the post-surveys compared to the pre-surveys. All increases were statistically significant. The largest gain was seen for the principle of programming, and the highest ratings both before and after participation in PD were for creativity.

Table 5 Confidence before and after PD institute - results of paired sample T test
Full size table

Surveys and interviews also provide some qualitative insight into how participants gained knowledge, skills, or confidence related to teaching CS. Specifically, as they described their PD experience, participants often indicated how their own understanding had grown or how they now felt more confident or equipped to bring CS concepts into the classroom. For example, one teacher contrasted the summer institute to a previously attended PD opportunity:

[In another workshop] I feel like I didn’t learn anything. I was just sitting there. And this time I did learn things. I do have materials for planning my entire year next year. And I feel like I have a foundation to start with. When [cybersecurity presenter] shared resources with us, I told her, ‘I’ve been a year researching...and I couldn’t find anything I really liked.’

Participants brought varied prior experience related to CS (see Table 2) but the data suggest that both novices and veterans reported growth in their knowledge and confidence. For instance, survey comments focused on CS content as the best aspect of the institute came from someone calling herself “a VERY NEW beginner in the field” and from someone who appreciated “understanding advanced computer science information.” Some participants identified specific CS content or topics they appreciated learning, especially artificial intelligence and cybersecurity, as well as programming with specific tools, such as Micro.bits and Scratch.

Application of CS Content and Pedagogy in Conceptual Lesson Designs

Participation in a virtual PD institute helped teachers form a conceptual understanding of CS and take steps toward infusing CS in their lesson plans. The construction of lesson plans was guided by the CS standards teachers selected based on relevance to their learning goals and outcomes. As shown on Table 6, teachers were able to articulate their understanding of CS by integrating CS principles and CS pedagogical strategies into their conceptual lesson plans.

Table 6 Code distribution in lesson plans
Full size table

CS Principles

Teachers incorporated all seven CS principles into their lesson plans. The most frequently used principles were programming (N=13), algorithms (N=15), abstraction (N=7), and creativity (N=7). Jane and Emma, who taught at the same public elementary school, planned to use Scratch to promote programming and student creativity. Specifically, after having students investigate the various functions of Scratch, they planned to have them program an animation to creatively introduce themselves. In another lesson, Lena utilized algorithms as a way of guiding students through cryptography. Lena’s plan placed students in small groups to encrypt and decrypt messages based on codes they created. Kara’s lesson utilized algorithms through CS unplugged. In this lesson, students were to write instructions for their classmates related to making a paper plane, which they tested and modified. The least frequently used principles were data (N=4), Internet (N=3), and impacts of computing (N=2). Bianca, for instance, merged social studies with CS to create a lesson plan about the importance of data and how to manipulate them. The goal was for students to clean, filter, and slice a dataset about world happiness on Google Sheets to create visuals based on student-generated questions.

CS Pedagogy

Analysis revealed the integration of four CS pedagogical strategies into lessons – collaborative learning, CS inquiry, scaffolding, and CS unplugged. These strategies are critical for addressing equity concerns (Goode et al., 2014). Collaborative learning was the most commonly used strategy (N=12). Specifically, teachers planned to use strategies such as pair programming or group work to promote collaboration. For instance, Sue’s lesson configured learners in pairs to create a skit on Scratch using world languages. Similarly, CS inquiry was also commonly found in lesson plans (N=11). Teachers’ lessons frequently encouraged students to experiment with programming through open-ended tasks. Sam’s lesson, for example, engaged students in reviewing programming concepts as a way of developing sequences and simple loops to address problems using java script. The goal was for students to engage with a video-game task in order to navigate a hero character through various challenges using efficient strategies.

Results indicated that teachers (N=14) integrated three types of scaffolding in their lesson plans to support CS instruction, reflecting the use-modify-create framework (Lytle et al., 2019). Specifically, some lessons started with using pre-existing artifacts, such as an algorithm or CS resource (N=3). Emma, for instance, designed a cybersecurity lesson that engaged students in decoding hieroglyphic messages as a way of understanding how to use code to write and send messages. The next scaffolding phase involved using and modifying pre-existing artifacts to suit students’ needs as designers (N=3). In this phase, Emma’s lesson asked students to practice writing and modifying hieroglyphics to produce coded messages. The final scaffolding phase, create, moved students away from using and modifying existing artifacts to creating new ones (N=8). In this phase, Emma’s lesson engaged students in creating strong Internet passwords based on lessons learned from encrypting and decrypting hieroglyphic messages with their peers.

CS unplugged was the least incorporated strategy (N=6). While some lessons were built entirely around CS unplugged, others incorporated unplugged activities as a way to introduce the lesson and relevant concepts without the use of technology. For example, Holly created a programming lesson using Micro:bits. Prior to coding, Holly’s lesson had students work in pairs to explain the rules for playing the popular game of rock, paper, scissors following a step-by-step approach (i.e., algorithm). Students were to compare and troubleshoot any missing or confusing steps, replicating the Micro:bit activity. Using pair-programming, students were to subsequently program their devices to play rock, paper, scissors and keep track of scores on Google Sheets.

Teachers’ Use/Plans to Use New Resources and Strategies from the Virtual PD

In interviews, teachers expanded on classroom implementation of resources and strategies they had learned in the virtual institute. Of the 17 individuals interviewed, 14 could describe an activity that they had implemented or planned to implement with their students or a strategy or resource they had discovered. A difference between the cohorts is relevant here. Interviews for the fall 2020 cohort occurred in spring 2021, allowing more time for actual implementation, while interviews for the summer 2021 cohort occurred shortly after the PD institute. Thus, it is not yet known whether their plans for implementation were carried out.

Most frequently, what had been implemented entailed projects using the Scratch programming language: “[Scratch] was the only thing I actually used in practice. We used Scratch and they had to tell a story about themselves and how the pandemic was affecting them personally.” One participant described implementing a CS unplugged “barcode activity for teaching algorithms.” This activity, introduced during both institutes, helps teachers understand how the barcode checkout system works. Another teacher described learning strategies for successfully implementing Code HS curricula (although they noted these ideas came from informal discussion, not planned agenda items): “There were some other teachers that were using it, and that sharing of ‘how did you make this work? How do you work around that?’”

When summer 2021 participants described concepts they planned to bring into the classroom in 2021-22, the most common were artificial intelligence (AI), programming with Micro:bits, and cybersecurity. There were many mentions of CS curriculum resources including Code.org CS unplugged activities. Two differences were observed in teachers’ narratives. First, some participants simply listed topics (e.g., “well, Micro:bits are cool”) without providing adequate context related to their implementation, while others laid out concrete and elaborated plans: “we’re going to go over the negatives and the positives of AI. We’re going to teach AI and [their final project] will be to create a presentation choosing something in real life that could be addressed with AI.”

Second, interviews included a mix of teachers who were new to the virtual PD institute and those with experience in prior years (face-to-face). Newcomers were more likely to identify model resources or strategies while veterans usually spoke in terms of enhancing or deepening existing practices, including ideas they had previously gleaned from the institute. For instance, a first-time participant said, “some of the activities with different programs that were shared, I can utilize and change up ways in which I do certain things to make it fun for students. So, utilizing Scratch for instance, to help my students introduce themselves.” A veteran planned to “continue using the Explore model that Code.org uses for having students explore some of the program..[and] practices that we’re already using as far as pair programming” but to also strengthen another pedagogy “really taking the time to sit down and give the students the opportunity to summarize what they’re doing.”

Although it was not an explicit question, many teachers raised the pandemic context. Most fall participants described how Covid-19 had constrained their classrooms and their prospects for implementation of ideas from the PD. These challenges included virtual or hybrid environments, shortened or canceled CS class time, and limitations on group work or shared supplies including technology. Summer 2021 interviewees were generally hopeful about the return to in-person learning but were unsure about what protocols would be in place and the implications for CS instruction.

Valuable Elements of Virtual PD

Participants’ ratings of the overall institute quality were strong. Everyone who completed a post-survey (N=30) rated the institute “average” or better (M=4.45, SD:0.63), and over half (51.7%) called it “excellent.” Further, participants rated almost every aspect of the PD positively (see Table 7). Out of 18 statements, 16 had average ratings of at least a 4.00, meaning an overall level of agreement. Only ratings of facilities and time did not (likely reflecting limitations of the online environment, discussed below). The highest-rated items pertained to relevance and learning.

Table 7 Virtual PD institutes feedback
Full size table

When teachers identified what they found most valuable about the PD on the post-survey, 6 common themes emerged (see Table 8). Many recurred in interviews, with participants emphasizing the importance of (a) exchanging curriculum ideas, tools, and resources; (b) professional collaboration and networking; and (c) learning CS content. These themes were highly interwoven.

Table 8 Most valued aspects of the virtual PD institutes
Full size table

Teachers reported gaining insight into new resources and tools, as well as learning “practical tips” for teaching CS. One wrote, “Every workshop offered a real opportunity to implement what we learned in our classroom.” An interviewee pointed to the sessions on cybersecurity and artificial intelligence, saying both facilitators “had very useful resources, so I could just pick it up and I could start teaching it.” Another — also referencing cybersecurity—commented: “I didn’t know about that, and I’m like, ‘oh, I want that!’ Because that will get the kids to understand!” Generally, participants found a strong match between the institute’s resources and their students’ needs.

Sharing resources in this way also nurtured collaboration and professional relationships. One teacher wrote, “Hearing other teachers’ ideas and experiences helped to give me a better understanding of ways I could use certain activities/ideas in my classroom.” Another explained how the institute helped break down the isolation often experienced by CS educators:

I love being able to communicate with other educators that are in the same computer science field because often you are kind of alone when you’re in these schools. You know, [if you teach] math, you can talk to all the other teachers that are teaching math . . . but technology [cannot do that].

The challenging context of Covid-19 made peer support especially important. For instance, one teacher valued “listening to other educators and how they are teaching during a pandemic.”

Breakout rooms allowed teachers to deepen their CS content knowledge and work with those with similar teaching contexts or interests. For example, one high school teacher valued being able to “sit down and actually practice some of the lessons that we’re getting ready to do in the AP Computer Science Principles.” Parallel sessions also offered variety: “It wasn't like you just had one track to follow. You could bounce around between different rooms, and find out, get more information.” This ability to “bounce around” with a single click was one affordance of virtual PD. The virtual format also increased access, especially for geographically far-flung participants, and some found it convenient and productive. One teacher who had attended both formats found online “more focused with fewer distractions. It seems, when we're all in a room together, not that I don't want other people to speak, but sometimes we go off the rails.”

Yet the data also reveal the constraints of online PD. Some teachers were simply “sick of [virtual]” or felt that “obviously in person is better.” Others found learning CS concepts through a screen challenging and felt that in “face-to-face...you can get better understanding.” Finally, virtual PD was by necessity shorter in scope and duration than the face-to-face institute. Some participants expressed the need for more time or a slower pace.

Discussion and Implications

Despite growing attention to PD, we continue to know little about the CS concepts that teachers succeed or struggle with, the pedagogies they engage with, their self-efficacy, and the manner in which they apply PD learning into their practice (Pollock et al., 2017; Rich et al., 2021). Findings from this work indicated that the virtual PD institutes achieved several important successes, despite highly demanding circumstances and the complexities of the pandemic. Specifically, teachers reported increased confidence and preparation to teach CS, particularly regarding programming and algorithms. These findings could be attributed to the increased attention of the virtual institutes on foundational concepts related to programming and algorithms. Building teachers’ confidence in foundational CS concepts, such as programming and algorithms, is key to their future success; literature suggests that the most common challenges noted by CS teachers is their own CS subject matter knowledge (Sadik et al., 2020; Sentance & Humphreys, 2018).

Further, analysis of conceptual lesson plans indicated that teachers incorporated CS principles in their curricula. The most widely utilized principles again included programming and algorithms while CS principles related to abstraction and creativity were more limited. These findings are not surprising since these concepts received less attention during the virtual institutes. Further, extensive practice with programming may be needed before teachers feel confident implementing projects that rely on abstraction (Rich et al., 2021). Data, Internet, and impacts of computing received even less attention. However, impacts of computing were typically woven in lessons in the form of pedagogy. In fact, lesson analysis indicated that all teachers included CS pedagogical strategies including collaboration, inquiry, CS unplugged, and scaffolding. Research indicates that such practices are critical for addressing equity concerns, engaging students with real CS learning, and broadening participation in computing (Goode et al., 2014).

Similarly, interview data revealed that several teachers applied content, pedagogy, and technology modeled during the PD in their classrooms following their participation in the virtual institutes. This finding is important given the increased focus on core content area instruction during the pandemic. Some teachers, however, described implementation challenges associated with the pandemic, including time constraints. This finding is consistent with other reports which indicate that teachers temporarily suspended CS instruction during the pandemic (Code.org et al., 2021). Of those teachers who applied PD learning into practice, novice teachers utilized more resources or strategies shared during the institutes while more experienced teachers focused on deepening existing practices. This finding is consistent with research indicating that teachers typically adopt, use, and modify PD content and pedagogy depending on their cultural and social contexts (Grossman et al., 1999). Specific to CS, findings are consistent with prior work indicating different approaches to the uptake of CS content and pedagogy introduced during the PD (e.g., Jocius et al., 2021; Rich et al., 2021).

Finally, results indicated that participant satisfaction remained generally high and all data sources point to the particular value that participants found in the PD community of CS educators. The opportunity to collaborate and share resources and ideas was also important. Some veterans expressed gratitude that the community was able to reconvene during the pandemic. These findings are consistent with prior research which indicates that teachers find value in collaborating and networking with peers (e.g., Goode et al., 2020). Further, the opportunity to learn new CS content was highlighted by participating teachers , both new and returning, indicating the need to differentiate instruction through multiple tracks or break out sessions that allow teachers to continue building their expertise.

Implications for Practice

Findings from this work have implications and can inform the design of effective online CS PD programs aimed at broadening participation in computing. Specifically, two implications for practice are evident.

Differentiate Instruction

Findings indicate that participants expressed greater interest for breakout sessions and curriculum resources that closely match their teaching context and level of experience. If participants are welcome to return to the PD for multiple years, it is important that they are able to engage in continuous and sustained learning and exploration of new materials, tools, and resources. Similarly, novice teachers should be allowed the time and space to learn foundational skills and feel confident about their ability to engage diverse students in CS. Research indicates that teachers who are confident in their ability to teach CS are better equipped to implement PD learning into practice and spark their students’ interest in CS (Code.org, 2018; Google Inc. & Gallup Inc. 2016). Towards this end, providing choices through parallel sessions that help teachers deepen their understanding of both content and pedagogy is important. In this work, we created different tracks to continue supporting experienced teachers. Further, our help desk in the summer of 2021 was intended to provide a resource for those teachers who needed extra support throughout the PD.

Capitalize on the Strengths of Online Instruction and Strategize about its Constraints

Like other scholars (e.g., Jocius et al., 2021), we were initially concerned that transitioning our face-to-face institute into a virtual format would limit participants’ opportunities to learn and openly share practices. However, findings indicated that teachers valued several online components, including the flexibility to move from one breakout room to another as well as share and collaborate with other teachers. Further, the online format enabled participants from different regions of the state to attend and enriched the diversity of experiences shared within the group. It was reassuring that almost 50% of teachers reported peer collaboration as their most valued aspect of the virtual institutes. Nonetheless, it is important to note that many participants met each other in prior face-to-face PD offerings and had the opportunity to build community. Therefore, just like the PD reported by Goode et al. (2020), the social presence created throughout the institutes had its foundations in the face-to-face meetings.

Despite the successes, engaging teachers in hands-on programming activities was more challenging online and it frequently took longer to troubleshoot and support novices. Towards this end, just-in time support should be readily available through private meeting rooms that make it easier for novices to voice their questions. Further, online videos and other code visualization tools could be useful for building teacher knowledge of foundational CS concepts (Qian et al., 2018).

Implications for Research

As noted, the background and CS preparation of teachers participating in the virtual institutes varied and our findings did not differentiate based on teacher characteristics. Future research should examine how teacher background characteristics, such as CS knowledge, confidence in teaching CS, prior PD experiences, and initial preparation, may influence their PD experiences. Given that most CS teachers come from different disciplines and backgrounds, identifying how PD experiences influence diverse sets of teachers is key. Similarly, examining whether PD experiences differ based on whether teachers teach standalone CS courses compared to integrating CS in other disciplines will also be important. Furthermore, future research should examine the manner in which teachers apply sound pedagogical strategies in their classrooms, including strategies that connect with the realities of the pandemic. Many of the examples provided by teachers regarding applications of computing to address impacts and community-based issues pertained to the pandemic (e.g., computing applications that facilitate contact tracing). Therefore, to sustain student CS engagement in the immediate future it may be valuable to identify ways in which the teaching of CS can connect to community issues associated with the ongoing Covid-19 pandemic.

Limitations

There are three limitations associated with this work; one related to the PD and two related to the research itself.

PD Limitation

We recognize that the virtual institutes provided a marginally sufficient number of hours recommended by best practices in teacher PD. Specifically, research indicates that short PD experiences lasting less than 14 hours show no statistically significant effect on student learning (Darling-Hammond et al., 2009). Further, PD associated with increased student achievement typically spreads out over 6-12 months. While our virtual institutes included approximately 15-20 hours of synchronous, asynchronous, and independent work, these hours were not spread out over time and could not include the year-long follow-up support available during the face-to-face PD institute. Although we documented positive outcomes as a result of teachers' participation in a virtual PD institute, at this time we do not have evidence of sustained implementation of CS in practice. However, it is notable that 21 participants engaged in continuous PD through their participation in our prior face-to-face institutes, in some cases over multiple years.

Research Limitations

Participating teachers included a combination of experienced and novice CS teachers. Due to the small sample size our analysis did not disentangle findings based on prior experience or PD participation. Additionally, interviews for summer 2021 participants were completed at the end of the institute. Therefore, it is not known whether participants applied learning from PD in their classrooms following their participation in the summer institute.

Conclusion

As the interest in CS education is growing, there is enormous pressure among K-12 systems to offer CS coursework. However, there is tremendous need for K-12 teachers with the background, preparation, and experience needed to teach CS using pedagogical practices found to broaden participation in computing. While research on face-to-face PD programs with the potential to impact teacher learning and practice has recently emerged, the Covid-19 pandemic necessitated changes in the manner in which such programs were delivered. With the continuous challenges and uncertainty of the pandemic, it becomes essential to examine how online PD approaches can leverage the affordances of technology to prepare CS teachers. In this paper, we examined teacher outcomes as a result of their participation in virtual PD institutes. Findings indicated that teachers benefited from participation in the virtual institutes improving their confidence in teaching CS, application of CS in lesson designs, and implementation in practice immediately following their participation. Further, findings point to implications for the effective design of online PD which helps build teachers’ understanding of CS content, pedagogy, and technology.