Over the past decade or so, numerous reports have reflected concern among policymakers, practitioners, and researchers that the USA is falling short in producing a next generation of science, technology, engineering, and math (STEM) talent to replace those who will soon retire (Business Roundtable 2005; Council on Competitiveness 2008; National Academy of Sciences 2007; National Research Council 2008; National Science Board 2012; National Science Foundation 2010; President’s Council of Advisors on Science and Technology 2010). Of particular national interest is the need for a homegrown STEM workforce development pipeline to fill strategic jobs in secure national laboratories, defense agencies, and other organizations that require US citizenship (Casey 2012). These demands are compounded by the fact that it takes more than a decade to produce a worker capable of filling a high-level scientific research and engineering position, and at least four technicians and technologists to support each working scientist. Moreover, apart from the demand for dedicated STEM professionals, it is increasingly apparent that STEM skills are vital to every sector of the modern economy. In responding to these requirements, our nation’s policymakers have challenged US K-12 education systems to help young people develop the STEM capabilities necessary to fulfill these workforce needs in order to keep our future robust, our economy growing, and our nation safe.

Theoretical Foundations

During the past 60 years, career development theorists have made substantial advances in terms of seminal research on career development, helping to shape current understandings of how people develop awareness of and interest in careers, as well as how they prepare for those careers. Developmental theorists generally concur that career development proceeds along a continuum of iterative experiences, in which individuals develop, assess, refine, and act on their career interests, knowledge, and skills, and further, that occupational decision making is a process that addresses complex issues of social and psychological development (Crites 1969; Ginzberg 1972; Ginzberg et al. 1951; Havighurst 1953; Holland 1973; Roe 1957; Super 1951). They are also generally in agreement regarding how this development leads to adult career choices.

In light of the complexities and rapid rates of change in the twenty-first century world in which we live, modern career development theorists have contributed to collective understandings by examining the importance of social interactions and cultural context on career development. Social cognitive career theorists (e.g., Lent et al. 1994) further expanded this perspective to recognize the mutually influential relationship between people and their environment.

With roots in social learning theory (McIlveen and Schulthesiss 2012), today’s social constructionists view career as the result of one’s interactions with others in a larger social, political, and historical context (Young and Popadiuk 2012). They propose that an individual’s career is a result of the discourse in which one engages, interactions with one’s communities of practice, and the systems within which one lives. The life design approach theory (Savickas 2005) posits that individuals construct who they are by developing their social identity as they interact with society (Staunton 2015) and that individuals are authors who, when helped to reflect on the patterns and life themes of their own stories, connect with work settings to construct and reconstruct their careers (Savickas 2005, 2011, 2013; Savickas and Hartung 2012).

Developmental contextualists argue that even though humans are products of biological predispositions and psychological constructions, they are embedded in contexts that shape their developmental pathways (Vondracek et al. 1986; Vondracek 1998; Porfeli and Vondracek 2009). This group further notes that twenty-first century career success is related to the ability of individuals to handle the complexity and change of modern career pathways, the balance of family and work, and the redefinition of “career” found in retirement. In a living systems theory of vocational behavior, Vondracek et al. (2014) propose that each individual can be viewed as a living system (a dynamic, self-directing, self-constructing entity) that interacts with a variety of contexts.

In the modern, industrialized world of two-career households and shared family responsibilities, work contexts are viewed more broadly. Richardson (2012) integrates feminist and social justice perspectives into the discourse on social constructionism of careers by drawing attention to four social contexts within which work occurs: (a) market (paid) work; (b) personal care work (caring for self, dependents, significant others, and community); (c) relationships, in both market work and personal care work; and (d) the social inequities inherent in each. Her work describes how people construct their lives through these contexts. She proposes that twenty-first century students should broaden their understanding of “work” to include personal care work and should construct and reconstruct their life stories to achieve a healthy work–life balance.

Exploration is understood to be an important component of self-constructed identity formation and a cornerstone of career development (Flum and Kaplan 2006), associated with qualities that are much needed in today’s rapidly changing world of work: flexibility and tolerance for ambiguity; a sense of agency, self-determination, autonomy, and successful adjustment; and openness to new experiences. This schema posits that exploratory skills must be learned and that education systems should support mastery goals for exploratory orientation. Renninger and Hidi (2002) associate exploration and interests, suggesting that when students engage in content related to their individual interests, they are pursing mastery goals for exploratory orientation and are better able to concentrate and perform difficult tasks (Renninger 2000). Therefore, when school systems provide opportunities for K-12 students to explore and develop interests in STEM content, students are provided with the necessary foundation to pursue higher-level STEM courses and eventually STEM careers.

Considering these concepts—the developmental nature of career development, the importance of the relationship between individuals and their environments in constructing their career futures, and the benefits of exploration—in light of today’s complex world driven by technology, what should the continuum of STEM career development experiences look like for individuals moving through the US education system? More particularly, how might these ideas be translated to advance program implementation and evaluation research in federal grant-funded programs intended to support STEM career development experiences?

Evolution of an ITEST Workforce Education Model

In June 2012, the National Science Foundation’s (NSF) Innovative Technology Experiences for Students and Teachers Learning Resource Center (ITEST LRC) convened a STEM Workforce Education Working Group, consisting of principal investigators and evaluators from the foundation’s ITEST program. This group worked remotely, and in face-to-face meetings facilitated by staff from the ITEST LRC, to consider what a common, high-level, ITEST STEM workforce education model might look like. They further examined what types of evidence ITEST projects should be collecting in order to assess ITEST’s contributions to STEM workforce education. After extended discussion of ITEST program elements and intended outcomes, and a review of the literature on workforce education models and career development, the LRC working group proposed a conceptual model to help visualize how major elements of the ITEST program are anticipated to come together to realize STEM content and career outcomes for students. Based on their work, the group chose to focus on five elements consistently prioritized by funded ITEST projects: (a) STEM content development activities, (b) STEM career development activities, (c) partnerships (e.g., with business entities), (d) educator professional development, and (e) cultural context. The result was the ITEST STEM Workforce Education Helix conceptual model (Fig. 1). The “helix model” is designed to illustrate the iterative relationship between STEM content development and STEM career development activities, implemented as students advance from kindergarten through high school within the cultural context of schools, with teachers supported by professional development (PD), and through programs supported by effective partnerships.

Fig. 1
figure 1

ITEST STEM workforce education helix

Full size image

In August of 2014, the NSF STEM Learning and Research Center (STELAR), a continuation of the ITEST LRC, convened a new Data & Impact Working Group to extend their earlier efforts. This reconstituted ad hoc organization was charged specifically with exploring the question of what types of data STEM education projects should be collecting. Reflecting on the iterative nature of career development and the collective experience of ITEST projects focused on STEM workforce education, the group also created a simple matrix that clarifies and explains four specific categories of outcomes commonly targeted by ITEST projects, all theoretically linked to student persistence in STEM education and progress toward STEM careers. These categories include (a) dispositions, (b) knowledge, (c) skills, and (d) actions, with the resulting STEM Outcomes Matrix (Fig. 2) illustrating how each might be represented in both STEM content and career outcomes as illustrated in the helix model. The LRC working group was purposeful about the outcome matrix further proposing a simple theory of action across the outcome categories, that action outcomes (e.g., persistence) are predicated on students gaining the right combination of dispositions, knowledge, and skills from their learning. Naturally, this framework may be generalized across any STEM content or career domain, but Fig. 2 presents an example considering biology (content) and engineering (careers), noting that rows are color coded to match the helix.

Fig. 2
figure 2

STEM outcomes matrix with example outcomes

Full size image

Considering the range of theoretical orientations described earlier, the helix and matrix models are consistent with the principle ideas that (a) STEM career development is an iterative process, and (b) STEM content and career outcomes build upon one another in nonlinear and recursive ways as young people move along diverse pathways toward STEM careers. Examining these ideas through the collective experiences of ITEST grantees (including the five projects examined as Sample Project Cases for this work), the LRC working group has confidence that their models are effective conceptual elaborations of (a) how students progress iteratively through STEM workforce education activities (the helix), and (b) how outcomes for those students might effectively be defined and categorized (the matrix). Both should be useful schemata that program designers can apply during the planning and implementation stages of workforce development projects similar to those supported by the NSF’s ITEST program.

However, these models come up short in terms of utility for clarifying specific theories of action that must be explicated and tested by program research and/or evaluation efforts—at least to the extent that those studies assess not only the timeliness and quality of program activities and outputs but also generate broader understandings relating to how those activities result in desired outcomes. (Note that we invoke this purpose to make the terms “evaluation” and “research” operationally synonymous for the purposes of this discussion; we make no further effort here to define how they might otherwise be the same or different.) The programmatic goal of evaluation for this purpose requires collecting and analyzing data to test specific theories of how STEM content and career development activities interact with typical elements of teacher professional development and partnerships, and with factors from a project’s cultural context. One tool typically applied to this sort of consideration is the logic model.

About Logic Models

Models are abstractions developed to serve specific purposes, such as the planning of an architectural build or development of an airliner using computer-aided design. The purpose not served by the helix and matrix, but crucial to design and evaluation of STEM workforce education programs, is clarifying the theoretical relationships among factors thought to influence the effectiveness of the education pipeline in meeting workforce needs in STEM and STEM cognate career areas. Viewed broadly, any theory of how the current workforce education system “causes the intended or observed outcomes” (Rogers et al. 2000) should help efforts to strengthen the pipeline to be successful—accepting that the word “causes” may be an overstatement, given the state of true impact research in this area.

Logic models may be tabular or presented graphically (Kellogg Foundation 2004; Weiss 1997; Wholey 1987), as visual representations of relationships among factors related to the problem at hand (Renger and Titcomb 2002). To be effective, logic models should illustrate not only these elements but also the linkages among them (McLaughlin and Jordan 1999), explicating a theory of how they interact to effect change (Chen 1990). As a convention, these connections generally act from left to right, illustrating a series of if–then propositions among the elements.

Ultimately, the clarity of program theory that results from use of an effective logic model should help improve both the quality of implementation and the impact of programs like those typically applied to meet STEM workforce education needs (McLaughlin and Jordan 1999). Shared understanding of a program’s theory of action empowers communication, team cohesion, the dissemination of ideas, and the identification of assumptions (either accurate or unfounded). Explicit program theory can help evaluators and managers identify projects, or elements of projects, that are either crucial to attaining outcomes and thus should be retained, or are redundant or have implausible linkages to outcomes and thus should be eliminated. A logic model can also guide determination of performance measures and inform evaluation study design, data collection, and analysis testing hypotheses indicated by if–then linkages, using data measuring the model elements that they connect (Crew and Anderson 2003). Clarity of program theory can also improve the quality and usefulness of findings if, for example, successes or failures are benchmarked against specific expectations rather than politics or stakeholder intentions (Sengupta 2002).

Toward a STEM Workforce Education Theoretical Model

The STEM workforce education logic model illustrated in Fig. 3, below, posits a structure of theoretical relationships among STEM education program activities and outcomes of various types, specifically aligned to the most current ITEST solicitation released by the NSF (2015). This model, with elements inferred from solicitation introductory content and the seven broad ideas defined by the ITEST guiding questions (p. 5 of the solicitation document), is intended to frame a research agenda for the program. While it might be safely proposed that parallels exist in postsecondary education, this model is specifically limited by the purpose of ITEST to K-12 STEM learning programs, educators (both formal and informal), classes (where teaching and learning overlap in the model), and students. The logic model moves theoretically from left to right with each category describing a set of conditions expected to advance the next. For example, the model suggests that outcomes for educators (dispositions, knowledge, skills, and ultimately actions) must be realized before outcomes for students can be reasonably expected to result.

Fig. 3
figure 3

STEM workforce education logic model

Full size image

The logic model is consistent with the precursor models described above in several regards. Outcomes for both students and teachers are aligned with the categories established by the LRC working group’s outcome matrix. Further, logic model elements are color coded by the standard established in the helix model graphic, with most outcomes identified in terms of either STEM careers (red type) or STEM content (black type). Elements illustrated in gray type are operationalized in the ITEST solicitation as not particular to any of the categories in the earlier conceptual models.

This proposed theoretical model should accommodate both traditional and postmodern career development theories mentioned earlier in this discussion. In order to determine their “fit” within STEM work environments, students need to understand the dispositions, knowledge, and skills predominant within STEM workplaces and be able to compare and align to those their own dispositions, knowledge, and skills as they develop over time. As one’s view of career is a result of discourse and interactions with a context, schools need to provide opportunities for students to develop the disciplinary outcomes necessary to engage with STEM technicians and professionals and to activate use of those tools in STEM workplaces. If one accepts that career development is a process occurring in stages that can be guided, education systems seeking to develop students’ STEM content understandings and career capacities would benefit most by connecting learning and careers throughout the K-12 continuum. To be purposeful in STEM workforce education across all grades, both formal and informal educators must develop their own dispositions, knowledge, and skills to support STEM workforce learning. The model proposes that effective teaching and learning actions by educators will promote STEM career development of students leading to persistence in STEM learning and entry into STEM careers. It may go without saying, but career entry outcomes are almost certainly distal enough to K-12 ITEST projects that they will not practically be measured within the scope of a single grant award, even as they must be theoretically linked to funded activities and more proximal outcomes.

It must be noted here that, since this model is new and as yet untested by a substantial body of research, we do not pretend to know how individual elements are theoretically directly linked. Theoretical if–then linkages are implicit between the columns here, following the convention of tabular logic models. The establishment of specific connections among particular elements—variables eventually bearing on workforce placement outcomes—is a key purpose of the research agenda framed by NSF for the ITEST program and illustrated in this model. Doing so through the collection and analysis of data will help determine what strategies show the greatest promise, and ultimately measured effect, on particular workforce outcomes for students.

About Workforce Education Outcomes

For the purpose of the proposed STEM workforce education logic model, we define the term “outcome” very specifically, as a persistent change for individuals or groups that will remain after the program (or other innovation) is completed or no longer acting on them. We focus on this definition as opposed to outputs of program activities, or the goals, objectives, aims, purposes, or other desired results that might be defined for a given program in less-specific terms than we propose here. While not proposing universal definitions, we aim for similar specificity in our usage of additional key terms for this discussion, intended to clarify outcome domains useful to development and assessment of workforce readiness (e.g., Council of Chief State School Officers 2013). Dispositions, in the present context, are attitudes and inclinations, or qualities of mind or character. It may be argued that dispositions toward STEM learning are not malleable, that they are fixed and cannot be influenced by learning experiences, but for the purposes of this discussion, dispositions are held to be flexible in school-aged children and thus open to change by programs or other educational innovations. Knowledge is an accumulation of understanding (i.e., awareness or comprehension) within a particular domain or domains—in this instance, STEM content and STEM careers. A skill is the ability to do something, mental or physical, either to act or exercise agency. Actions are skills made real, observable steps that individuals take toward additional STEM education or career pursuits, both in and out of school.

Outcomes for Educators

For the purposes of this model, the term “educators” (replacing the more specific “teachers”) is framed in the broadest possible sense, to include a full range of titles and roles in both formal (K-12 schools) and informal or out-of-school-time education settings, such as science clubs, competitions, or programs like 4-H. Of particular interest here are career or guidance counselors, whose contributions to students’ decisions about courses and postsecondary education options may well play an important role in priming the STEM education pipeline. Again, teacher knowledge and skills implicitly include the two domains of interest—STEM content and STEM careers—but a third outcome area should also be considered for this group, STEM pedagogy. This domain includes the instructional approaches tailored to the unique characteristics of STEM learning and the effective communication of STEM concepts, such as project-based learning, workplace- or laboratory-based learning, and the use of specialized technologies (Means et al. 2008). Dispositions (e.g., STEM instruction self-efficacy) may also be particularly important for teachers, since those who do not value science or math, or do not feel confident teaching STEM subject matter, are not likely to be effective in realizing targeted outcomes for learners.

Outcomes for Students

In the proposed model, outcomes for students are defined within the now familiar structure of dispositions, knowledge, skills, and actions. It is worth reinforcing here that the arrangement of these outcome domains proposes two particular relationships among them. First, the left-to-right, theoretical if–then organization of the model suggests that dispositions may be precursors to knowledge development, which in turn precedes skill development. Second (and perhaps more certainly) is that achievement of some set of dispositions, knowledge, and skills is necessary in order for students to persist with their STEM education and ultimately enter into STEM careers as productive participants in the workforce.

Testing Model Fit with Examples from Recent ITEST Projects

The ITEST program makes particular mention of the link between STEM learning in both the formal in- and out-of-school-time contexts and workforce education. The current program solicitation (NSF 2015) explains that “ITEST projects must contribute to systematic understanding of student outcomes related to STEM workforce awareness, interests, skills, knowledge or readiness” (p. 4). In fact, STEM workforce constitutes one of the three primary categories that form the strategic framework for the NSF Education and Human Resources (EHR) directorate, in which the ITEST program is located. Workforce investments are intended to improve the education and preparation of a STEM workforce that will be ready to capitalize on unprecedented advances in technology and science and to address global, social, and economic challenges yet to be imagined.

In order to better understand how the proposed model might contribute to broader understandings of STEM workforce learning, data from a sample of ITEST projects were analyzed applying a comparative case study approach (Yin 2003). The results highlight ways in which projects may currently support students’ development of dispositions, knowledge, and skills related to STEM content and careers. With such an emphasis on workforce education and links to connect STEM career development learning experiences with workforce experiences that will increase participation in the STEM pipeline, it may be useful to examine the possibility that such learning–workforce connections are substantially overlooked, underdeveloped, or both across projects in the ITEST program, while being perhaps among the most critical factors in addressing the EHR mandate.

Reflecting on both the directorate’s and the program’s mandates in the area of workforce education, we identified sample ITEST projects, their workforce intentions, and the workforce connections fostered as a result of implementation. These case studies, seen through the helix lens, aim to shed some light on the gaps individual projects might need to address in order for the ITEST program to fulfill its objective of expanding and improving STEM workforce pathways, while exploring the role that the proposed STEM workforce development logic model might contribute to such inquiry.

Limitations of Analysis

Consistent with purposes described by the Common Guidelines for Education Research and Development (Institute of Education Sciences, U.S. Department of Education, & the National Science Foundation 2013), the exploratory study described here is not intended to represent the ITEST program as a whole, but instead to shed light on what kinds of workforce connections and related contributions might be occurring within a subset of current and recent projects. It is intended to be an example of the kinds of data and research that might be addressed by future, more rigorous studies. This study was conducted in a post hoc fashion, in a sense mining specific data from contexts not originally designed to yield such outcomes or serve this exploratory purpose. Nevertheless, we believe this effort provides a window into how workforce and career development linkages may be systematically studied within all ITEST projects, but as well as how those linkages currently appear weaker than EHR staff members likely hope.

In an effort to include a range of projects in our sample, we queried the ITEST principal investigator (PI) and evaluator community about workforce education components in current and former projects. First, we contacted colleagues with whom we have personal relationships, and then, we searched the STELAR (2016) database on ITEST projects. We limited our queries to projects that engaged high school students and STEM professionals, reasoning that these two filters would yield projects that contained at least some deliberate workforce education design, as compared to projects that focused on elementary or middle school students or those in a particular subject area. Of a total of 250 STELAR-inventoried ITEST projects, a search through these filters returned 32 matches. Following outreach to the PIs of these projects, six from that group responded to a short online survey designed to inform the case comparisons.

The data from these six projects were added to those from six others that one of the authors is or has been associated with, providing a sample (mixed random and author biased) of twelve projects from which we might discern where traction is being gained by projects along the workforce education dimensions in the models central to this discussion. The authors acknowledge that, because it is based on convenience sampling methods, this first effort likely suggests findings that may illuminate certain trends and patterns within the samples but are not expected be generalizable across all ITEST projects. Thus, it is the nature of such an exploratory study.

Exploratory Survey

Cursory review of results from the survey suggests that most of the sampled projects engaged in some kind of STEM workforce learning effort (as generally expected by the ITEST solicitation) but did not actually connect their project-based classroom or teacher workshop learning activities to workforce modeling or workplace-connected experiences. All but one of the twelve respondents claimed to have a workforce education trajectory component, defined as one or more specific STEM career development activities designed to prime students and teachers to take steps toward joining the STEM workforce, or perhaps model doing so in some authentic way. For an item relating to engagement of different workforce components, the highest ratings of extent of use (a “4” on a four-point scale) were noted for activities that were passive by nature (e.g., providing information about STEM careers and organizing meetings or presentations by STEM professionals; 63 % claimed to have included these to a large extent). By comparison, components of a more active nature (e.g., providing actual workplace experiences or visits to STEM workplaces) warranted only a 45 % large extent rating; fewer than half the projects included these activities as a significant program element (Table 1).

Table 1 Responses to “To what extent did the workforce development activity…”
Full size table

Responses to another item (see Table 2) suggest that visits to workplace sites occurred in nearly 75 % of our sample projects, a very good rate. However, the inclusion of workforce partners as part of the core team or the active engagement of students through internships or other job-related experiences fell short in the rankings (36 %). Again, a site visit typically involves a “show-and-tell” passive modality, while an internship actively engages learners in activities related to STEM career outcomes.

Table 2 Projects applying specific interactions with STEM professionals
Full size table

While the intent of this instrument was strictly to gain a quick read on how the targeted ITEST projects have integrated workforce education elements and to establish contact with PIs and evaluators for richer data gathering, a cursory scan of the results illustrates the need for program officers and project managers to consider how they might improve workforce education aspects of programming, both in design and in practice.

Developing a Rating Scale

For discussion purposes, we rated each of the twelve sample projects on how they measured up against the six dimensions described in the helix—teacher professional development, partnerships, and cultural context, each in terms of both STEM content and STEM career development (and so by inference, STEM outcomes). Ratings were based on a composite score of survey responses, conversations with project researchers, and analysis of available project materials and data. Purposes for these analyses were to (a) investigate how the dimensions identified in the LRC working group’s helix model align with those in actual projects, in terms of supporting students’ acquisition of STEM content and career outcomes, and (b) begin developing a method useful to examining these dimensions among different projects.

Discussion of Terms

In the helix, three cross-cutting dimensions support students’ workforce development trajectories, each driving upward though the grade levels from elementary through postsecondary education and intersecting with either content or career development in each loop. In rating each of these components, the question arises as to how an individual project addresses a specific dimension, as compared to what an idealized or optimal condition might be. For example, when looking at STEM content related to partnerships (labeled Part CON per the scheme shown in Table 3), we examine the dispositions, knowledge, and skills delivered to educators about partnerships—what they are, examples of how to build and sustain them, how they will strengthen workforce education opportunities, and so on. This orientation was selected as a result of analyses of existing projects and reports and stands in contrast to what might be considered the optimal definition of partnership content—how partnerships deepen and contribute to STEM content and career outcomes, a definition not yet described by existing reporting. This distinction suggests that, from the sampled projects, few if any are engaging partners in the deeper ways that might significantly improve student understanding of the connections between instruction and the workforce (Table 3).

Table 3 Rating scale definitions of terms, current versus optimal
Full size table

A four-point scale was developed to rate each of the three cross-cutting dimensions of Teacher Professional Development, Partnerships, and Cultural Context along the two potential directions of traction—content and career development—resulting in six dimensions. By determining means of scores from a generalized rating rubric (1 = No Evidence, 4 = Much Evidence), we arrived at a hierarchical graph that positions the relative values of each dimension as each is currently found in this sample of ITEST projects (Fig. 4). We are mindful again of the biases inherent in this exploratory study, reflecting the personal connection with six of these projects maintained by one of our authors.

Fig. 4
figure 4

Means of helix dimension mean ratings, sorted (n = 11)

Full size image

Blue bars represent content helix dimensions, and red bars represent career dimensions. Notice that the content bars generally rate higher than those illustrating means of career dimensions. This should cause no surprise, as most NSF ITEST projects develop content first, and PIs may be most familiar with the development and delivery of content, especially through activities relating to professional development (highest rating) and partnerships (second highest rating). Both areas of cultural context scored the lowest.

Sample Project Cases

Five cases highlight how these data apply to actual ITEST projects. These are from the project evaluation portfolio of one of the authors and include a mixture of Strategies-type projects (typically 3-year, single-region-focused, proof-of-concept model projects) and Successful Project Expansion and Dissemination (SPrEaD) projects—the latter generally 4- to 5-year, multi-region, proof-of-scaling model projects, often based on a successful Strategies project. Some projects are current, and some have been completed.

Case 1, completed in 2012, involved urban youth in Detroit. A university team partnered with the city’s Office of Homeland Security and Emergency Management and the public school system to provide over 100 high school students with a two-year training program in geographic information systems and information technology (GIS/IT), culminating in summer internships with a city department or contractor involving work on real-world projects through which they practiced their skills. By the end of the project, students reported significant gains in their determination to seek STEM careers or postsecondary STEM learning opportunities on a range of measures, attributed to participation in internships based on specifically developed STEM expert skills (Xie and Reider 2014). Furthermore, their skill-based experiences introduced them to, and engaged them in, countless opportunities to learn soft skills, including workplace behaviors and protocols (e.g., showing up on time or adhering to a dress code), many of which are well understood by more affluent students but which are not necessarily part of the intrinsic skill set of this population. Finally, many internships took place within the local areas and contexts that had personal meaning to the students, which provided personal motivation for participation and completion, underscoring the value that contexts help shape developmental pathways toward careers (Vondracek et al. 1986; Vondracek 1998; Porfeli and Vondracek 2009).

Case 2 is a SPrEaD project which began in 2014 following the Case 1 Strategies project implemented by the same team. In this scaled-up version of the program, the team has partnered with a state-level instructional agency, a state-level mapping organization, and a statewide online course provider. Compared with its predecessor, with students from a single city participating, this model comprises the entire state of Michigan, including rural communities, at multiple levels of participation—the highest being workforce internships, primarily in fields where jobs often go unfilled.

Case 3 engaged middle school students in three regions of Virginia, following them for 3 years into high school as they learned marine engineering and shipbuilding STEM content, workforce mainstays of the region. Shipyards and related facilities partnered in providing field trips and STEM professional visits to workshops. This project did not have internships, and while the students came from three very different types of communities (port city, landlocked rural, and suburban), cultural context and related connections were not especially well identified.

Case 4, another SPrEaD project, provides minority disadvantaged girls with opportunities to learn and engage with constructive technologies to tell personal stories and develop media artifacts about social challenges to which they have personal connections. These experiences have led to degrees of mastery not realized in their in-school learning environments, mastery that reflects intrinsic motivation (Renninger 2000). Now in its ninth year of existence, and with different program variations supported by different funding sources, the core program operates in multiple states and will be developing an independent university center to support the growing enterprise. The core of its implementation design is cultural context; thus, its program scores in this area are very high. Additionally, through experimentation with different technologies, participants explore their identities as technologists and, ultimately, potential careers. Interviews with girls revealed recurring patterns of both identity and career exploration factors identified by Flum and Kaplan (2006). This program engages several partnerships, but they are devoted to student and site recruitment to meet expansion demands (e.g., public libraries, Boys’ and Girls’ Clubs, church organizations) rather than to model or provide workplace experiences. While the program does have connections with the workforce (workshops often include a female STEM professional giving a talk or presentation), there is no placement component.

Finally, Case 5 is a multistate SPrEaD expansion of a successful earlier project, with over 200 teachers trained in the use of scientific probes and models connected to classroom computers and tablets to help students learn the investigative process and explore how data can justify scientific hypotheses. K-12 classrooms in four states participated formally, but many other students and teachers elsewhere participated informally. This program fell short on partnerships and cultural context related to career development and practices, partly due to the wide range of student ages (Grades 3–12) and differing cultural contexts across all sites. The program made a concerted effort to inform teachers and students about the STEM workforce, with a STEM career Web site that other ITEST projects accessed. The connections were passive, however.

Questions to Consider

At this juncture, it would be appropriate to refocus on a few key questions: What is the impact of ITEST on the workforce trajectory of youth? How does the ITEST program help students apply STEM content to STEM workforce education? How can the ITEST program improve the application of STEM content knowledge to career development?

The case studies—while different in topic, content area, geography, and design—do show some patterns among the helix dimensions. Professional development for STEM content (PD CON in Table 4) is uniformly high, and partnerships related to STEM content (Part CON) are low. Cultural context for both content and career (Cultural CON and CAR, respectively) are, with one exception, the lowest rated categories. This suggests that while individual projects may address cultural context directly, as does Case 4, most do not. Cultural context related to career development as the lowest scoring clearly shows that programs might well address this component more directly to help establish tighter bonds between STEM content learning and workforce development knowledge.

Table 4 4-Point ratings of helix dimensions of case study projects, sorted by average rating
Full size table

Program evaluation and internal research in these projects have focused primarily on two realms of outcome, probably not coincidentally those most familiar to school-based education researchers and related to the highest rated dimension among the case study data—changes for students in terms of specific content dispositions, knowledge, and skills; and changes for teachers across the same three outcome areas. The findings about the link between these workforce-directed projects and workforce education outcomes could be more robust. As long-time participants and contributors to the ITEST community, we believe it is important to bring this issue to the forefront. Further, we propose that both program evaluation and ITEST project designs can be improved by (a) attending to action outcomes for both educators and students, as desired results of realizing outcomes in the other three matrix domains, and (b) considering STEM career activities and outcomes with equal attention as those targeting STEM content.

To better tell the ITEST story, the authors of this article propose that PIs and evaluators might engage the concepts illustrated by the helix model, the outcomes of the matrix, and the theoretical framework defined by the workforce education logic model, to develop a set of instruments and methodologies to study in greater depth the workforce outcomes and linkages to development activities that should be inherent in ITEST project designs. Doing so will better inform program teams, the NSF, and ultimately the broader field of STEM workforce education. ITEST provides a rich and accessible testbed like no other, in which the STEM learning community might study these connections. We should consider new tools to take advantage of the opportunity to do so.

Further Implications

This work was borne of a perceived need to address a number of key gaps in knowledge and evaluation research on STEM workforce education. The authors hope that the models and issues discussed in this article have implications for the design, development, and evaluation of K-12 STEM workforce education programs. It is our anticipation that, with these models as a guide, project planners and designers can think more broadly about questions they should be asking and will emphasize a balance of content and career education activities in their STEM projects, programs, and courses. Such projects should look beyond content to collect additional types of data, including measures of more distal outcomes. The ITEST community should benefit from synergies among projects and thus draw more complete and informative conclusions about how ITEST contributes to the STEM workforce. Furthermore, the models proposed in this article should make it easier to leverage the research agenda proposed by the seven guiding questions in the solicitation, as the first step in developing an evaluation framework and set of questions—perhaps even some common measures or instrumentation—that could underpin all ITEST projects, encompassing the full range of aspects of STEM career and workforce education.

These steps have illuminated new challenges and gaps in knowledge and data that might lead toward a better understanding of the impact of ITEST. Our work suggests that while substantial volumes of data and research findings exist relating to the educational components most familiar to us (curriculum, content gains, and professional development outcomes), more is needed to identify how ITEST impacts the career development of future STEM professionals. Possible next steps might include conducting a study to investigate and develop specific instruments, research methods, and recommendations that focus on STEM career and workforce education components. At that point, we may achieve a more balanced representation of the program’s influence on student STEM motivation and participation that will inform education programs and workforce domains as young people pursue educational experiences to prepare for STEM careers.