Assessing the Long-Term Impact of the Preparing Future Faculty Seminar

Abstract

The Preparing Future Faculty (PFF) initiative for graduate students was launched in the United States in 1993 as a partnership between the Council of Graduate Schools and the Association of American Colleges and Universities to prepare graduate students for faculty careers at different institutional types and to provide them with teaching-related professional development. PFF programs have proliferated U.S. universities over the last two decades, but there has been limited research on the long-term impact of these programs. This study at the University of Michigan examines the career paths and attitudes of graduate students who participated in an annual, intensive, five-week PFF seminar between 2007 and 2013. The descriptive study explores the following research questions: (a) What are the career outcomes for PFF seminar participants compared with nonparticipants? (b) Does the exposure to a variety of institutional types in the PFF seminar lead to a greater appreciation of and openness to job opportunities at diverse institutions? and (c) Does the PFF seminar assist graduate students in their job searches and their chosen career paths? The results of this study contribute to our understanding of what universities can do to effectively prepare graduate students for their future careers.

Keywords: graduate student development, evaluation, programs

Doctoral education provides a myriad of growth opportunities for graduate students to develop as researchers and scholars. However, many studies acknowledge that in some instances, graduate students have fewer opportunities to receive systematic preparation for their multiple roles (Austin, 2002; Golde & Dore, 2001, 2004; Luft, Kurdziel, Roehrig, & Turner, 2004; Nyquist et al., 1999; Walker, Golde, Jones, Bueschel, & Hutchings, 2008; Weidman, Twale, & Stein, 2001; Wulff, Austin, Nyquist, & Sprague, 2004). Research on graduate student socialization suggests that these students would be better prepared for a faculty career if they were more aware of the impact of institutional context in their job search process and of best pedagogical practices for teaching and how students learn (Austin & McDaniels, 2006; McDaniels, 2010; Smith, 2001). A 1999 survey of new faculty indicated that they would have appreciated the opportunity to revise and develop new courses as graduate students, which would have provided them with early feedback and greater confidence (Smith, 2001).

In response to these early concerns, the Council on Graduate Studies and the American Association for Colleges and University collaborated in the early 1990s to fund preparing future faculty (PFF) programs with three main goals: “to provide graduate students with on site experience of faculty life at diverse academic institutions,” “to provide forums … at which graduate students and faculty from diverse institutions could speak candidly about professional expectations,” and “to encourage graduate programs themselves to integrate the professional development of graduate students … more directly into graduate education” (DeNeef, 2002, p. 1). University partnerships or clusters were formed from a range of institutional types, which created a variety of PFF programs, including faculty discussions about tenure, faculty work life, and teaching challenges; preparation of job market materials; and observations of classes and faculty meetings (DeNeef, 2002; Goldsmith, Haviland, Dailey, & Wiley, 2004). The final report describing the national PFF movement assessed the perspectives of PFF program alumni from seven campuses using surveys and interviews (DeNeef, 2002). The results from the survey indicated that the participants found the program to be quite valuable with 16 activities rated as being “moderately” to “highly” valuable. The activities with the three highest mean scores were attributed to having discussions with faculty, gaining experience with mentors’ classes, and developing teaching statements (DeNeef, 2002).

To assess how doctoral granting institutions are responding 10 years after the national call for greater professional development, Palmer (2011) surveyed over 250 institutions to categorize the availability of professional development programming. With regard to teaching focused programs, a majority provided Teaching Assistant (TA) preparation programs/TA orientations (56.7%), but only a minority provided higher education pedagogy courses within the discipline (31.7%), PFF or PFF like programs (27.6%), teaching certificates (23.1%), or lead TA/mentoring programs (22.8%). Although this research highlights the prevalence of programs, it is unclear whether graduate and professional students are taking advantage of these resources and whether they are adequately preparing graduate and professional students for their future careers.

Since the early 2000s, few studies have systematically evaluated the impact of PFF programs across a variety of institutions. However, there are some researchers who have examined the impact of PFF programs on the graduate and professional students. For instance, Connolly, Savoy, Lee, and Hill (2016) examined the impact of teaching professional development programs on graduate students from Science, Technology, Engineering and Math (STEM) fields across three institutions. In this study, STEM graduate students who engaged in more teaching professional development (e.g., greater than 55 hours) by participating in activities such as a pedagogy course were more likely to become faculty members and were more likely to report higher levels of teaching self efficacy (Connolly et al., 2016).

Furthermore, Wurgler, VanHeuvelen, Rohrman, Loehr, and Grace (2013) examined the impact of the University of Indiana’s Sociology Department’s PFF initiatives on job satisfaction, confidence, and competence of its graduates. When comparing those graduate students who completed a three course sequence to those who did not (including those who may have taken only the first course in teaching), the PFF participants reported lower levels of job satisfaction but higher confidence in their first teaching assignments, although not statistically significant. In addition, PFF participants were more likely to express a greater sense of professional competence within their first faculty positions. For those who completed the three course sequence, they also reported that the benefits of the program included pedagogical knowledge (e.g., teaching tools, scholarship of teaching, and learning), professionalism (e.g., knowledge of the academy and expectations), and institutional support (e.g., faculty and student support).

Although research has been conducted recently to provide a sense of the current landscape of PFF programs (Connolly et al., 2016; Palmer, 2011) and the impact of one institutional PFF program on participants’ job satisfaction and competence (Wurgler et al., 2013), there are still many more questions to be answered with regard to PFF offerings. For example, what are the career outcomes for graduate students who participate in PFF programs? Does the exposure to a variety of institutional types lead to a greater appreciation of and openness to job opportunities at diverse institutional types? In the long term, how would participation in a PFF program assist graduate students in their job searches and their chosen career paths?

Program Background

To address these research questions, we examine the impact of one of the PFF programs offered at the University of Michigan (U M). In particular, the PFF seminar, which began in 2000, has been designed to address three goals, for participants to learn about the landscape of higher education that is most relevant to future faculty, reflect upon effective teaching practices, and develop academic job search materials. This 10 session seminar is held over five weeks during the months of May and June to provide graduate students and postdocs with the opportunity to delve deeper into the aforementioned topics through interactive presentations from Center for Teaching and Learning (CTL) experts, a variety of faculty panels, campus visits to institutions that vary in mission and scope from U M, and teaching philosophy/course syllabus document creation. This program provides an opportunity for graduate students and postdocs to reflect on their current and future teaching and, ultimately, the prospect of an academic career as a part of an intensive interdisciplinary cohort experience. This reflection is followed by structured peer feedback activities, including small feedback groups consisting of participants from similar disciplinary backgrounds as well as intentional cross disciplinary groupings. The program also has been described in length elsewhere (Cook, Kaplan, Nidiffer, & Wright, 2001; Hershock, Groscurth, & Milkova, 2011).

Each year, roughly 60 advanced graduate students (along with a few postdoctoral fellows) are selected in a competitive application process. Over the years, the acceptance rate has varied between 50% and 80%; however, applicants may reapply in subsequent years. To ensure disciplinary diversity, approximately a third of the participants are from the humanities, a third from social sciences, and a third from STEM fields.

Although detailed demographic data have not been collected for these cohorts, we asked participants to self report a few characteristics in an online intake survey before the seminar began. For instance, between 2007 and 2013, 6%–20% of the seminar participants reported completing their undergraduate education outside of the United Staes. Approximately 10%–20% of participants reported that they were first generation college students, and between 20% and 50% indicated they were first generation graduate students.[1]

Research Context and Methodology

For our long term evaluation of the PFF seminar, we investigated PFF seminar participants’ career outcomes and their attitudes about the long term impact of the seminar on their careers. We also explored differences in career outcomes between PFF seminar participants and those who did not participate in the seminar. In selecting a comparison group to describe career outcomes of PFF seminar participants, we were concerned about comparing U M PFF seminar participants to all U M graduate students and postdocs who did not participate in the seminar because those who participate in PFF type programs may be systematically different from those who do not. Graduate students and postdocs who actively invest in their own professional development may be more likely to succeed in a career than those who do not take an active role in their own development. In order to avoid this potential selection bias, we used PFF seminar applicants who did not participate in the program as our comparison group. This included both students who were not selected for the seminar as well as students who declined the offer to participate due to personal or professional conflicts. We select students for the seminar based on whether their professional development goals and teaching philosophy align with the learning goals of the seminar, not based on whether we presume students will be successful in securing a faculty position, so these two groups are comparable in terms of their prospects on the academic job market and their motivation to engage in their own professional development.

Among the population of PFF seminar applicants and participants, we focused on seven recent cohorts (2007–2013). We chose this time period because the major elements of the program were comparable. We examined the syllabi to confirm that the learning objectives for the seminar were constant during this seven year period. These were:

1. Design a specific course you could teach at U M or another institution utilizing principles of backward design and course alignment

2. Apply the research on selected topics in teaching and learning to your own teaching

3. Reflect on your own practice and explain your approach to teaching to colleagues and future employers, both in writing and orally

4. Assess the various institutional contexts in American higher education and evaluate their implications for your career as a faculty member

5. Identify ways the changing student populations, structure, governance, and accountability issues in higher education will affect your role as an emerging academic professional

6. Analyze and evaluate issues that will face you as a new faculty member in your first academic job and assess their implication for the development of your faculty career

7. Identify and use resources to help you adjust to and succeed in your first faculty position

Additionally, the assignments were essentially unchanged (e.g., creating a teaching philosophy statement and syllabus, providing feedback to peers on their teaching documents, participating in an online discussion, etc.). Although some of the assigned readings changed from year to year, the overall themes that emerged from the texts reflected the overall learning objectives. Finally, the PFF seminar was cofacilitated by the same instructor, Dr. Tershia Pinder Grover, and while the teaching approach may have varied slightly, the continuity provided by having the same instructor ensured that the essence of the program was comparable.

The PFF seminar participants from 2007 to 2013 included 370 U M graduate students and postdocs. PFF seminar applicants from 2007 to 2013 included 209 graduate students and postdocs.[2] However, 22 applied more than once, so we included them only once in the most recent year of their application or the year of their participation. Of the 557 remaining individuals in the population of our study, six were excluded since they applied in 2013 and deferred to 2014. Thus, the total population for our study is 551.

We took a two pronged approach to describe career outcomes for and attitudes of PFF seminar applicants and participants: (a) we investigated job outcome[3] data available through the U M Rackham Graduate School, and (b) we conducted a survey of PFF seminar applicants and participants.

In 1999, the Rackham Graduate School began collecting job outcome data for its Ph.D. graduates. It conducts an annual job outcome survey of each graduate’s employment history for 10 years postgraduation. The graduate school provided us with job outcome data for PFF seminar applicants and participants for the first year after students acquired a Ph.D. from U M. This dataset did not include those who only received a terminal Master’s degree. Due to the graduate school’s restrictions on sharing this confidential job outcome data, we were only able to acquire the job outcomes data for the first year after graduation from the graduate school. Given the absence of longer term job outcome data (e.g., job outcomes after three or five years), we are limited in giving an accurate portrayal of longer term job outcomes. However, the data available allowed us to provide an accurate comparison of the impact of the seminar one year after degree completion. Of the 551 individuals in our study, 62% of whom were Ph.D. graduates from U M who were included in Rackham annual job outcome survey (342 cases), 34% were current Ph.D. students, and 4% were not Rackham students (e.g., were a postdoc at U M), so data were not available on their job outcomes. Using the initial job outcome data, the 342 cases were separated into three groups: “academic job outcome” participants were in a tenure track job or postdoc fellowship, “professional job outcome” participants were employed outside of traditional positions, and “other” participants were not employed or did not report job outcome data.

For our survey of PFF applicants and participants, we researched e mail addresses for the 551 individuals who had completed or applied to the PFF seminar. We used U M e mail aliases for those who still had them, and conducted internet and social networking site searches to find e mail addresses for those in our population without U M e mail addresses. We were unable to find e mail addresses for 13 individuals, so 538 individuals were contacted.

We received Institutional Review Board “not regulated” status exemption to conduct our survey for program evaluation purposes. We sent a solicitation e mail inviting the applicants (185) and participants (353) to complete our survey (Appendix A for solicitation e mail). We surveyed applicants and participants regardless of their current career stage. The survey was sent to Ph.D. graduates, graduate students, and postdoctoral scholars. For those who did not respond to our first solicitation message, two follow up reminders were sent. The survey was open for approximately one month (February 12, 2014 to March 14, 2014). Response rate was 31.4% for applicants (58) and 56.4% (199) for participants.

For both participants and applicants, the survey included questions about career stage, occupation, faculty position, and institutional context (for those who indicated they have or will soon begin a faculty position), and attitudes about professional development opportunities at U M (Appendix B for survey instrument). For PFF seminar participants only, we also solicited feedback about what experiences, sessions, and materials from the PFF seminar were most useful to participants in their job search and long term career.

Initial Job Outcomes: Rackham Graduate School Data

The U M graduate school data on job outcome were divided into two groups: “applicants” who did not participate (34%) and “participants” in the seminar (66%). Cases were also divided by their initial job outcomes (i.e., their job immediately following degree conferral). Using the initial job outcome (excluding current Ph.D. students), the 367 cases included in the annual job outcome survey were separated into six groups: tenure track faculty (n = 74), nontenure track faculty (n = 50), academic administration (n = 12), postdoctoral scholar (n = 132), other professions outside of academe (n = 32), and missing for those who were either not employed (n = 25) or did not report job outcome data (n = 42).

Initial job outcomes in tenure–track positions were roughly equivalent for applicants and participants (approximately one–quarter of both groups had tenure–track positions). However, the PFF seminar participants were more likely than applicants to have initial job outcomes in either nontenure track faculty or academic administrative positions (Table 1).

Applicants were somewhat more likely than participants to have initial job outcomes in postdoctoral positions and other professions outside of academe. A chi square test was performed, and these differences in initial job outcomes were found to be statistically significant, X² (3, N = 300), p = .05. In sum, in terms of tenure track positions, we see similar outcomes for PFF seminar participants and applicants. While there are slight differences between participants and applicants in other types of job outcomes (nontenure track faculty, academic administration, postdoctoral positions, and other positions), we would not suggest that PFF seminar participation necessarily accounts for these differences.

Current Job Outcomes: Survey Data

While the graduate school job outcome data offer information about initial job outcome, our survey elicited data about respondents’ current positions. Among respondents to the survey, the majority (60%, or 153) indicated that they now have (or soon would begin) a faculty or postdoc position.[4] Among the remaining 40% of respondents, the most common responses about career stage included: 13% now have or will soon begin a nonfaculty position, 11% were currently seeking faculty positions, and 6% planned to begin applying within the next year.

The 153 respondents who indicated they now have (or soon would begin) a faculty or postdoc position were asked to describe their current position as “tenure track,” “nontenure track,” or “postdoctoral scholar.” A majority (58%) indicated they were tenure track, 14% indicated they were nontenure track, and 28% indicated they were postdoctoral scholars. We then divided the respondents into “applicants” and “participants.” We found that a greater proportion of participants (60%) had tenure track positions compared to applicants (50%), and a greater percentage of applicants (43%) had postdoctoral scholar positions compared to participants (24%) (Table 2). However, a chi square test found that these differences were not statistically significant, X² (2, N = 151), p = .08. In other words, similar to the graduate school year 1 job outcome data, our survey analysis did not find significant differences in current job outcomes for participants compared to applicants.

Institution Type and Focus of Position

We also asked survey respondents who indicated they now have (or soon would begin) a faculty or postdoc position to provide information about their institution type and the focus (teaching, research) of their position. We asked respondents to indicate at what type of institution (according to Carnegie classifications) they currently hold a position (Table 3).

A chi square test was conducted to see if the differences in institution type of job outcomes were significant,[5] and the differences are not statistically significant, X² (4, N = 146), p = .87. Among both applicants and participants, the vast majority are currently in a position at a doctorate–granting university. Given that U M is an elite doctorate granting university, this is consistent with what one might expect for their job outcomes.

Respondents were also asked to characterize the focus of their position as: exclusively research, mostly research, even division between research and teaching, mostly teaching, and exclusively teaching. The most common response was an even division between research and teaching (36%), followed by mostly teaching (25%) and mostly research (21%). Among participants compared to applicants, a greater percentage of participants characterized their positions as “mostly teaching” compared to applicants. A greater proportion of applicants characterized their positions as mostly research or exclusively research compared to participants. However, a chi square test found that these differences were not statistically significant, X² (4, N = 149), p = .21 (Table 4). In other words, applicants and participants had comparable emphasis on research compared to teaching in their current roles.

We also divided the focus of respondents’ current positions by institution type. As one would expect, positions at doctorate granting and special focus institutions had a greater focus on research, while the focus was heavier on teaching at master’s colleges/universities and baccalaureate colleges (Table 5).

A larger percentage of those at baccalaureate colleges (72%) and master’s colleges/universities (69%) characterized their role as mostly teaching compared to a small proportion at doctorate granting institutions (7%). Similarly, while no respondents at baccalaureate colleges and master’s colleges/universities characterized their position as exclusively research focused, 15% of those at doctorate granting universities and 43% of those at special focus institutions characterized their work as exclusively research.

Perceived Helpfulness of Seminar Activities

The next section focuses only on participant feedback to illustrate the long term reported impact that various components of the PFF seminar have on their careers. The PFF seminar has a strong focus on exposing participants to faculty work life at different institution types (e.g., liberal arts colleges, comprehensive universities, specialty focus institutions, and community colleges) and different types of positions (e.g., nontenure track faculty positions). Survey respondents who participated in the PFF seminar (79% of the respondents) were asked about the value of these aspects of the program. When asked how helpful the PFF seminar was for developing respondents’ understanding of faculty work life at other types of institutions, most (78%) indicated it was very or extremely helpful (Table 6).

Participants were given the opportunity to elaborate on their responses to this question, and 55 respondents wrote additional comments. A qualitative analysis of these comments was performed to identify common themes.[6] The most common theme (named by 20, or 36%, of those who responded) was that exposure to institution types that respondents had no prior exposure to as undergraduate or graduate students was particularly valuable. As one respondent put it,

The second most common theme (by 15, or 27%, of those who responded) was that the campus visit was transformational in understanding work life at different institutions. As one former participant noted, “The campus visit was a big help to me. I visited a liberal arts college and while I did not end up with a liberal arts job, I found that I was much better prepared for interviews at liberal arts schools as a result of the campus visit.”

Respondents were also asked (on a five point Likert scale) which elements of the PFF seminar were helpful to them in their career. A total of 80% found campus visits to different institutions very or extremely helpful for their careers (Table 6). A total of 77% indicated that discussions about work life at different institutional types with non U M faculty were very or extremely helpful for their careers (Table 6).

Those who indicated they had participated in the PFF seminar were asked a series of questions about the helpfulness of the seminar for their careers. We asked respondents to indicate on a five point Likert scale (from not at all helpful to extremely helpful) the helpfulness of 11 components of the seminar, including discussions of higher education, discussions and modeling of inclusive teaching, discussions and demonstrations of instructional technology, U M faculty discussions with junior faculty about work life, U M faculty discussions with senior faculty about tenure, U M faculty discussions with lecturers (i.e., nontenure track faculty), non U M faculty discussions about work life at different institutional types, campus visit, interview practice, writing a statement of teaching philosophy and designing a syllabus (Figure 1).

Figure 1. Most Helpful Aspects of the Seminar for Participants’ Careers

The PFF seminar components were all rated very highly, ranging from 3.8 to 4.5 of 5.0. The lowest rated items (both rated 3.8) were discussions and demonstration of instructional technology and job interview practice. The highest rated item was writing a statement of teaching philosophy (4.5), followed by campus visits and non U M faculty discussions about work life at different institutional types (both rated 4.3). U M faculty discussions with junior faculty (4.2) and discussions of higher education (4.1) were also seen as extremely helpful. While all aspects of the seminar were rated very highly in terms of helpfulness for respondents’ careers, a significant minority of respondents could not recall two components of the seminar practice job interviews (35%) and discussions with U M lecturers (19%).

The PFF seminar has a strong emphasis on teaching professional development and includes training on the use of classroom assessment techniques, course design principles, the scholarship of teaching and learning, instructional technology, active learning techniques, and inclusive teaching and learning strategies. Respondents were asked how helpful these teaching professional development experiences in the seminar were (on a 5 point Likert scale from not at all valuable to extremely valuable). The average rating was 4.1, with 73% of respondents indicating the topic as very or extremely valuable for their careers (Table 6). A total of 68 of these respondents shared qualitative feedback when asked to elaborate. The most common themes that emerged in these open ended comments as helpful for professional development were active learning techniques and principles of course design. Representative comments include “Learning about and practicing backwards course design made a big difference for how I think about preparing syllabi and lesson plans” and “The lessons about active learning have stuck to me to this day.”

Survey respondents were also asked in an open ended question to share what other ways the PFF seminar was helpful to them in their career or job search. The 115 responses to this question were analyzed to identify the most common themes. First, about a quarter of former PFF seminar participants (22%, or 25 respondents) noted that the seminar increased their understanding of the many options for careers in academe. To illustrate, one respondent said the seminar “[h]elped me understand the options available for a PhD in academia.” Second, a fifth of respondents (20%, or 23) indicated that the seminar helped them prepare for their academic job search and interviews. For example, one respondent said, “The seminar assisted me in particular with structuring the career job search in ways that made the process more comfortable, engaging, and tangible as well.” The third most common theme (15%, or 17) was that it deepened their understanding of what type of job would be the best fit for them. As one respondent put it, “I really felt that it boosted my confidence going into the job market, as I was sure that I knew the type of institution at which I wanted to find a job.”

We also asked past participants to share what else the PFF seminar could include that would have been helpful to them in their careers. A total of 104 respondents listed suggestions for topics that would have been helpful to them, and four common themes emerged. First, participants desired even more practice with interviews (13%, or 14). For example, student comments included “more interview practice” and “more opportunities for mock interviews.” The second theme that emerged was a desire to receive training and/or consultation about job packet materials other than the syllabus and teaching philosophy statement (13%, or 14), including requests for help with cover letters, CVs, and research statements. Participants desired the “review of an entire application package—cover letter, CV, research statement, teaching statement.” Third, some respondents (7%, or 7) thought job offer negotiation training would have been useful. Finally, a substantial minority (6%, or 6) of respondents mentioned that the focus exclusively on faculty jobs was myopic given difficult job market conditions and hoped that the PFF seminar would create space for discussion of jobs beyond academe. One student desired “tips on how to seek out nonacademic positions, just because the market has gotten so bad.” Another noted, “right now the funding climate is really tough and a lot of people are opting out of academia. I think a topic (even a single afternoon) on alt ac or nonacademic jobs might be useful.”

Perceived Helpfulness of Seminar Teaching Documents

PFF seminar participants work on two job search documents as part of the seminar, including a teaching philosophy and a course syllabus. To prepare them to write these documents, participants receive training on how to write an effective statement of teaching philosophy and principles of course and syllabus design. Participants then draft their teaching related job documents and receive several iterations of feedback from colleagues in the seminar.

Participants were asked whether they used the statement of teaching philosophy and syllabus in their faculty job searches. Among PFF seminar participants who indicated they did do a faculty job search, 95% used (or planned to use) their statement of teaching philosophy, and 56% used (or planned to use) their syllabus in their faculty job search (Table 7).

Participants were also asked in two open ended questions how they have used their statement of teaching philosophy and their syllabus. A total of 168 respondents provided comments in the open ended question about how they used their teaching philosophy statement, and a qualitative analysis of common themes was performed on respondents’ answers. The most common theme mentioned by 142 respondents (85%) was that the document was used in job applications. As one respondent put it, “I used it as part of my teaching portfolio and also as part of my job application materials.” The second most common theme mentioned by 31 respondents (18%) was the use of the philosophy in tenure and promotion files later in their careers. For example, one participant reported, “I used it both for job applications and for my tenure/promotion case. It did in fact provoke a question from my promotion committee.”

A total of 138 respondents shared how they used their syllabus, and a qualitative analysis was performed to identify the most common themes. Most frequently, respondents indicated that a syllabus was required for job applications (47% of comments, 65 respondents). One illustrative quote documents this use, “I have since used my course syllabus for job applications for full time positions.” A significant minority of respondents (21 or 15%) indicated that they used the syllabus to teach the class with little or no revision. An equivalent proportion (15%, or 21 respondents) also reported that they used the principles of course design to significantly modify the syllabus to teach different courses. Respondents also indicated that they used the syllabus in their job interviews, including as a campus visit discussion topic (20 or 14%) or to help prepare a sample class for a teaching demonstration (10 or 7%).

Discussion

The first research question of our descriptive study was what the career outcomes are for PFF seminar participants compared to those who applied but did not participate. We found slight differences in the types of initial job outcomes of past PFF seminar participants compared to applicants who did not have a tenure–track position as their initial job outcome, but there were no substantive differences in the number who had tenure track positions. Although this is a descriptive rather than an empirical explanation study, we did use chi square tests to explore whether differences in current job outcomes between applicants and participants were statistically significant and found that differences are not significant. We also did not find significant differences in the types of institutions where respondents are currently working as faculty; the majority of both groups are at doctorate–granting universities. In sum, we did not find substantial differences in academic job outcomes when we compared PFF program participants with program applicants who did not participate.

Our second research question was whether exposure to a variety of institutional types in the PFF seminar leads to a greater appreciation of and openness to job opportunities at diverse institutions. As noted earlier, the literature on preparedness of graduate students and new faculty finds that these populations report a lack of awareness of the impact of institutional context in their job search process (Austin & McDaniels, 2006; McDaniels, 2010). Unfortunately, because the graduate school’s job outcome data are confidential, we were not able to assess the impact of the seminar on institution type differences of initial, first year job outcomes. In our survey, participants in the PFF seminar reported that exposure to a variety of institutional types in the PFF seminar was extremely valuable and reported that this led to a greater understanding of work life at diverse institutional types. Many program participants reported that exposure to faculty in different institutional contexts was useful to them in determining fit as they went on their academic job search. However, when we compared the Carnegie classifications of the institutions of participants to applicants, we did not find statistically significant differences. We did not have data about what type of institution respondents preferred to secure a position when they were seeking a job, and it is possible that job outcomes do not match the preferences of job seekers. An additional significant limitation is that we did not ask applicants about their openness to jobs at diverse institution types, so we do not have a comparison group for this particular research question. Taking these factors and limitations into account, our findings suggest that while PFF participation did not affect where participants ultimately got jobs, it provided activities that helped to increase participants’ understanding of work life at different institutions and gave them greater clarity about where they would “fit” best.

Finally, we wanted to explore whether and how the PFF seminar assists graduate students in their job searches and their careers in the longer term. In terms of what participants reported as extremely helpful for their careers from a long term perspective, they especially valued writing a statement of teaching philosophy, campus visits, and discussions with faculty about work life at different institutional types. Teaching related professional development was also highly valued, especially learning about active learning techniques and principles of course design. Given that the literature states how graduate students and new faculty report that they are less informed about best pedagogical practices for teaching and how students learn (Austin & McDaniels, 2006; Smith, 2001), it is not surprising that participants found teaching related professional development valuable. PFF seminar participants indicated that their work on teaching–related job documents (especially a teaching philosophy statement) was one of the most valuable aspects of the seminar for their careers in the long term. Teaching philosophy statements were extremely valuable in job searches, and many found that they continued to be valuable in tenure and promotion files. Participants also noted a desire to receive training on preparing other job packet materials, including cover letters, CVs, and/or research statements.

When asked what other ways the seminar was valuable to them in their careers in the long term, participants reported a greater understanding of the many options for careers in academe, increased preparedness for their academic job search and interviews, and better understanding of what type of job would be the best fit for them. Taken together with the high value placed on campus visits and discussions with faculty at different institution types, it is clear that the long term perceived impact of the seminar is greater knowledge about the landscape of higher education and self awareness about where one fits best in that landscape.

Future Work

Our descriptive study has some important explanatory limitations, and there are several interesting areas for potential future research both on our campus and across institutions. One question that remains unanswered is whether the graduate students who apply to PFF like programs to take an active role in their own professional development have different career outcomes than students who do not seek participation in such programs. It may be that students who even apply to PFF programs are already more actively engaged than the average student in seeking out knowledge about faculty work life and strategies for success. One future direction for more empirically driven research would be to compare the groups studied here (applicants to and participants in such programs) to the broader pool of all graduate students.

Due to data limitations, we only were able to explore job outcomes, not job preferences. A very interesting future direction for research would be to assess changes in job preferences before and after participation in PFF programs, as well as assess whether those preferences match ultimate job outcomes. For example, it may be that a student concludes after our PFF seminar that he would most like a community college job only to find that he is only invited for interviews at doctorate granting universities. How can PFF programs be valuable not just in shaping preferences for job outcomes but in preparing students for satisfaction regardless of institutional context? To address these questions, we could evaluate our PFF seminar participants’ job goals at the beginning and end of the seminar and later compare these data to long term job outcomes and satisfaction. This would help graduate student and faculty developers to better understand how job preferences evolve and align with job outcomes and what that means for faculty work life satisfaction. If graduate students’ preferences do not line up with job outcomes, this could have important implications for how PFF programs support future faculty with strategies for work life satisfaction regardless of institutional context.

Another limitation is that participants were enrolled in the seminar between 2007 and 2013, but all respondents took our long term evaluation survey at the same time (in 2014). Respondents who participated most recently will remember the seminar most but have had the least time to find employment. Conversely, respondents who participated less recently will have difficulty remembering specific aspects of the seminar, although they are most able to report long term career outcomes. A superior research design for future work would be to conduct a long term impact assessment of participation in the seminar at a consistent time interval, for example, after five years, to control for discrepancies in length of time since participation in the seminar.

A final limitation of our study is that we are not able to assess whether applicants or participants engaged in other professional development activities that might have influenced their professional development and/or job outcomes. For example, on our own campus, several departments have significant departmental training opportunities related to PFF; we have a one day campus wide PFF conference that was offered annually between the same period of our study (2007–2013); our CTL offers individual teaching philosophy statement consultations; and we have an intercampus mentorship program to enable graduate students to connect with faculty at different institution types. Many national disciplinary organizations also offer professionalization programming to graduate students. Given that we do not have data on participation of our research subjects in these other opportunities, we cannot account for variance in job outcomes that may result from engaging in a range of PFF professional development opportunities.

Given the difficult academic job market realities in many fields, initial job outcomes (one year after graduation) do not necessarily reflect long term career outcomes for many scholars. Another promising future direction for research would be to explore job outcomes of PFF seminar participants compared to applicants using longer term job outcome data. Although our graduate school was only able to provide us with initial job outcome data due to confidentiality reasons, one promising future direction if data could be deidentified and accessed would be to explore whether we see differences in job outcomes after three, five, or even ten years.

Even further, it may be helpful to build upon the work of Palmer (2011) and Connolly et al. (2016) by considering the impact of a wide range of PFF like programs across a variety of institutions. It would be valuable to compare the long term impact and perceived helpfulness of different types of programs and activities across institutions and disciplinary contexts. Given the prevalence of these programs, in what ways do they adequately prepare graduate and professional students for their future careers? What PFF activities are most effective for supporting students’ long term careers across institution types?

Additionally, our study was not able to explore possible differences across disciplines (survey respondents were not asked to identify their disciplinary background). For example, in our institutional context, STEM students have fewer opportunities to teach while they are graduate students. Do students in STEM fields benefit from the PFF program’s teaching professional development more than students with more opportunities to teach? The faculty job market has been particularly challenging for humanities students in the last decade. Does participation in PFF programs impact job outcomes more for students in humanities fields, where the market is particularly challenging? Future research could explore these (and other) questions about disciplinary differences in the impact of PFF programs.

Finally, when asked what else could have been offered to help participants in the PFF seminar, several program participants indicated that the focus on PFF does not take in to account the diverse career outcomes of doctoral students, especially in the current higher education environment where the proportion of tenure track jobs is decreasing relative to the proportion of nontenure track jobs. A question that faculty developers who coordinate PFF programs must wrestle with is how to best prepare graduate students for success in their careers both in and beyond faculty roles.

Recommendations

Based on our findings, there are several suggestions for other practitioners as an outgrowth to our suggestions for future work. First, our findings suggest that the most valuable features of PFF programs include: (a) substantive engagement with faculty at other institution types and (b) preparation of foundational job documents. Regarding engaging with faculty at diverse institutions types, we found that participants reported both visits to other campus, and discussions with faculty from outside of our institution were extremely valuable to them for their long term careers. As much as it is logistically possible, identifying nearby institutions of different types and traveling with participants to those different campuses and/or bringing guest faculty speakers from those institutions in to speak with participants is especially valuable for their understanding of the range of faculty careers possible. PFF program organizers may also want to leverage video conferencing technology in order to extend beyond geographic boundaries. When coordinating with campus liaisons and faculty, it is often helpful to recruit faculty who represent a range of social identities, ranks, disciplinary backgrounds, and experiences. With regard to job documents, participants found that the time spent in the seminar intensively writing a teaching philosophy statement was extremely valuable to them on both in their academic job market search and in the long term for their tenure and promotion files. As participants prepared their job documents, we structured the seminar to allow multiple opportunities for participants to think about their own personal goals, interests, and motivations through prewriting prompts, generative interviews, and peer discussions.

As mentioned earlier in the paper, these peer discussions include conversations with a small cohort of individuals within similar disciplines and cross disciplinary conversations. The cross disciplinary conversations help reveal assumptions and disciplinary norms, while the ongoing discipline based conversations help refine documents. Given that CTLs are uniquely positioned to train graduate students in drafting and refining teaching philosophy statements, incorporating this training into PFF programming significantly benefits graduate student professional development. Although our data did not specifically address this, educational developers might also consider incorporating diversity statements as a part of their preparation given their increased use recently; as diversity statements have become more commonly requested for faculty positions, Center for Research on Learning and Teaching added diversity statement related training to the PFF seminar in addition to our teaching philosophy statement training in 2015.

Given the limitations of our data, another recommendation we would make to our educational developer colleagues is for more institutions to conduct long term assessment studies of PFF programs. If we can compare data from a range of institutions and across disciplines to better understand the long term value of these programs, PFF programs and graduate students across the country will benefit from those insights.

Conclusion

Our long term evaluation study indicates that the PFF seminar did not have a long term impact on faculty job outcome differences between participants and applicants to the program. However, the PFF seminar gave participants a greater perceived understanding of their options in academe and gave them confidence in their preparedness for their job search and interviews. Participants also found that the documents they generated in the program (a teaching philosophy statement and syllabus) were of continued value to them as junior faculty in their teaching and in preparing their tenure files. Thus, participants in these programs reported significant long term benefits of the participation in PFF activities. Our findings suggest that exposure to different institution types, training on research based principles for course design and learning, and drafting of job search teaching documents are keys to the long term positive impact of such programs.

References

• Austin, A. E. (2002). Preparing the next generation of faculty: Graduate school as socialization to the academic career. The Journal of Higher Education, 73(1), 94–122.
• Austin, A. E., & McDaniels, M. (2006). Preparing the professoriate of the future: Graduate student socialization for faculty roles. In J. C. Smart (Ed.), Higher Education: Handbook of Theory and Research (pp. 397–456). Netherlands: Springer.
• Connolly, M. R., Savoy, J. N., Lee, Y. G., & Hill, L. B. (2016). Building a Better Future STEM Faculty: How Doctoral Teaching Programs Can Improve Undergraduate Education. Madison, WI: Wisconsin Center for Education Research, University of Wisconsin Madison.
• Cook, C., Kaplan, M., Nidiffer, J., & Wright, M. (2001). Preparing future faculty faster. AAHE Bulletin, 54(3), 3–7.
• DeNeef, A. L. (2002). The preparing future faculty program: What difference does it make? PFF Occasional Paper Series. Number 8. Washington, D.C.: Association of American Colleges and Universities.
• Golde, C. M., & Dore, T. M. (2001). At Cross Purposes: What the Experiences of Doctoral Students Reveal about Doctoral Education. Philadelphia, PA: Pew Charitable Trusts.
• Golde, C. M., & Dore, T. M. (2004). The survey of doctoral education and career preparation: The importance of disciplinary contexts. In D. H. Wulff, A. E. Austin & Associates (Eds.), Path to the Professoriate: Strategies for Enriching the Preparation of Future Faculty. San Francisco, CA: Jossey Bass.
• Goldsmith, S., Haviland, D., Dailey, K., & Wiley, A. (2004). Preparing Future Faculty Initiative: Final Evaluation Report. American Association of Colleges and Universities. San Francisco: WestD. Retrieved from http://www.aacu.org/pff/pdfs/PFF_Final_Report.pdf.
• Hershock, C., Groscurth, C., & Milkova, S. (2011). Approaches to preparing future faculty for teaching. In C. Cook & M. Kaplan (Eds.), Advancing the Culture of Teaching on Campus (pp. 97–117). Sterling, VA: Stylus.
• Luft, J. A., Kurdziel, J. P., Roehrig, G. H., & Turner, J. (2004). Growing a garden without water: Graduate teaching assistants in introductory science laboratories at a doctoral/research university. Journal of Research in Science Teaching, 41(3), 211–233.
• McDaniels, M. (2010). Doctoral student socializations for teaching roles. In S. K. Gardner & P. Mendoza (Eds.), On Becoming a Scholar: Socialization and Development in Doctoral Education (pp. 29–44). Sterling, VA: Stylus.
• Nyquist, J. D., Manning, L., Wulff, D. H., Austin, A. E., Sprague, J., Fraser, P. K., Calcagno, C., & Woodford, B. (1999). On the road to becoming a professor: The graduate student experience. Change, 31, 18–27.
• Palmer, M. (2011). Graduate student professional development: A decade after calls for national reform. Studies in Graduate and Professional Student Development., 14, 1–17.
• Smith, K. (2001). Pivotal events in graduate teacher preparation for a faculty career. The Journal of Graduate Teaching Assistant Development., 8(3), 97–105.
• Walker, G. E., Golde, C. M., Jones, L., Bueschel, A. C., & Hutchings, P. (2008). The Formation of Scholars: Rethinking Doctoral Education for the Twenty First Century. San Francisco, CA: Jossey Bass.
• Weidman, J. C., Twale, D. J., & Stein, E. L. (2001). Socialization of graduate and professional students in higher education: A perilous passage? ASHE ERIC Higher Education Report, 28(3). San Francisco: Jossey Bass.
• Wulff, D. H., Austin, A. E., Nyquist, J. D., & Sprague, J. (2004). The development of graduate students as teaching scholars. In D. H. Wulff & A. E. Austin (Eds.), Paths to the Professoriate: Strategies for Enriching the Preparation of Future Faculty (pp. 46–73). San Francisco, CA: Jossey Bass.
• Wurgler, E., VanHeuvelen, J., Rohrman, S., Loehr, A., & Grace, M. (2013). The perceived benefits of a preparing future faculty program and its effect on job satisfaction, confidence and competence. Teaching Sociology, 42(1), 50–60.

Appendix A: Solicitation E mail

Dear [FirstName],

The Center for Research on Learning and Teaching (CRLT) is conducting a survey of graduate students and postdoctoral scholars about the long term impact of the May Preparing Future Faculty (PFF) seminar at the University of Michigan.

Your honest and timely response will assist us in tailoring programs and understanding the perspectives of graduate students and postdoctoral scholars from U M.

We ask that you complete the survey before March 10, 2014 at 11:59 PM.

Your participation is voluntary and all responses are confidential.

You may access this survey at the following website. (Just click on the link or copy and paste the link to open the survey):

[SurveyLink]

Thank you for your participation!

Center for Research on Learning and Teaching

Appendix B: Survey Instrument

U M Rackham CRLT Graduate Student Professional Development

We are interested in learning about the long–term impact of Rackham’s professional development opportunities on the careers of U–M graduate students and postdoctoral scholars. We would appreciate your completing the following survey.

CRLT staff may incorporate information from this survey in reports to school/college committees, granting agencies, other individuals and organizations interested in CRLTs programs, and in professional publications.

Your participation in this survey is voluntary. You may skip any questions you do not wish to answer. The survey should take you no more than 10 minutes to complete.

Clicking “Next” indicates your consent to participate in this survey research. If you have questions about the study, please contact Laura Schram (lnschram@umich.edu) or Tershia Pinder–Grover (tpinder@umich.edu) at the Center for Research on Learning and Teaching at U–M.

1. What year did you receive your graduate degree (Ph.D., terminal Master’s, or equivalent)?

• 2007

• 2008

• 2009

• 2010

• 2011

• 2012

• 2013

• I have not received my degree yet. Other (please specify)

2. At what stage are you in your career? Please check the ONE response that best describes your situation.

• I now have (or soon will begin) a faculty or post doc position

• I now have (or soon will begin) a nonfaculty position (e.g., industry, government, alternative academic)

• I am currently seeking faculty positions

• I am currently seeking nonfaculty positions

• I will begin applying for faculty positions within the next 12 months

• I will begin applying for faculty positions more than 12 months from now

• I am working on my degree and do not currently know when I will pursue a faculty career

• I am working on my degree and do not currently know if I will pursue a faculty career

• I am working on my degree and do not plan to pursue a faculty career

3. At what type of institution did you find your position?

• Associates College

• Doctorate–Granting University

• Master’s College/University

• Baccalaureate College

• Special focus institution (e.g., engineering, health professional school)

• Tribal College

• Other (please specify)

4. How would you categorize the funding of the institution where you found your position?

• Public

• Private, nonprofit

• Private, for–profit

5. How would you characterize the focus of your position? Please check the ONE response that BEST describes your situation.

• Exclusively research

• Mostly research

• Even division between research and teaching

• Mostly teaching

• Exclusively teaching

• Other (please specify)

6. How would describe your current position?

• Tenure track faculty

• Nontenure track faculty

• Postdoctoral scholar

• Other (please specify)

7. What is your current occupation?

8. Did you participate in the Rackham–CRLT Preparing Future Faculty (PFF) seminar while at U–M? (The PFF seminar is a five–week intensive seminar to prepare graduate students for faculty careers.)

• Yes

• No

9. What year did you participate in the PFF seminar?

• 2007

• 2008

• 2009

• 2010

• 2011

• 2012

• 2013

• Don’t know

As part of the PFF seminar, you prepared two documents: a statement of teaching philosophy and a syllabus. We would like to know about the usefulness of these documents in your career.

10. Did you or do you plan to use the statement of teaching philosophy in your faculty job search?

• Yes

• No

• N/A (did not do a faculty job search)

• Other (please specify)

11. How have you used the statement of teaching philosophy (e.g., was it required of job applications, did it guide the preparation of a sample class, did it provoke questions from search committees, did you use it in your tenure/promotion file), if at all? Please describe below.

12. Did you or do you plan to use the syllabus in your faculty job search?

• Yes

• No

• N/A (did not do a faculty job search)

• Other (please specify)

13. How have you used the syllabus (e.g., was it required of job applications, did it guide the preparation of a sample class, did it provoke questions from search committees, did you use it in your tenure/promotion file), if at all? Please describe below.

14. We would like to find out whether the Preparing Future Faculty (PFF) seminar was helpful to you in your career. Please indicate how helpful the following elements of the PFF seminar were to you in your career:

Options:

Not at all helpful

Slightly helpful

Moderately helpful

Very helpful

Extremely helpful

Do not recall

• Discussions of higher education

• Discussions and modeling of inclusive teaching

• Discussions and demonstration of instructional technology

• U–M faculty discussions with junior faculty about work life

• U–M faculty discussions with senior faculty about tenure

• U–M faculty discussions with lecturers

• Non–U–M faculty discussions about work life at different institutional types (e.g., Community College faculty and faculty at different campuses)

• Campus visit (to Albion College, Eastern Michigan University, Kalamazoo College, Kettering University, University of Toledo)

• Interview practice (e.g., mock interviews or teaching job talk practice)

• Writing a statement of teaching philosophy

• Designing a syllabus

Comments

15. The PFF seminar has a strong emphasis on teaching professional development (e.g., use of classroom assessment techniques, scholarship of teaching and learning, instructional technology, active learning, multicultural learning strategies). How valuable was the PFF seminar for your own teaching?

• Not at all valuable

• Slightly valuable

• Moderately valuable

• Very valuable

• Extremely valuable

• Do not recall

Please elaborate below.

16. The PFF seminar has a strong emphasis on exposing students to other institutional contexts (e.g., readings on the higher education landscape, discussions with faculty in a variety of institutional contexts, visits to other campuses). How helpful was the PFF seminar for your understanding about faculty work life at other types of institutions?

• Not at all helpful

• Slightly helpful

• Moderately helpful

• Very helpful

• Extremely helpful

• Do not recall

Please elaborate below.

17. In what other ways did the PFF seminar assist you in your career or job search?

18. What else might the PFF seminar include that would have helped you in your career?

19. What other professional development opportunities did you engage in while at U–M to help you with your future career?

20. Other comments about professional development opportunities at U M?

1. During these years, the demographic data associated with first generation status and undergraduate institution varied widely, so we selected a range of figures to most accurately reflect the population.

2. We did not distinguish in our records which participants were graduate students versus which were postdocs. Therefore, we cannot break down data reported on participants into these categories. Please note that all data reporting on job outcomes excludes current graduate students who have not yet completed their degrees.

3. Rackham Graduate School no longer uses the terminology “placement data” but rather “job outcome data.” This terminology reflects that graduates are not passively placed into careers by their departments but that graduates have agency to actively seek and earn a job of their choosing.

4. We asked respondents when they received their graduate degree (Ph.D., terminal Master’s, or equivalent). However, we did not ask respondents to indicate which type of degree they received. Therefore, we cannot make conclusions about differences between those who graduate with a Master’s degree compared to those with a Ph.D.

5. Note: In order to have the minimum expected value of 5 required for a chi square test in each cell, we had to collapse “associates” and “special focus” institutions in to one category.

6. All initial qualitative analyses were conducted by Laura Schram, the only member of our research team with a social science doctorate and extensive training in qualitative methodologies. Initial coding was then reviewed by other members of the research team, and modifications were made based on the team’s discussion of any coding concerns.