Abstract

This article includes a brief rationale and review of the literature on peer review of teaching (PRT). Based on that literature review, it offers a proposal for an optimal formative review process that results in a teaching portfolio that would reflect a faculty member’s efforts and successes in a critically reflective PRT process, and contributes to ongoing teaching improvement. It then looks at potential areas of faculty resistance and concern and offers a discussion of potential strategies to overcome those concerns.

Keywords: peer review, faculty, higher education, faculty development, teaching development

What Is Peer Review of Teaching?

When properly done, peer review of teaching (PRT) is a critically reflective, complex, and multifaceted, collaborative, and developmental approach to improving instructional excellence. It has been shown to improve teaching and to promote dialogue among peers concerning insights about teaching and the instructional processes, and overall improvement of instructional practices (Bell, 2001; Shortland, 2004). It includes the triangulation of student, peer, administrative, and self evaluation. Its criteria and procedures should be embedded in both disciplinary culture and good educational practice and be informed by university and departmental missions. Finally, its development and implementation can be a valuable part of academic roles and responsibilities.

Much of what we do as faculty is subject to peer review, scholarship, awards, grants and promotions, and tenure. Ware (2008) found that 90% of faculty viewed peer review of scholarship favorably, yet introducing a systematic process of peer review of teaching in an institution or department is often met with suspicion and resistance. As one faculty member put it: “When someone criticizes my scholarship, they are criticizing my ideas; when they criticize my teaching, they’re criticizing me!”

The relational nature of teaching contributes to this sense that teaching is “personal,” and that criticism is “rejection.” At the same time, PRT has been shown to be an effective way to improve teaching in higher education. Research suggests that PRT, when systematically embraced and implemented, improves the quality of teaching at an institution (Donnelly, 2007; Lomas and Nichols, 2005; Harris, Farrel, Bell, Devlin, & James, 2008; Toth, K. E., & McKey, C. A. (2010) Additional benefits of PRT include:

  • Feedback on all key aspects of teaching—such as the identification of learning goals and objectives and the design of curricula, resources, and assessment—accommodates the full spectrum of university teaching and learning contexts.

  • The adaptability of PRT within diverse teaching and learning environments, such as clinical, field based, and online teaching.

  • Recognition and utilization of the way in which teaching and learning practices are embedded within the disciplines.

  • Opportunity for dialogue about, and the exchange of ideas on, teaching and learning that is framed by recognized principles of good teaching practice both generally and as defined in the disciplines.

  • Both the reviewer and the reviewee benefit from the dialogic quality of PRT.

  • Engaging in critical reflection on a colleague’s teaching yields insights into an individual’s own practice, while feedback from peers provides a unique perspective on teaching that other evaluation methods may lack.

  • PRT strengthens the teaching culture of an institution; it has the potential to contribute to collegial academic cultures in which critical reflection on teaching is valued and encouraged.

  • Collegiality is built by increasing communication between teaching staff, and enhancing knowledge of the broader curriculum also benefits the immediate academic environment and the institution (adapted from Harris et al., 2008, p. 6).

  • PRT strengthens the message that teaching is valued at the institution.

  • PRT provides an opportunity to identify and learn from expert teachers.

Limitations of Student Evaluation of Teaching

It has become increasingly clear that student ratings are a necessary but insufficient method of evaluating the quality of teaching and learning in a classroom, be it face to face or online. In their review of the literature on student evaluation of teaching (SET) effectiveness, Spooren, Brockx, and Morelmans (2013) state “… the inclusion of SET in a more holistic approach that stimulates teachers to be and remain to be reflective practitioners concerning their teaching …” (p. 624) is necessary. Further, there is strong evidence that student ratings do not necessarily persuade teachers to improve, either because they are too harsh and ultimately discourage faculty from placing emphasis on teaching or because they are too vague and faculty are left uncertain about what steps are needed in order to improve their teaching (Emery, Kramer, and Tian, 2003; Braskamp, 2000). While student ratings appear to be a valid measure of student satisfaction with classroom teaching, satisfaction is only one of several teacher and course characteristics that contribute to student learning (Ackerman, Gross, and Vigneron, 2009). Further, students are “… not in a good position to judge factors such as instructor knowledge, quality of readings, course management, and academic standards used in grading, but these factors may have a strong impact on student learning” (Murray, 2005). It is also clear that numerous contextual and demographic factors that are often beyond the faculty member’s control can affect student ratings (Benton and Cashin, 2012; Jacobs, 2002; Marsh, 2007), including:

  • Course size (large classes tend to receive lower ratings overall).

  • Subject (statistics and math, for instance, get rated lower than humanities courses).

  • Gender (female faculty tend to get lower ratings from male students enrolled in their courses, especially in subjects that are more traditionally perceived as male dominated, e.g., engineering).

  • Race (faculty of color tend to receive lower ratings from students, compared to the ratings of their white peers).

  • Cultural communication styles (expressiveness, assumptions about roles and acceptable classroom behaviors, and logic structures, that differ from the mainstream).

  • Language variations (accents): This includes both issues of understanding, and stereotypes associated with particular cultural groups.

Comprehensive Assessment of Teaching

Key measures identified as optimal in assessing teaching performance that are above and beyond SETs include comprehensive consultations distinguished by Gray and Bergmann (2003), Penny and Coe (2004), and Toth and McKey, 2010):

  • Active involvement of teachers in the review process.

  • Interaction with peers around issues of teaching and learning in a nonthreatening setting (conferences, workshops, seminars, faculty learning communities).

  • Articulation and examination of the instructor’s conception (philosophy) of teaching and learning in general, and within their discipline specifically.

  • Emphasis on critical reflection that encourages pedagogical thinking on the part of the instructor.

  • Use of high quality feedback information from multiple sources (e.g., analysis of student ratings for improvement, syllabus review, peer observations; video recording of teaching, teacher reflections that articulate their rationale for the various strategies and approaches they employ, and instructor self ratings that can be compared to those given by students and peers) that can be collected in a portfolio and presented for a summative review that focuses both on teaching quality and teaching improvement.

  • Sufficient time for ongoing dialogue and interaction with reviewers.

  • Developing iterative goals for continuous improvement and follow up.

Faculty Learning Communities

Due to the highly personal nature of teaching and the potentially threatening nature of peer review, PRT is best engaged in collaboratively (Hatzipanagos, and Lygo Baker, 2006; Ambler, Chevan, Clarke, & Mathews, 2014) within a faculty learning community, that is, an ongoing collaboration with peers that is focused on learning and academic exploration. Faculty learning communities are “a continuous process of learning and reflection, supported by colleagues, with an intention of getting things done” (McGill and Beaty, 2001, p. 11) that “emphasize collaborative visioning regarding what learning can be and collaborative construction of models of learning” (Layne, 2012, p. 1).

Ten Necessary Qualities for Building Community

The following qualities guide the design and process of a faculty learning community (Ambler et al., 2014; Cox, 2005).

  • Safety and trust. In order for participants to connect with each other, there must be a sense of safety and trust. This is especially true as participants reveal weaknesses in their teaching or ignorance of teaching processes or literature.

  • Openness. In an atmosphere of openness, participants can feel free to share their thoughts and feelings without fear of retribution. For example, in “Community Using Difference to Enhance Teaching and Learning” at a large US university, participants were able to openly discuss ways that other participants or colleagues offended them.

  • Respect. In order to coalesce as a learning community, members need to feel that they are valued and respected as people. It is important for the university to acknowledge their participation, and financially support community projects and attendance at related conferences.

  • Responsiveness. Members must respond respectfully to each other, and the facilitator(s) must respond quickly to their participants. The facilitation should welcome concerns and preferences and, when appropriate, share these with individuals and the community.

  • Collaboration. The value of collaboration in consultation and group discussion for individual members’ projects, and on achieving learning outcomes, hinges on the group’s ability to work with, and respond to, each other. Joint projects and presentations should also be welcomed.

  • Relevance. Because learning outcomes are enhanced by relating the subject matter to the participants’ teaching, courses, scholarship, and life experiences, all participants should be encouraged to seek out and share teaching and other real life examples to illustrate their learning outcomes.

  • Challenge. Expectations for the quality of outcomes should be high, engendering a sense of progress, scholarship, and accomplishment. Sessions should include, for example, those in which individuals share syllabi and report on their individual projects.

  • Enjoyment. Activities must include social opportunities to lighten up and bond, and should take place in invigorating environments. For example, a retreat can take place off campus at a nearby country inn, state park, historic site, or the like.

  • Esprit de corps. Sharing individual and community outcomes with colleagues in the academy should generate pride and loyalty.

  • Empowerment. A sense of empowerment is both a crucial element and a desired outcome of participation in a faculty learning community. In the construction of a transformative learning environment, the participants gain a new view of themselves and a new sense of confidence in their abilities. Faculty leave learning communities with better courses, and a clearer understanding, of both themselves and their students as a result of their participation. Key outcomes include scholarly teaching, and contributions to the SoTL.

The Formative and Summative Peer Review of Teaching

Two approaches to PRT are most commonly identified in the literature: formative (i.e., ongoing and teacher driven) and summative (a one time evaluation of teaching quality for a specific purpose) (Chism, 2007a). Whether PRT is mandatory or voluntary, formative or summative, it is still necessary to decide on a well defined process, be clear about the distinction between formative and summative PRT, and acknowledge the importance of sequencing. Quality preparation of both reviewer and reviewee to engage in in the review process is essential if the teacher is to achieve maximum benefits from the review (Chism, 2007, b); Marsh. H. W. (2007 ; Lomas and Kinchin, 2006; Cavanaugh, 1996).

Formative Peer Review of Teaching

Formative PRT is focused on the long term enhancement of teaching and learning. Even when mandatory, the process should be primarily driven and guided by the faculty member’s personal goals, by feedback from students and/or colleagues, and/or by a desire to address problems in a specific course or academic context (Arreola, 2007).

The overall consensus in the literature on PRT (Bernstein, Jonson, & Smith, 2000; Blackmore, 2005; Bovill, 2008) is that a high quality formative review should have the following characteristics:

  • Faculty ownership. The faculty member’s own concerns and goals for her/his teaching should guide this aspect of the process. Even if PRT is mandatory, faculty members need to have input on the central focus of the review process. At the same time, if PRT takes place within a department or at an institution that also requires a summative review, the process and the criteria should be transparent and reflective because the criteria will ultimately be applied in the summative process. If at all feasible, some consensus between and among the faculty about what “good teaching” actually is should be achieved through a critical and collegial dialogue.

  • Confidentiality. Discussions of teaching should be kept between the faculty and the peer reviewer and are not shared with others unless the reviewee chooses to do so. Keeping discussions confidential allows the peers to develop a trusting relationship, one that provides a safe environment where faculty can be open to reflecting on the feedback they receive and develop the knowledge and strategies necessary to improve their teaching practice.

  • Relationship of equals. Colleagues need to work together as equals, rather than a reviewer serving as an expert and the colleague being reviewed as an object of scrutiny. The process is consultative, not evaluative, and can include academics from within or outside of the school or institution, or within an academic learning community where co mentoring and dialogue around the issues of teaching and learning can take place.

  • Collegial feedback. The colleague making an in class observation, reviewing a course syllabus, or performing other PRT tasks, provides feedback that is constructive and collegial rather than evaluative, which ideally takes place within a dialogic format, where issues can be discussed, problems addressed, and plans made for steps toward improvement (Byrne, Brown, & Challen, 2010; Roxa & Martensson, 2009).

  • Open ended process. Teaching in higher education should be viewed as scholarship (Bastow, 2008; Richlin & Cox, 2004; Simpson & Anderson, 2010) and like all forms of scholarly endeavors, it occurs over time with cycles of practice, review, and application of feedback. As a result, PRT consultations shouldn’t be viewed as isolated events, but rather as part of an ongoing process of improvement.

  • Multiple sources of information. In order to gain a comprehensive and valid perspective on one’s quality of teaching, the review needs to be grounded in several sources, which may include (but are not limited to) course teaching materials, classroom visits, written evaluations from students, conversations with students, video recordings of teaching, the faculty member’s teaching portfolio (teaching statement, collection of teaching artifacts, such as student feedback, syllabi, student assessments, examples of student work), self evaluations, and teaching reflections. The triangulation of student, teacher, and peer reviewer perspectives is crucial to the creation of high quality feedback.

As faculty progress through their formative PRT process, opportunity for instructional improvement and documentation of efforts and accomplishments around issues of learning and teaching can include (but need not be limited to) the four categories described below. Categories I through IV are seen as progressively sophisticated. Activities undertaken should be accompanied by a brief reflection that explores what learning and insights into their teaching resulted from each (Cosh, 1998).

  • Category 1: Direct Effort to Improve Teaching

    •  ▫ Courses/Workshops on topics relevant to university teaching

    •  ▫ Review of course materials with evidence of feedback and resulting action

    •  ▫ Development of a teaching philosophy

    •  ▫ Development of a teaching portfolio

    •  ▫ Classroom observation conducted by:

      •   ◦ An interdisciplinary peer

      •   ◦ An intradisciplinary peer (including other faculty from the same institution or another)

      •   ◦ An instructional development professional

    •  ▫ Systematic collection of formative feedback from students using Classroom Assessment Techniques (Angelo & Cross, 1993)

    •  ▫ Small Group Instructional Diagnosis (or SGID) midsession review, conducted by a peer and followed by a consultation

    •  ▫ Video recording of one’s teaching with self reflection and consultation

    •  ▫ Observation of expert teachers, followed by conversation and reflection with the faculty observed

  • Category 2: Course Materials and Curriculum

    •  ▫ Multiple curricula developed for face to face and online courses

    •  ▫ Materials, resources, activities, and student assessments

    •  ▫ Rationale for why a particular teaching or learning option was chosen

    •  ▫ Alignment of assessments and learning and teaching strategies with course goals and objectives

    •  ▫ Alignment of course curriculum with departmental and institutional learning outcomes

  • Category 3: Grants Related to Teaching Development, Curriculum Development, and/or SoTL

  • Category 4: Scholarly Publications on Topics of Learning and Teaching

    •  ▫ Scholarly textbooks, chapters in books used as texts, and other publications designed primarily for classroom and instructional settings

    •  ▫ Journals articles based on classroom research, activities, or development of theories and models of learning and/or teaching

Summative Peer Review

Summative reviews are done in order to evaluate a faculty member’s work for annual reviews, lifting probation, merit, and promotion (Crisp et al., 2009; Murphy, MacLaren, and Flynn, 2009). Such reviews are mandatory and have a bearing on the success or failure on the faculty within a defined context. The overall consensus is that summative review contrasts with formative in several respects. In summative peer review, the following should be considered:

  • The primary goal is to formally evaluate performance. A summative review of faculty work often includes goals for future growth and development, but the primary purpose of the review is to assess the faculty member’s performance relative to specific and well defined criteria.

  • It is viewed as identifying teaching excellence. Ideally, the focus of a summative review should be identifying and rewarding teaching excellence, rather than punishing those who fall short.

  • The criteria are set by the department or institution and are understood by both the reviewer and the reviewee. In a summative review, the faculty member’s goals are certainly relevant, but external criteria (departmental, school, institutional, etc.) are decisive. A summative review may involve discipline based criteria (e.g., from accrediting agencies and standards boards) as well as criteria defined by a particular discipline, school, or institution. Transparency of evaluation standards and trust resulting from a supportive climate for faculty created throughout the formative review process is essential here.

  • Limited confidentiality. A summative review is semipublic in that it will likely be seen and reviewed by key department members and upper level administration.

  • Relationships are unequal. For summative review, the relationship between the reviewer and the person being reviewed is not a true peer relationship. A summative review is, by definition, required to make judgments of the other’s teaching and to evaluate that teaching not only on its own merits but also based on “ideal” criteria; thus, the reviewer becomes an evaluator.

  • Evaluative feedback. The feedback from a summative review may be couched in collegial and supportive language but is nonetheless an evaluation.

  • Deadline driven. The faculty member cannot set her/his own deadlines for review but must meet deadlines set by the department and/or university based on contractual timelines for performance evaluations.

  • Multiple sources of information. Ideally, a summative review of teaching, like a formative review, is based on multiple sources of information collected over time by means of multiple observations, review of course materials, discussions with and evaluations by students, and other interactions between reviewer(s) and faculty member. Having a multiplicity of data for review is even more important in a formal summative review that contributes to lifting faculty probation, promotion, or merit and should be built on a quality formative process.

It can be useful to consider summative PRT as a mandatory formative process where documentation that requirements were met (i.e., that the faculty engaged in the required number of activities) with any discussions kept confidential between those directly engaged in the peer review. In this model, peer reviewers could get together and identify issues that could be addressed at a school level regarding enhancement of teaching without violating individual confidentiality.

What Is Good Teaching?

While trying to define something as variable as “good teaching” can easily result in endless lists of strategies, learning theories, and discussions of accessibility and inclusiveness, it is an essential question to consider when PRT is being conducted (Elton, 1998). Coming to a consensus about what “good teaching” actually involves can help remove fear about the degree of the “subjectivity” that is so troublesome to the faculty being reviewed. A survey of studies (Lefoe, Philip, O’Reilly, and Parrish, 2009; Bransford, Brown, & Cocking, 2000; Kember, 2009; Layne, 2012; Scott, Coates, & Anderson, 2008; Toth, and McKey, 2010) of both “master teachers” and students suggests that there are key characteristics that occur repeatedly and are drawn with a broad enough brush to allow for a variety of different teaching styles, disciplinary concerns, and teacher personalities. These characteristics include:

  • Love of one’s subject, while vital, is necessary but not sufficient. It is the desire to share one’s love of the subject that is required in a teacher.

  • A working understanding and concern for how students learn comes both from having a basic knowledge of learning theories in higher education and from eliciting feedback from the students themselves.

  • The ability to make the material stimulating and interesting by using multiple teaching and learning strategies that are inclusive of all students and promote deep learning.

  • A facility for engaging with students at their level of understanding and then pulling, pushing, and enticing them to move beyond their initial assumptions, think critically, and integrate new knowledge and perspectives into preexisting knowledge.

  • Is knowledgeable in their discipline and yet able to explain the material clearly to novice learners.

  • Demonstrates caring about student learning, and respect for students needs and concerns through interaction inside and outside the classroom, including opportunities for student mentoring.

  • Includes students in the learning process through interaction, questioning, responsiveness, and an ability to create relevance by relating material to their experiences, interests, and real world concerns and events.

  • Uses assessment methods that are congruent with course objectives, materials, and activities.

  • Focuses on key concepts and students’ current and future understanding of them, rather than simply “covering content.”

  • Demonstrates a commitment to giving high quality feedback on students’ work.

  • Has a desire to solicit and incorporate feedback from students and peers about how ones teaching can be improved.

The preceding criteria are a place to start the dialogue among faculty about good practice, not a definitive list. It is purposefully broad and should be clarified, added to, and elaborated on based on institutional and disciplinary contexts.

Best Practice in Peer Review

In summary, PRT should ideally:

  • Begin early in the faculty appointment in conjunction with opportunities for education, reflection, and support around the issues of teaching and learning.

  • Take place in faculty learning communities.

  • Be openly discussed among faculty who should be intimately involved in determining the range of teaching practices they wish to include in the reviews and the specific criteria that will be used for evaluation.

  • Occur in a climate and relationship of trust and safety where it is viewed and “helpful” rather than pejorative.

  • Be approached developmentally and viewed as a shared commitment to improving student learning.

  • Be rigorous and relevant and based on the recognition that teaching is a scholarship.

  • Be driven by the needs and concerns of the faculty being reviewed and doing the reviewing.

  • Include multiple sources of data that are collected over time, are integrated within the context of the discipline and the school.

  • Include self assessment that allows individual faculty members to explain the goals and intentions of their courses and teaching, and the philosophy of their teaching that informs their practice, and encourages self reflection to improve their teaching.

  • Allow for different teaching styles that are appropriate and effective for the courses and students being considered.

  • Be part of an overall, ongoing process to continually improve teaching on the individual and institutional level.

  • Be transparent with regard to the criteria and rationale used.

  • Be situated within the culture and values of the institution.

  • Have instructional enhancement as its ultimate goal. Thus, it is necessary for faculty to:

    •  ▫ Have a variety of opportunities for teaching improvement and critical reflection on their teaching before a summative review is performed.

    •  ▫ Obtain both formative reviews from multiple sources (intradisciplinary and extradisciplinary peers, and students who can provide feedback from a variety of perspectives).

    •  ▫ Be supported in making necessary and desired changes. Faculty should use what they learn through the multiple sources of feedback (self, student, peer, administrator) to make informed decisions to improve their teaching, and then see ongoing input and feedback about these changes.

  • Finally, any summative review should be reflective of the educator’s efforts to improve teaching over time and involve documentation and critical reflection about that process.

Components of Peer Review of Teaching

Multiple sources of information should be used in PRT so that a variety of perspectives on the educator’s efforts can be considered (Bernstein, Burnett, Goodburn, & Savory, 2006; Chism, 2007a; Seldin, Miller & Seldin, 2010). Possible sources include:

  • Students. Students are an excellent source of information on what helps them learn.[1] Student feedback should be collected at various points throughout the course (ongoing formative, midsession, and end of session) through:

  • Written formative assessment from students throughout the course on their learning.

  • Consultation between the faculty and student focus groups.

  • Consultation, usually at midsemester, with all the students in a class (SGID = Small Group Instructional Diagnosis) must be done by a neutral party who knows how to solicit quality information and synthesize it in a useful format for the instructor (CTL staff and/or out of department instructor).

  • SET forms.

  • Samples of student work.

  • Colleagues who can provide feedback on aspects of the course or teaching materials they have reviewed. Colleagues can also confirm or clarify expectations within the department, standards in the discipline, and much more.

    •  ▫ Intradisciplinary peers from within or external to the institution.

    •  ▫ Extradisciplinary peers (expert teachers, teaching and learning professionals, peer mentoring support groups).

  • Peer observation of teaching.

  • Videotaping one’s teaching.

  • Instructor self reflection.

  • Information on the course design, goals, objectives, and so on.

  • Comments on the course (personal and incorporating student feedback): What they think worked well (or didn’t), how this class compares with other courses taught by this faculty member or with the same course taught previously, and so on.

  • Perspective on what was learned from the feedback given and what changes might have resulted.

  • Materials from the course: syllabus, assignments, texts, handouts, exams or other assessments, and the like.

Data must be gathered at multiple points in time. Making changes to one’s teaching is often part of a larger goal to improve student learning, a process that occurs over time. The faculty member needs feedback at several points in time to monitor progress. Ideally, a feedback loop is constructed whereby a faculty member implements a change, receives feedback on the change, and then makes appropriate adjustments in response to the feedback.

Format of the Review

The most common format for PRT has been one on one, usually a new faculty member being reviewed by a more experienced faculty member. While this is a very workable model, it is not the only one, and there is some evidence that other formats can be just as effective in facilitating teaching improvement:

  • Academic learning communities of 3 to 5 members where there are discussions of teaching around issues that are raised by the PRTs that are done in a round robin format (i.e., faculty 1 reviews faculty 2, faculty 2 reviews faculty 3, faculty 3 reviews faculty 1). While the results of the PRT remains between the reviewer and the reviewee, problem solving and discussions of strategies, approaches, and the whys and wherefores of improvement can be discussed as a group.

  • Mutually reviewing pairs of faculty peers who are at the same level.

Because the nature and quality of feedback and reflection are key, either of these formats should be started by a teaching and learning facilitator (preferably from the institution’s CTL) who outlines the process and criteria to be used, and can be called upon to provide additional support and insight into issues that arise (Shortland, 2010). Alternatively, they can be part of or following ongoing workshops on teaching and learning in higher education.

Faculty Concerns and Resistance

Research into faculty resistance to engaging in mandatory PRT (Ambler, et al, 2014; Hamersley Fletcher & Orsmond, 2004; Lomas and Nichols, 2005), especially when it is attached to a summative PRT process, suggests that instituting and carrying out mandatory PRT is not an easy task. Interviews with faculty and administration on US and Australian campuses uncovered explicit concerns about PRT that are congruent with categories of concern in existing literature.

  • Fear of exposure as a “bad teacher”:

    •  ▫ Will it only be done summatively, that is, will a sudden, one shot peer observation of teaching be used to determine their standing as a “good or a “bad” instructor?

    •  ▫ Will data be used for punitive purposes, rather than to identify excellence as well as areas that can be improved?

    •  ▫ Will their results be treated with confidentiality?

    •  ▫ Critiques of teaching take on a personal quality that is identity focused and humbling.

  • Fear of negative fallout with colleagues:

    •  ▫ Will giving negative feedback create tension or alienation between colleagues?

    •  ▫ Do they risk losing colleagues’ respect if improvement is needed?

  • Distrust of administrative interference and resistance to “top down” implementation:

    •  ▫ Feeling of surveillance, sorting, punishment by department if they are reviewed negatively.

    •  ▫ Will the process be vague or unclear to reviewers and/or reviewees?

    •  ▫ Lack of staff cooperation.

  • Concern about ambiguity and/or rigor of the process:

    •  ▫ With the methods and instruments used be valid and reliable? Qualitative and/or quantitative?

    •  ▫ Will reviewer(s) have adequate background and understanding of context, content, students, course, and session objectives?

    •  ▫ How will they find time to properly engage in the process when their schedules are already full?

    •  ▫ Will there be proper appreciation for diverse teaching styles and methods and strategies specific to the discipline?

Strategies to Overcome Resistance

The best way to overcome any form of resistance is to find out the concerns that cause it and, to the greatest degree possible, address them. Research on faculty concerns about engaging in PRT at Macquarie University suggests that overcoming concerns requires that they

… explicitly reposition peer review as a quality enhancement activity, and deliberately and self consciously distancing it from quality assurance and managerial control. This means placing peer review in an open culture in which teaching is a shared activity that is respected and rewarded. (Sachs, Parsell, & Jacenyik Trawoger, 2012, p. 10)

The literature echoes these findings and suggested concerns could be addressed by being sure that:

  • There is adequate training and preparation for reviewers.

  • The process is well defined and understood.

  • There is enough time to build formative data and improve skills before summative feedback is due.

  • The primary focus is on formative aspects of the process.

  • Criteria are developed for online as well as face to face teaching.

  • The purpose and criteria are clear.

  • Trust is built between academic peers engaged in the process and relationships are clearly defined (Chism, 2007b; Healey, Ambler, Irhammar, Kilfoil, & Lyons, 2014).

  • The process is transparent.

  • Timing/sequencing of feedback and reviews are geared to provide opportunity for optimal improvement.

  • Make it a clear and significant part of their ongoing review, reward and ranking process.

Final Recommendations

Recommendation 1: That universities use the following framework to implement peer review of teaching. Implementing a comprehensive PRT program is best accomplished by incorporating it into preexisting contexts. Expanding rigor and creating new programs and contexts will be much easier if it is smoothly integrated into structures with which faculty are already familiar.

Recommendation 2: Role of the reviewer. Peer reviewers should be identified by schools as skilled teachers and trained by the teaching and learning consultants in the philosophy, process, and criteria for doing a PRT. Faculty should be given the opportunity to choose both inter and intra disciplinary[2] peer reviewers from the roster of certified peer reviewers who can provide feedback on aspects of the course or teaching materials, teaching observations, and reflections on their teaching philosophy (Carter, 2008; Courneya, Pratt, & Collins, 2008).

Recommendation 3: Defining criteria. A discussion about the criteria for good teaching is led by the deans and department chairs, and that the outcomes inform the PRT process. Addressing faculty concerns will require that administration, centers for teaching and learning, colleges, and departments engage in dialogue around developing criteria that defines “good teaching” generally at their university and within the disciplines (Healey, and Jenkins, 2003), identifying and training the peer reviewers, and developing timelines that are appropriate and suited to departmental review schedules.

Recommendation 4: Valuing teaching excellence. Recognition and rewards should be given to certified reviewers for the time and expertise invested. The process and purpose of the PRT should be transparent, and its value should be appropriately valued and reflected in departmental and school review, reward, and ranking. Those who serve as peer reviewers as well as those who are given exemplary ratings during the review should be appropriately recognized and rewarded in the yearly review, tenure, and promotion processes. This is particularly crucial with the increasing pressure being put on departments to increase research activity. Stressing to all concerned Boice’s (2000) findings with regard to early attention to understanding how students learning and improving teaching skills ultimately results in better all around faculty performance, research, and publication.

Recommendation 5: A template should be created to help streamline documentation and compilation of evidence for both the faculty and the reviewers.

Conclusion

When properly done, PRT has been shown to greatly improve both the quality of teaching and the dialogue about teaching among faculty peers. For this to happen, PRT should be enacted in a community of learners, have significant formative assessment that is recorded in a teaching and learning portfolio, and involve a broad range of evidence that documents the faculty’s efforts to improve their teaching over time.

Notes

    1. All student feedback must be anonymous to the instructor if it is expected to be honest and substantive.return to text

    2. Intradisciplinary peers: from within or external to the university; interdisciplinary peers: expert teachers, teaching and learning professionals, peer mentoring support groups.return to text

    References

    • Ackerman, D., Gross, B. L., & Vigneron, F. (2009). Peer observation reports and student evaluations of teaching: Who are the experts? Alberta Journal of Educational Research, 55(1), 18–39. Retrieved from http://ajer.synergiesprairies.ca/ajer/index.php/ajer/index
    • Ambler, T., Chevan, M., Clarke, J., & Mathews, N., (2014). Climates of communication: Collegiality, affects, spaces and attitudes in peer review. In H. Eggins & R. Macdonald (Eds.), The scholarship of academic development (pp. 67–84). Philadelphia, PA: Society for Research into Higher Education & Open University Press.
    • Angelo, T., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. Jossey Bass.
    • Arreola, R. (2007). Developing a comprehensive faculty evaluation system (3rd ed.). Bolton, MA: Anker.
    • Bastow, K. (2008). Teaching as scholarship. Retrieved November 10, 2012, from http://www.pharmacy.unc.edu/research/labs/bastow lab/reflections on teaching as scholarship.pdf
    • Bell, M. (2001). Supported reflective practice: A proogramme of peer observation and feedback for academic development. Journal of Academic Development. 6 (1), 21 28.
    • Benton, S. L., & Cashon, W. E. (2012). Student ratings of teaching: A summary of research and literature. IDEA Papers. Retrieved from: http://www.theideacenter.org/sites/default/files/idea-paper_50.pdf
    • Bernstein, D., Burnett, A., Goodburn, A., & Savory, P. (2006). Making teaching and learning visible: Course portfolios and the peer review of teaching. University of Nebraska Press.
    • Bernstein, D., Jonson, J., & Smith, K. L. (2000). An examination of the implementation of peer review of teaching. New Directions for Teaching and Learning, 83(Fall), 73–86.
    • Blackmore, J. (2005). A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management, 19(3), 218–232.
    • Boice, R. (2000). Advice for new faculty members: Nihil Nimus. San Francisco, CA: Jossey Bass.
    • Bovill, C. (2008). Peer observation of teaching guidelines. Retrieved from the University of Glasgow Teaching and Learning Centre website: http://www.gla.ac.uk/services/learningteaching/contactus/drcatherinebovill/
    • Bransford, J. Brown, A. L., & Cocking, R. R. (2012). How people learn: Brain, mind and experience and school. Washington, DC: National Academy Press.
    • Braskamp, L. (2000). Towards a more holistic approach to assessing faculty as teachers. New Directions for Teaching and Learning, 83(Fall), 19–33.
    • Byrne, J., Brown, H., & Challen, D. (2010). Peer development as an alternative to peer observation: A tool to enhance professional development. International Journal for Academic Development, 15(3), 215–228.
    • Carter, V. (2008). Five steps to becoming a better peer reviewer. College Teaching, 56(2), 85–88.
    • Cavanaugh, R. (1996). Formative and summative evaluation in the faculty peer review of teaching. Innovative Higher Education, 20(4), 235–240.
    • Chism, N. (2007a). Peer review of teaching: A sourcebook (2nd ed.). Bolton, MA: Anker. [
    • Chism, N. (2007b). Why introducing or sustaining peer review of teaching is so hard, and what you can do about it. Department Chair: A Resource for Academic Administrators, 18(2), 6–8.
    • Cosh, J. (1998). Peer observation in higher education—a reflective approach. Innovations in Education and Teaching International, 35(2), 171–176.
    • Courneya, C., Pratt, D. D., & Collins, J. (2008). Through what perspective do we judge the teaching of peers? Teaching and Teacher Education: An International Journal of Research and Studies, 24(1), 69–79.
    • Cox, M. (2005). Introduction to faculty leaning communities. New Directions for Teaching and Learning, Special Issue: Building Faculty Learning Communities, 97, 15–23.
    • Crisp, G., Sadler, R., Krause, K., Buckridge, M., Wills, S., Brown, C., … Broughman, B. (2009). Peer review of teaching for promotion purposes. Syndey: Australian Learning and Teaching Council.
    • Donnelly, R. (2007). Perceived impact of peer observation of teaching in higher education. International Journal of Teaching and Learning in Higher Education. 19(2), 117–129. Retrieved from http://www.isetl.org/ijtlhe/
    • Elton, L. (1998), Dimensions of excellence in university teaching. International Journal of Academic Development, 3(1), 3–11.
    • Emery, C., Kramer, T., & Tian, R. (2003). Return to academic standards: A critique of student evaluations of teaching effectiveness. Quality Assurance in Education, 11(1), 37–46.
    • Gray M., & Bergmann, B. R. (2003). Student teaching evaluations: Inaccurate, demeaning, misused. Academe Online. Retrieved from http://www.aaup.org/aaup/pubsres/academe/2003/so/feat/gray.htm
    • Hammersley Fletcher, L., & Orsmond, P. (2004). Evaluating our peers: Is peer observation a meaningful process? Studies in Higher Education, 29(4), 498–503.
    • Harris, K., Farrel, K., Bell, M., Devlin M., & James, R. (2008). Peer review of teaching in Australian higher education. Sydney: ALTC.
    • Hatzipanagos S., & Lygo Baker, S. (2006). Teaching observations: A meeting of minds? International Journal of Teaching and Learning in Higher Education, 17 (2), 97–105. Retrieved from http://www.isetl.org/ijtlhe/
    • Healey, M., Ambler, T., Irhammar, M., Kilfoil, W., & Lyons, J. (2014). International perspectives on peer review as quality enhancement. In J. Sachs & M. Parsell (Eds.), Peer Review of Learning and Teaching in Higher Education, Professional Learning and Development in Schools and Higher Education (Vol. 9). New York, NY: Springer.
    • Healey, M., & Jenkins, A. (2003). Discipline based educational development. In H. Eggins & R. Macdonald (Eds.), The scholarship of academic development (pp. 47–57). Philadelphia, PA: Society for Research into Higher Education & Open University Press.
    • Hodges, L., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31(5), 279–286.
    • Hutchings, P. (1996). The peer review of teaching: Progress, issues and prospects. Innovative Higher Education, 20(4), 221–234.
    • Jacobs, L. (2002). Student ratings of college teaching: What research has to say. Retrieved from http://reference.kfupm.edu.sa/content/c/o/course_characteristics_77830.pdf
    • Kember, D. (2009). Promoting student centred forms of learning across an entire university. Higher Education 58:1–13.
    • Kemp, R., & Gosling, D. (2000). Peer observation of teaching. Retrieved from the Higher Education Academy Subject Centre for Education ESCalate: http://escalate.ac.uk/resources/peerobservation2/
    • Layne, L. (2012). Defining effective teaching. Journal on Excellence in College Teaching, 23(1), 43–68.
    • Lefoe, G., Philip, R., O’Reilly, M. & Parrish, D. (2009). Sharing quality resources for teaching and learning: A peer review model for the ALTC Exchange in Australia. Australasian Journal of Educational Technology 2009, 25(1), 45 59.
    • Lomas, L. & Kinchin, I. (2006). Developing a peer observation program with university teachers. International Journal of Teaching and Learning in Higher Education. 18 (3), 204 214
    • Lomas, L., & Nicholls, G. (2005). Enhancing teaching quality through peer review of learning and teaching. Quality in Higher Education, 11(2), 137–149.
    • Marsh. H. W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence based perspective (pp. 319–383). Dordrecht, Netherlands: Springer.
    • McGill I. & Beaty L. (2001) What is action learning? In Action Learning (eds I. McGill & L. Beaty), pp. 11–28. Kogan Page, London.
    • Murphy, T., MacLaren, I., & Flynn, S. (2009). Toward a summative system for the assessment of teaching quality in higher education. International Journal of Teaching and Learning in Higher Education, 20(2), 226–236. Retrieved from http://www.isetl.org/ijtlhe/
    • Murray, H. G. (2005). Student evaluation of teaching: Has it made a difference? Paper presented at the Annual Meeting of the Society for Teaching and Learning in Higher Education, Charlottetown, Prince Edward Island.
    • Penny, A. R., & Coe, R. (2004). Effectiveness of consultation on student ratings feedback: Meta analysis. Review of Educational Research, 74, 215–253.
    • Richlin, L., & Cox, M. (2004). Developing scholarly teaching and scholarship of learning through faculty learning communities. New Directions for Teaching and Learning, 97, 127–135.
    • Roxa, T. & Matensson, K. (2009). Significant conversations and significant networks – exploring the backstage of the teaching arena. Studies in Higher Education, 34 (5), 547 559.
    • Sachs, S., Parsell, M., & Jacenyik Trawoger, C. (2012). Social, communicative and interpersonal leadership in the context of peer review. Sydney, Australia: Office of Learning and Teaching, Macquairie University.
    • Scott, G., Coates, H., & Anderson, M. (2008). Learning leaders in times of change: Academic leadership capabilities in times of change. Strawberry Hills, New South Wales: Australian Learning and Teaching Council & University of Western Sydney.
    • Seldin, P., Miller, J. E., & Seldin, C. A. (2010). The teaching portfolio: A practical guide to improved performance and promotion/tenure decisions. Hoboken, NJ: Wiley.
    • Shortland, S. (2010). Feedback within peer observation: Continuing professional development and unexpected consequences. Innovations in Education and Teaching International, 47(3), 295–304.
    • Simpson, D., & Anderson, M. B. (2010). Teaching as scholarship. In W. B. Jeffries & K. N. Huggett (Eds.), An introduction to medical teaching (pp. 187–198). New York, NY: Springer.
    • Spooren, P., Brockx, B., & Morelmans, D. (2013). On the validity of student evaluation: State of the art. Review of Educational Research, 83(4), 598–642. Retrieved from http://rer.sagepub.com/content/83/4/598
    • Toth, K. E., & McKey, C. A. (2010a). Differences in faculty development needs: Implications for educational peer review program design. Canadian Journal of Higher Education, 40(1), 53–68. Retrieved from http://ojs.library.ubc.ca/index.php/cjhe
    • Toth, K. E., & McKey, C.A. (2010b). Identifying the potential organizational impact of an educational peer review program. International Journal for Academic Development, 15(1), 73–83.
    • Ware, M. (2008). Peer review: Benefits, perceptions and alternatives. London, England: Publishing Research Consortium.