Systematic Plan of Evaluation Part II: Assessment of Program Outcomes

https://doi.org/10.1016/j.teln.2017.11.001Get rights and content

Abstract

As an accrediting agency recognized by the U.S. Department of Education (USDE) and the Council for Higher Education Accreditation (CHEA), the Accreditation Commission for Education in Nursing (ACEN) has established Accreditation Standards and Criteria for the evaluation of nursing programs, including the evaluation of outcomes. This article focuses on the essential components and processes for systematic evaluation of program outcomes, including licensure examination pass rate, program completion rate, and job placement rate.

Introduction

This article is the second in a two-part series discussing the assessment of nursing program outcomes. The assessment of outcomes for a nursing program is a major component of the 2017 ACEN Standards and Criteria, specifically Standard 6 Outcomes. A program's compliance with Standard 6 Outcomes is an essential measure of the program's quality. These outcomes demonstrate to communities of interest that the program provides quality education preparing the graduates to work in the profession of nursing. Faculty members of quality nursing programs participate in regular assessment of these outcomes to identify areas for program improvement. For ACEN-accredited programs or those seeking ACEN accreditation, this assessment requires the development and utilization of a written systematic plan of evaluation that includes the assessment details of each end-of-program student learning outcome as well as each program outcome. Part I of this article focused on the assessment of the end-of-program student learning outcomes, which are the knowledge, skills, or behaviors that the students should be able to demonstrate upon completion of the nursing program. The focus of Part II is the assessment of the program outcomes, including licensure examination pass rate, program completion rate, and job placement rate.

For a program to demonstrate compliance with Standard 6 Outcomes, the faculty must include the aforementioned program outcomes in the systematic plan of evaluation. As a summary of the assessment of outcomes, the faculty should select appropriate assessment methods and then establish specific, measurable expected levels of achievement for each program outcome that are consistent with the assessment method and will result in meaningful data about the outcome. Faculty must also determine the frequency with which they collect, trend, and analyze these data to inform program decision-making. The plan should include documentation of this analysis and the decisions made based on the analysis of data.

This procedure for program assessment should sound familiar to nursing faculty, since it follows the nursing process. In many regards, the systematic plan of evaluation is simply a nursing care plan, but in this case, the “patient” is the nursing program. Just as a “patient” has better outcomes when the care plan is consistently utilized, the faculty can take better care of their nursing program by using a systematic plan of evaluation. The collection and analysis of data documented in a program's systematic plan of evaluation will assist faculty in making decisions about targeted strategies for improvement, when needed. Equally important in this process is documenting these actions and evaluations in the systematic plan. The Criteria for Standard 6 provide guidelines for the evidence of this faculty-driven process, and the written systematic plan of evaluation provides a record of these efforts. Finally, just as a nursing care plan should involve ongoing evaluation and revision as needed, the systematic plan of evaluation should be viewed as a document that demonstrates continual assessment and evaluation of outcomes with revisions as needed to ensure that the program's evaluation process is as effective and current as possible, and that student success is facilitated by the faculty as well as the strategies implemented.

For nursing programs pursuing initial accreditation with the ACEN, including those that may not yet have graduates, the faculty must still develop a systematic plan of evaluation related to the end-of-program student learning outcomes and the program outcomes during the Candidacy process. Newly established programs in Candidacy must begin collecting outcomes data from the time in which Candidacy is achieved. By developing the plan in advance, the faculty are able to begin collecting and analyzing the data with the initial cohort. For nursing programs that have been in existence and are pursuing Candidacy, several years of outcomes data are often available to include in the plan, even though the data available may be incomplete as required by Standard 6.

Section snippets

Graduates' Achievement on the Licensure Examination (Criterion 6.2)

The first required program outcome included in the systematic plan of evaluation is licensure examination pass rate. “Licensure is the process by which a governmental agency gives affirmation to the public that the individuals engaged in an occupation or profession have minimal education, qualifications, and competence necessary to practice in a safe manner” (ACEN Accreditation Manual, Glossary, p. 3). One of the primary goals of pre-licensure nursing programs throughout the United States is to

Graduates' Achievement Related to Program Completion (Criterion 6.3)

The program completion rate is the second program outcome included in the systematic plan of evaluation. The program completion rate is the percentage of students who graduate within a defined period of time; sometimes, these are also referred to as graduation rates. The definition by the ACEN for a nursing program completion rate is “the number of students who complete the program in no more than 150% of the stated nursing program length, beginning with enrollment in the first nursing course” (

Graduates' Achievement in Job Placement (Criterion 6.4)

The job placement rate is the third program outcome required in the systematic plan of evaluation. Similar to licensure examination pass rates and program completion rates, job placement rates can reflect program quality. The intent of Criterion 6.4 is for the program to demonstrate the extent to which students are successful in achieving employment as a registered nurse. The ACEN Glossary defines job placement as the “Percentage of graduates employed in a position for which a nursing program

Additional Program Outcomes

It is important to note that although the 2017 ACEN Standards and Criteria no longer require Standards 1–5 in the systematic plan of evaluation document, the faculty may elect to retain certain portions of the plan for internal use only; the program does not present to peer evaluators these portions of the plan. Additionally, faculty must consider requirements of other agencies for program evaluation. Some colleges and universities may have ongoing program assessment requirements, such as

Communication Regarding Program Outcomes

Finally, in accordance with ACEN Policy #29 Advertising and Recruitment of Students, programs are required to make student achievement data available to the public. The student achievement data are the required program outcomes from the systematic plan of evaluation and should include the graduates' success on the licensure examination, the students' completion of the nursing program, and graduates obtaining a job in the field in which they have been prepared. Though programs are only required

Conclusion

For some faculty simply hearing the words “systematic plan of evaluation” increases their anxiety level. The intent of this article, as well as Part I, was to demystify the process of developing and having a plan of evaluation that is utilized consistently by the faculty to systematically review a nursing program and its outcomes. The three required program outcomes: licensure examination pass rate, program completion rate, and job placement rate are indicators of program quality and must be

References (4)

  • Accreditation Commission for Education in Nursing

    ACEN 2017 standards and criteria-associate

    (2016)
  • Accreditation Commission for Education in Nursing

    Accreditation manual/glossary

    (2016)
There are more references available in the full text version of this article.

Cited by (4)

  • Data analytics in nursing education: Trended Tracking matters for theory, research, and practice

    2021, Teaching and Learning in Nursing
    Citation Excerpt :

    Fig. 2 illustrates the progression pathway). Although accrediting agencies may not mandate disaggregation by pathways or measurement of graduate articulation (enrollment in advanced degree or specialty programs), supplemental and disaggregated data add richness to interpretation, decision-making, and actions, and may be included in reports to accrediting agencies (Nunn-Ellison et al., 2018). Furthermore, recruitment, retention, and articulation of diverse students to more closely mirror populations served and close the gap in health and health care disparities is a priority yet nursing education and workforce inequities and disparities exist; attrition is higher for under-represented groups such as African Americans, Latinx, first generation college students, sexual and gender minorities, and non-native English speakers (Alexander & Johnson, 2021; American Association of Colleges of Nursing, 2017; Aragon et al., 2020; Fontenot & McMurray, 2020; Green, 2020; Jeffreys, 2012b; Jeffreys, 2020a; Kruse et al., 2020; National League for Nursing, 2016; Patterson & Mikovits, 2021).

  • Using Data for Program Improvement: The Role of the ACEN Report to Constituents

    2020, Teaching and Learning in Nursing
    Citation Excerpt :

    This support is to ensure the most accurate information is provided in the Annual Report and ultimately aggregated into the Report to Constituents for use by ACEN-accredited programs. If faculty are unsure if the data are being calculated correctly for their ACEN peers, they are encouraged to review the ACEN Glossary located on the ACEN website for the definition of program completion and job placement or review the previous article written by the ACEN Directors regarding program outcomes (Ellison-Nunn, Ard, Beasley, & Farmer, 2017). The ACEN Directors are also available for assistance if needed by calling or emailing the ACEN office.

  • Next Generation NCLEX®: Using the ACEN Standards and Criteria to Prepare

    2020, Teaching and Learning in Nursing
    Citation Excerpt :

    Then, the systematic plan of evaluation should demonstrate these conversations and analysis of the data preimplementation/postimplementation. For more information on assessing the effectiveness of the systematic plan of evaluation, faculty are encouraged to review the previous article related to the assessment of program outcomes (Ellison-Nunn et al., 2017). Although any change to the NCLEX® has a tendency to engender anxiety, nursing faculty are essential in preparing graduates for success.

  • Fundamentals of terminology related to Accreditation Commission for Education in Nursing

    2018, Teaching and Learning in Nursing
    Citation Excerpt :

    However, the ACEN-required program outcomes are licensure examination pass rates, program completion rates, and job placement rates. See the previous articles related to the assessment of end-of-program student learning outcomes and program outcomes (Beasley, Farmer, Ard, & Nunn-Ellison, 2017; Nunn-Ellison, Ard, Beasley, & Farmer, 2017). Hopefully, the responses to these questions from your peers have provided the reader with a better understanding of the terminology utilized by the ACEN relative to the Standards and Criteria.

Acknowledgment of financial support: None.

Presentation Information: No previous presentation.

View full text