The use of skills inventories to assess and grade practice: Part 1 – Design and implementation
Introduction
The reliability and validity of tools for assessing skills and clinical competence have attracted discussion by academics and practitioners alike (Nicol and Freeth, 1998, Ashworth et al., 1999, Robb et al., 2002, Dolan, 2003, Gill et al., 2006). In the United Kingdom, the relevance of practice-based assessment is not in dispute since the professional regulatory body, the Nursing & Midwifery Council (NMC), stipulates students must attain proficiency at the required standard in the context of practice (NMC, 2004: 22). This is unchanged with the new standards published by the NMC in September 2010 (standard 8, requirement 8.1.5). The measurement of competence in practice, however, is challenging (Quinn and Hughes, 2007, Murrells et al., 2009, Gray and Donaldson, 2009). This can be even more problematic when the assessment strategy attempts to differentiate between academic levels such as diploma or degree level practice, or grade clinical performance.
Smith (2007) noted inflation of practice grades in midwifery degrees skewed the overall classification and called into question the reliability and validity of the assessment process. Pre-registration midwifery programmes in the United Kingdom are now all graduate programmes and required to grade practice. Practice grades achieved must contribute to the final academic award and classification (Nursing & Midwifery Council, 2007a, Nursing & Midwifery Council, 2007b, Nursing & Midwifery Council, 2009). The move to an all graduate nursing profession from September 2011 requires equal weighting in the assessment of theory and practice in contributing to the final award (NMC, 2010). This may require the creation of tools for grading practice at different academic levels comparable with midwifery pre-registration programmes.
Ashworth et al. (1999) reported in their study that differentiating between levels of practice was problematic for nurse educators. Assessment tools which attempted to distinguish between diploma and degree level practice were unreliable as the language used was not sufficiently precise. In addition, language was not commonly shared by nurse educators when specifying academic level. Watson et al. (2002) in their systematic review of the literature also questioned the reliability of the assessor highlighting the risk of subjectivity and socialisation. In particular, they identified the risk of bias if the assessor becomes familiar with the work of the student if assessed over time. Alternatively, if assessment is conducted as a snapshot observation, competence may be affected by ‘stage fright’, local circumstances and resource deficiencies (p. 424 Watson et al., 2002). Similar concerns have been highlighted by Hanley and Higgins, 2005a, Hanley and Higgins, 2005b in their literature review and qualitative study on students’ perceptions of a clinical competence assessment tool in the intensive care environment. A further challenge for nurse educators is deciding which skills to assess to establish competence; a problem emphasised by Bradshaw and Merriman (2008). Hanley and Higgins (2005b) found an assessment tool can lack context and culture. Skills were not sufficiently specific for the clinical speciality. More recently Murrells et al. (2009) have argued that if the NMC approach of assessing individual competences is adopted it may detract from more holistic skills.
Against the backdrop of this debate, this paper is the first of two recounting the creation and implementation of a tool to assess and grade nursing practice in critical care settings at post-registration degree level. It describes the process of designing skills with practice-based assessment criteria and a tool to grade the knowledge and performance assessed.
Section snippets
Skills inventories as an assessment strategy
Healthcare is changing on a global scale. It is more complex and resource-intensive to manage diverse provision. Living with long term conditions such as heart failure at one end of the spectrum and survival of the neonate at the other, results in the need for a broad range of specialist skills. Nursing is not exempt and the skills agenda is now a significant driver for providers of education in the United Kingdom. The modern nursing career is increasingly constructed around patient pathways
Design of grading tool
Having defined the knowledge and skills to be assessed the course team considered the design of a tool to grade clinical practice. One style of grading tool was required for use across all post-registration courses to ensure parity in degree classification. As clinical staff were to be the assessors, the grading tool also had to, be ‘user friendly’ (Pfeil, 2003), and incorporate clinical language. Hanley and Higgins (2005b) found assessment was impeded if students could not interpret language
Implementing the grading tool
Clinical staff responsible for assessing practice were familiar with the style of the skills inventories and were confident in awarding a pass or fail grade. The introduction of the grading tool made a new demand on assessors to examine practice beyond pass/fail.
Each student was allocated a work-based clinical mentor who held a mentor qualification. In the United Kingdom, a mentor is someone who has undertaken a mentor preparation programme approved by the regulatory professional body the
Discussion
Recurring criticisms within the literature relate to assessment instruments which need to be objective, reliable and valid (Coates and Chambers, 1992, Bradshaw, 2000, Pfeil, 2003, Bradshaw and Merriman, 2008, Murrells et al., 2009). Critics of the performance outcomes model of competence complain of a reductionist approach with an emphasis on behaviour change (Milligan, 1998, Watson, 2002, Scholes, 2006). Others argue that assessment of competence and skills could potentially improve fitness
Conclusion
The ascendancy of the skills agenda is a significant driver with the implementation of the NHS Knowledge and Skills Framework (DH, 2004b) to address competence in practice. Increasingly employers require evidence of competent practice and cost-effective education that is fit for purpose. The modern nursing career is now constructed around patient pathways and a competency-based work system (Skills for Health, 2006, Department of Health, 2008). The skills inventory offers a valid solution for
Acknowledgements
The authors wish to acknowledge the contribution of the critical care course leaders at the School of Nursing and Midwifery, and clinical staff in the local NHS Trusts who assisted in the design of the skills inventories and grading tool. Also Pam Pooley and Lorelei Blackburn who prepared the skills templates and grading tool.
References (49)
Grading student clinical practice performance: the Australian perspective
Nurse Education Today
(2000)- et al.
Problems of competence-based nurse education
Nurse Education Today
(1991) - et al.
Evaluation of tools to assess clinical competence
Nurse Education Today
(1992) - et al.
Evaluation of a clinical performance assessment tool (CPAT) within a critical care context
Australian Critical Care
(2006) - et al.
Assessment of clinical practice in intensive care: a review of the literature
Intensive and Critical Care Nursing
(2005) - et al.
Assessment of practice in intensive care: students’ perceptions of a clinical competence assessment tool
Intensive and Critical Care Nursing
(2005) Defining and assessing competence: the distraction of outcomes and the importance of educational process
Nurse Education Today
(1998)- et al.
Assessment of clinical skills: a new approach to an old problem
Nurse Education Today
(1998) - et al.
Measurement of clinical performance of nurses: a literature review
Nurse Education Today
(2002) Clinical competence: starship enterprise or straitjacket?
Nurse Education Today
(2002)
Editorial – technical skills training in the 21st Century
New England Journal of Medicine
‘Levels’ of attainment in nursing practice: reality or illusion?
Journal of Advanced Nursing
Competence and British nursing: a view from history
Journal of Clinical Nursing
Nursing competence 10 years on: fit for practice and purpose yet?
Journal of Clinical Nursing
Agenda for Change. Final Agreement
The NHS Knowledge and Skills Framework (NHS KSF) and the Development Review Process
Modernising Nursing Careers – Setting the Direction
Next Stage Review. A High Quality Workforce
Assessing student nurse clinical competency: will we ever get it right?
Journal of Clinical Nursing
Failing Students: A Qualitative Study of Factors that Influence the Decisions Regarding Assessment of Students’ Competence in Practice
The development of competency standards for specialist critical care nurses
Journal of Advanced Nursing
Concepts of competence
Journal of Interprofessional Care
The Bologna Process: Setting Up the European Higher Education Area
Work-based learning as a means of developing and assessing nursing competence
Journal of Clinical Nursing
Cited by (3)
Assessment methods and tools to evaluate postgraduate critical care nursing students' competence in clinical placement. An integrative review
2022, Nurse Education in PracticeCitation Excerpt :Nevertheless, students report not having enough time for reflection and supervision with their preceptors during clinical placement (Sundler et al., 2019). The OSCE approach has gained popularity in nursing education; however, it is time consuming and, at best, a simulation of practice (Hatfield and Lovegrove, 2012; Lovegrove and Hatfield, 2012). OSCEs should be recorded to increase transparency.
The use of skills inventories to assess and grade practice: Part 2 - Evaluation of assessment strategy
2012, Nurse Education in PracticeCitation Excerpt :Clinical mentors may also assess individual skills. Assessors are responsible for assessing individual skills (Lovegrove and Hatfield, 2011). Throughout this paper one term will be used to refer to both clinical mentors and assessors, that is, assessors.
Implementation of mock competency skill assessments to improve student outcomes
2015, Nurse Educator