Article
Human Factors Evaluation of the Universal Anaesthesia Machine: Assessing Equipment with High-Fidelity Simulation Prior to Deployment in a Resource-Constrained Environment

https://doi.org/10.1016/j.jnma.2019.03.005Get rights and content

Abstract

Background

Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM®), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM® to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM® familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone.

The goals of this study were: 1. to assess the usability of the UAM® (machine-user interface, simulated patient use, symbology, etc.) across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM® in the intended austere clinical environments.

Methods

Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine® at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM® use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm) and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA) to control for participant demographics. Usability debriefings were analyzed qualitatively.

Results

Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM®. Mean participant ratings were high for ease of use (5.4 ± 0.96) and clarity of instruction (6.2 ± 0.87) on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95% CI, 0.33–34.7) or power failure scenarios (P = 0.012; 95% CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM®.

Conclusions

The UAM® design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.

Introduction

Approximately 5 billion people worldwide lack access to safe and affordable surgical and anesthesia care. In 2010, an estimated 16.9 million deaths (32.9% of all deaths worldwide) were due to conditions that required surgical care.1 In high-income countries (HICs), perioperative and anesthetic-related mortality rates have significantly decreased despite the need to care for ever-sicker patients. Patients continue to have a high risk of anesthesia-related mortality in low- and middle-income countries (LMICs).2 Consequently, improving surgical outcomes in LMICs requires improvements to perioperative anesthesia care.3 The anesthesia machine, is an essential life support device that is particularly vulnerable to dysfunction in resource-constrained environments.

Global health disparities exist in anesthesia care, which limit access to quality surgical care in many LMICs. Examples include Liberia, which has zero anesthesia physicians and 60 nurse anesthetists, as well as Sierra Leone with 2 anesthesia physicians nationally (personal communication Ministry of Health of Liberia and Ministry of Health and Sanitation of Sierra Leone). Anesthesia care providers in LMICs face many challenges, including insufficient numbers of trained personnel, inadequate supply of essential medications, unreliable infrastructure, and equipment scarcity. In a recent study of 590 facilities in 22 LMICs, 35% of facilities lacked access to oxygen and 40% lacked functioning anesthesia machines.4 Often, anesthesia equipment donated or purchased for LMIC hospitals is not suited for the environment and will not function properly (if at all). Estimates indicate that 70%–90% of devices designed for industrialized countries will not function in LMICs.5 Factors that make an anesthesia machine unsuited to the local environment include machines that require compressed oxygen sources but are deployed in clinical locations without consistent availability; machines that are vulnerable to failure in the presence of an unstable or inadequate power grid but are placed where power failures are frequent; and machines placed where there is a mismatch between the biomedical maintenance support requirements and availability of parts and/or adequately-trained personnel. In these challenging locations, an anesthesia machine that is optimized for the low-resource environment is needed. Other challenging in-situ factors include environments with typically high heat and humidity as well as often mismatches between consumable requirements and consumable availability.

There have been repeated calls for device manufactures to develop technology that is adaptable enough to reach populations in Sub-Saharan Africa. The World Federation of Societies of Anaesthesiologists (WFSA) is participating in the development of an ISO standard (8835-7:2011) for anesthetic delivery systems in areas with limited access to anesthetic gases and inconsistent electrical power, and the WFSA Safety and Quality of Practice Committee has provided guidance to support anesthetic machines in different regions worldwide.6 Some equipment manufacturers have designed systems for these environments such as the Glostavent®, a low-resource optimized anesthesia machine. However, the need for machines which are reliably functional in these locations remains great. Currently, an overarching method for designing new equipment to support anesthesia care delivery in these settings is largely absent.

The Universal Anaesthesia Machine® (Gradian Health Systems, Inc.) is a low-resource optimized anesthesia machine specifically designed to be used in the austere clinical environment. Use of this machine in a controlled environment was examined to explore machine-user issues and to ascertain ease of use prior to the development of a curriculum for use and deployment on a large scale in a low-resource environment. We postulated that a simulation-based human factors evaluation could provide a safe, efficient, low-cost and complementary approach to traditional methods of testing equipment and devices intended for deployment to vulnerable populations in low-resource settings. While medical simulation cannot fully replicate the low-resource environment (e.g. high heat and humidity), targeted challenges within the environment (e.g. user issues, symbology, visual/aural cues during use) can be replicated so that the device performance can be assessed.

In this study, we report the results of a simulation-based human factors evaluation of the Universal Anaesthesia Machine (UAM®; Gradian Health Systems Inc. New York, NY). Study objectives were twofold. First, we assessed the usability of the UAM® across provider groups with differing levels of experience under normal and challenging clinical and environmental conditions. Second, feedback was gathered on available UAM training material and usability issues in order to maximally prepare Sierra Leone-based anesthesia providers (locally trained nurse anesthetists) still in-training on how to use the UAM®. Since the curriculum that we plan to develop is simulation based, information like that displayed in Appendix table C was useful in understanding what features of training require greater emphasis. We discuss implications of study results for a broader approach to meeting the equipment needs of care providers in LMIC settings.

Section snippets

Materials and methods

This study was approved by the Institutional Review Board of the Johns Hopkins University School of Medicine (JHSOM). Data collection occurred between September 2012 and July 2013 at the JHSOM Simulation Center. Concurrent with this study, the research team was conducting an observational study of anesthesia delivery in two tertiary-care hospitals in Freetown, Sierra Leone.7 Simulation scenarios were designed to reflect challenges found in this austere environment.6, 8, 9

Results

Table 1 summarizes demographics for the 35 anesthesia providers who participated in the study. Ages ranged from 25 to 59 years. Fourteen participants (40%) had international anesthesia mission trip experience prior to the study. Eight (22.9%) had previous experience administering halothane. Only two (5.71%) had previous clinical experience with the administration of draw-over inhalational anesthesia. All participants had clinical experience with multiple anesthesia delivery systems, and 22

Discussion

In this study, we evaluated the usability of the UAM, workload levels with scenarios and feedback for the development of training material by using high fidelity medical simulation as a method for human factors device evaluation. We also used our advanced simulation equipment to confirm that this unique device was safe to use on a vulnerable population (economically disadvantaged), prior to introducing it to the two most critical institutions in Sierra Leone. With this approach, we examined a

Conclusion

Though this study uncovered potential opportunities to improve UAM® usability, our findings suggest that the device is well designed for overcoming challenges in anesthesia functionality that can cause even the most high-end anesthesia machines to fail in the resource-constrained environment. The study also showed that the UAM® has a high ease of use and good clarity of instruction. The ability of the UAM® to function well during power failures and oxygen shortages allowed Western users,

Author's contributions

John Sampson helped to design the study and write the manuscript and provided critical review.

Benjamin Lee helped to design the study and write the manuscript and provided critical review.

Rahul Koka helped to design the study, analyze and interpret the data, and write the manuscript. He attests to the integrity of the original data and the analysis.

Adaora Chima helped to design the study, collect the data, analyze and interpret the data, and write the manuscript. She attests to the integrity of

Conflict of interest

All authors discloses that the research described in this manuscript was supported by an unrestricted grant from Gradian Health Inc. and that the curriculum described within the manuscript was later used for international training projects performed by the co-authors.

Attestation: All authors approved the final manuscript.

Funding

Funding was provided via an unrestricted grant from Gradian Health Systems, Inc.

Acknowledgements

This work was supported through a grant from Gradian Health Systems, Inc. We would also like to thank the support team from the Johns Hopkins ACCM Global Alliance of Perioperative Professionals (GAPP).

References (15)

There are more references available in the full text version of this article.

IRB: Research was approved by the Johns Hopkins University Institutional Review Board. The IRB can be contacted at [email protected] or via phone at 410-955-3008.

View full text