Abstract
This paper presents a case study pertaining to human factors/human–machine interaction (HMI) research undertaken over a four-year period, as part of a large Framework Seven project funded by the European Commission. Specifically, it reports on research concerning the operational validation of future cockpit concepts supporting all conditions operations. Primarily, it focuses on the innovation in the methodological approach adopted (and the theory underpinning this) and the key results. In so doing, this paper illustrates the requirement for HMI research to go beyond issues of task and user interface design, so as to properly address issues of operability. It is argued that the typical remit of operational assessment (i.e. the assessment of task workflows, workload and operator situation awareness) must be supplemented by a broader analysis of ‘operational’ issues related to the design of operational processes/procedures, teamwork, system information flow, process benefits and implementation barriers. This is enabled by the adoption of a stakeholder/participatory approach to evaluation and the application of both formal and informal HMI design/evaluation methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
All Condition Operations and Innovative Cockpit Infrastructure (ALICIA) Project Consortium (2009) Annex 1: Description of Work. Project Number 233682. Funded by European Commission—Seventh Framework Programme—Theme 7: Transport (Including Aeronautics)
Abbott T (1993) Functional categories for future flight deck designs (NASA Technical Memorandum TM-109005), NASA Langley Research Center, Hampton.
Abbott TS, Rogers WH (1993) Functional categories for human-centered flight deck design. In: Proceedings of the 12th Digital Avionics Systems Conference, AIAA/IEEE, New York
Bannon L (1991) From human factors to human actors: the role of psychology and human-computer interaction studies in systems design. In: Greenbaum J, Kyng M (eds) Design at work: cooperative design of computer systems. Lawrence Erlbaum Associates, Hillsdale, NJ, pp 25–44
Bannon L, Bødker S (1991) Beyond the interface: encountering artifacts in use. In: Carroll JM (ed) Designing interaction: psychology at the human-computer interface. Cambridge University Press, New York, pp 227–253
Baranzini D (2009) Team concepts in maintenance. Unpublished doctoral dissertation, Trinity College Dublin, Dublin
Bødker S, Burr J (2002) The design collaboratorium—a place for usability design. ACM Trans Comput Human InteracT 9(2):152–169
Bødker S, Grønbæk K (1996) Users and designers in mutual activity: an analysis of cooperative activities in systems design. In: Engeström Y, Middleton D (eds) Cognition and Communication at Work. Cambridge University Press, Cambridge, pp 130–158
Cahill J (2008) Envisioning improved work practices and associated technology requirements in the context of the broader socio-technical system. In: Human computer interaction (Open Access Book, In Press), I-tech Publishing, Vienna
Cahill J (2010) Flight crew task performance and the requirements for a new tool supporting the pre-flight, flight planning and briefing task. Unpublished doctoral dissertation, Trinity College Dublin, Dublin
Cahill J, McDonald N, Losa G (2008) Enhancing human performance and overall flight safety through improvements in airline system information flow. In: 28th Conference of the European Association for Aviation Psychology, Valencia, Spain, October 27–31, 2008
Cahill J, McDonald N, Losa G (2011) Intelligent planning and the design of a new risk based, intelligent flight plan. Cognit Techno Work 13(1):43–66
Cahill J, McDonald N, Losa G (2014) A socio-technical model of the flight crew task, human factors. J Human Factors Ergon Soc 1–27. doi:10.1177/0018720814532684
Carroll JM (1995) The scenario perspective on system development. In: Carroll M (ed) Scenario-based design: envisioning work and technology in system development. Wiley, New York, pp 1–17
Carroll JM (2000) Making use: scenario-based design of human–computer interactions. MIT Press, Cambridge, MA
Constantine LL, Lockwood LAD (1999) Software for use: a practical guide to the models and methods of usage-centered design. Addison-Wesley, Cambridge
Cooper A (2007) About face 3.0: the essentials of interaction design. Wiley, New York
Cousins JB, Earl LM (1992) The case for participatory evaluation. Educ Eval Policy Anal 14:397–418
Cousins JB, Donohue JJ, Bloom GA (1996) Collaborative evaluation in North America: evaluators’ self-reported opinions, practices and consequences. Eval Pract 17:207–226
Cullen A, Coryn E, Rug J (2011) The politics and consequences of including stakeholders in international development evaluation. Am J Eval 32(3):345–361
Derrison J (2007) The SESAR concept of operations. http://www.airtn.eu/documents/AirTN/AirTNForum1-6-Derisson.pdf. Retrieved 15 May 2010
Edwards E (1972) Man and machine: systems for safety. Proceedings of British airline pilots associations technical symposium. British Airline Pilots Association, London, pp 21–36
Eurocontrol (2009) Single European Sky. http://www.eurocontrol.int/ses/public/standard_page/sk_ses.html. Retrieved 10 June 2009
Falzon P (2008) Enabling safety: issues in design and continuous design. Cogn Technol Work 10(1):7–14
Ferreira A, Bonelli S, Dokic J, Napoletano L (2014) A HMI design approach for the future flight deck, presented at the Human Factors and Ergonomics Society Europe Chapter 2014 annual conference, Lisbon 8th–10th October 2014
Greene JC (2005) The generative potential of mixed methods inquiry 1. Int J Res Method Educ 28(2):207–211
Hackos JA, Redish JC (1998) User and task analysis for interface design. Wiley Computer Publishing, New York
Hawkins FH (1987) Human factors in flight. Gower Technical Press, Aldershot
Hutchins E (1995a) Cognition in the wild. MIT Press, Cambridge
Hutchins E (1995b) How a cockpit remembers its speed. Cogn Sci 19:265–288
Johnson WW, Battiste V, Bochow SH (1999) A cockpit display designed to enable limited flight deck separation responsibility. In: Proceedings of the SAE/AIAA 1999 world aviation conference. Anaheim
Library of Michigan (2014) Stakeholder evaluation handbook. http://www.michigan.gov/libraryofmichigan/0,2351,7-160-18668_54901_18688-59315–00.html
Mark MM, Shotland RL (1985) Stakeholder-based evaluation and value judgments. Eval Rev 9(5):605–626
McDonald N (2006) Modelling the human role in operational systems. In: Proceedings of the 17th European safety and reliability conference on safety and reliability for managing risk. Taylor and Francis, Estoril
McDonald N, Morrison R (2006) Modelling the human role in operational systems–theory and practice. In: 2nd resilience engineering symposium, Antibes France, 8–10 November 2006
Muller M (1991) PICTIVE—An exploration in participatory design. Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology. Association for Computer Machinery, New Orleans, pp 225–231
Muller M (2003) Participatory design: the third space in HCI. In: Jacko J, Sears A (eds) The human computer interaction handbook: evolving technologies and emerging applications. Lawrence Erlbaum Associates, Mahwah, pp 1051–1069
Muller M, Kuhn S (1993) Special issue on participatory design. Commun ACM 36(4):24–28
Preece J, Rogers Y, Sharp H (2002) Interaction design: beyond human–computer interaction. Wiley, New York
Reason P, Bradbury H (2008) The Sage handbook of action research: participative inquiry and practice. Sage, CA
Rozzi S, Bonelli S, Ferreira A, Napoletano L, Bécouarn L (2014) The operational potential of an in-flight weather awareness system: an explorative pilot-in-the-loop simulation. In: Proceedings of the human factors and ergonomics society Europe Chapter 2014 Annual Conference, ISSN 2333-4959 (online). Available from http://hfes-europe.org, Lisbon 8th–10th October 2014
Ricks WR, Jonsson JE, Rogers WH (1993) Cognitive representations of flight-deck information attributes. In: Proceedings of the 7th International Symposium on Aviation Psychology, vol I, The Ohio state University, Columbus
Schmidt K, Bannon L (2013) Constructing CSCW: The first quarter century. Computer Supported Cooperative Work (CSCW) 22(4–6):345–372
SESAR Consortium (2008a) SESAR definition phase. The Concept of Operations at a Glance, Brussels
SESAR Consortium (2008b) Definition Phase WP2.3 performance assessment—document reference DLT-0706-231-00-05, Brussels
Smith MJ, Carayon-Sainfort P (1989) A balance theory of job design for stress reduction. Int J Ind Ergon 4:67–79
Vicente KJ (1999) Cognitive work analysis: toward safe, productive, and healthy computer-based work. Lawrence Erlbaum Associates, Mahwah
Ward M (2005) Contributions to human factors from three case studies in aviation maintenance. Unpublished doctoral dissertation, Trinity College Dublin, Dublin
Weaver L, Cousins JB (2004) Unpacking the participatory process. J MultiDiscip Eval 1:19–40
Weiss CH (1986) The stakeholder approach to evaluation: Origins and promise. In: House ER (ed) New directions for educational evaluation. RoutledgeFalmer, Abingdon, pp 145–157
Acknowledgments
The authors would like to thank the European Commission for sponsoring this research. Further, we would like to thank the ALICIA Project Consortium for their support and members of the External Expert Advisory Group (EEAG). Also, we would like to thank August Westland, Mike Doiran and John Dutcher for their help with this research.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Breakdown of research phases and methods
Table 4 provides a breakdown of the specific HMI design/evaluation activities comprising the four research phases. Please note that this includes information about how these research activities relate to the broader project research structure (i.e. phases 1, 2 and 3 as outlined earlier), and the associated project timeframe (i.e. years 1–4).
Appendix 2: Phase 1 research: summary of process and decision points
See Table 5.
Appendix 3: Phase 1 research: task analysis template: decision to taxi out
See Table 6.
Appendix 4: Weather scenario
See Table 7.
Appendix 5: Phase 2 research: scenario framework: worked examples
Select Point in Process
-
Approach and landing
Select Decision Point
-
Decision to land
Select Weather Condition
-
Low visibility, fog (medium level)
Is this a credible situation (e.g. configuration of process resources and weather condition)?
Given this situation, what is the most credible delay (e.g. outcome) at landing?
Given this delay at landing (e.g. outcome), what is the most credible delay for the airport operation (e.g. consequences for other/multiple flights)?
How probable/likely is this (e.g. frequency of outcome)?
Appendix 6: Phase 2 research: key operational evaluation questions
See Table 8.
Rights and permissions
About this article
Cite this article
Cahill, J., McDonald, N., Morrison, R. et al. The operational validation of new cockpit technologies supporting all conditions operations: a case study. Cogn Tech Work 18, 479–509 (2016). https://doi.org/10.1007/s10111-016-0380-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-016-0380-4