Skip to main content
SpringerLink
Log in

Exploring the human factors in moral dilemmas of autonomous vehicles

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Given the widespread popularity of autonomous vehicles (AVs), researchers have been exploring the ethical implications of AVs. Researchers believe that empirical experiments can provide insights into human characterization of ethically sound machine behaviour. Previous research indicates that humans generally endorse utilitarian AVs; however, this paper explores an alternative account of the discourse of ethical decision-making in AVs. We refrain from favouring consequentialism or non-consequential ethical theories and argue that human moral decision-making is pragmatic, or in other words, ethically and rationally bounded, especially in the context of intelligent environments. We hold the perspective that our moral preferences shift based on various externalities and biases. To further this concept, we conduct three Amazon Mechanical Turk studies, comprising 479 respondents to investigate factors, such as the “degree of harm,” “level of affection,” and “fixing the responsibility” that influences people’s moral decision-making. Our experimental findings seem to suggest that human moral judgments cannot be wholly deontological or utilitarian and offer evidence on the ethical variations in human decision-making processes that favours a specific moral framework. The findings also offer valuable insights for policymakers to explore the overall public perception of the ethical implications of AV as part of user decision-making in intelligent environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bagloee SA, Tavana M, Asadi M, Oliver T (2016) Autonomous vehicles: challenges, opportunities, and future implications for transportation policies. J Mod Transp 24(4):284–303

    Article  Google Scholar 

  2. Pan S, Thornton SM, Gerdes JC (2016) Prescriptive and proscriptive moral regulation for autonomous vehicles in approach and avoidance. 2016 IEEE International Symposium on Ethics in Engineering, Science and Technology (ETHICS) 1–6

  3. Thornton SM, Pan S, Erlien SM, Gerdes JC (2016) Incorporating ethical considerations into automated vehicle control. IEEE Trans Intell Transp Syst 18(6):1429–2143

    Article  Google Scholar 

  4. Karnouskos S (2018) Self-driving car acceptance and the role of ethics. IEEE Trans Eng Manag

  5. Faulhaber AK, Dittmer A, Blind F, Wächter MA, Timm S, Sütfeld LR, Stephan A, Pipa G, König P (2019) Human decisions in moral dilemmas are largely described by utilitarianism: virtual car driving study provides guidelines for autonomous driving vehicles. Sci Eng Ethics 25(2):399–418

    Article  Google Scholar 

  6. Foot P (1983) Moral realism and moral dilemma. J Philos 80(7):379–398

    Article  Google Scholar 

  7. Kälvemark S, Höglund AT, Hansson MG, Westerholm P, Arnetz B (2004) Living with conflicts-ethical dilemmas and moral distress in the health care system. Soc Sci Med 58(6):1075–1084

    Article  Google Scholar 

  8. Cohen TR, Panter AT, Turan N, Morse L, Kim Y (2014) Moral character in the workplace. J Pers Soc Psychol 107(5):943

    Article  Google Scholar 

  9. Hamelink CJ (2000) The ethics of cyberspace. Sage

  10. Bostrom N, Yudkowsky E (2014) The ethics of artificial intelligence. The Cambridge Handbook of Artificial Intelligence 1:316–334

  11. Bonnefon J-F, Shariff A, Rahwan I (2016) The social dilemma of autonomous vehicles. Science 352(6293):1573–1576

    Article  Google Scholar 

  12. Gendler TS (2014) Thought experiment: On the powers and limits of imaginary cases. Routledge

  13. Goodall NJ (2017) From trolleys to risk: Models for ethical autonomous driving. Am Public Health Assoc

  14. Thomson JJ (1985) The trolley problem. Yale Law J 94(6):1395–1415

    Article  Google Scholar 

  15. West HR (2004) An introduction to Mill's utilitarian ethics. Cambridge University Press

  16. Alexander L, Moore M (2008) Deontological ethics. In Zalta, E. N. (Ed.), The Stanford encyclopedia of philosophy

  17. Huang K, Greene JD, Bazerman M (2019) Veil-of-ignorance reasoning favors the greater good. Proc Natl Acad Sci 116(48):23989–23995

    Article  MATH  Google Scholar 

  18. Frank D-A, Chrysochou P, Mitkidis P, Ariely D (2019) Human decision-making biases in the moral dilemmas of autonomous vehicles. Sci Rep 9(1):1–19

    Article  Google Scholar 

  19. Byrd N, Conway P (2019) Not all who ponder count costs: arithmetic reflection predicts utilitarian tendencies, but logical reflection predicts both deontological and utilitarian tendencies. Cognition 1–82

  20. Gigerenzer G (2010) Moral satisficing: rethinking moral behavior as bounded rationality. Top Cogn Sci 2(3):528–554

    Article  Google Scholar 

  21. Panagiotopoulos I, Dimitrakopoulos G (2018) An empirical investigation on consumers’ intentions towards autonomous driving. Transp Res C Emerg Technol 95:773–784

    Article  Google Scholar 

  22. Shah MU, Rehman U, Iqbal F, Hussain M, Wahid F (2021) An alternate account on the ethical implications of autonomous vehicles. 17th International Conference on Intelligent Environments (IE). IEEE, pp 1–5 

  23. Zoshak J, Dew K (2021) Beyond Kant and Bentham: how ethical theories are being used in artificial moral agents. In CHI Conference on Human Factors in Computer Systems, Yokohama, Japan 1–15

  24. Bergmann LT, Schlicht L, Meixner C, Konig P, Pipa G, Boshammer S, Stephan A (2018) Autonomous vehicles require socio-political acceptance — an empirical and philosophical perspective on the problem of moral decision making. Front Behav Neurosci 12(31):1–12

    Google Scholar 

  25. Geisslinger M, Poszler F, Betz J, Lütge C, Leinkamp M (2021) Autonomous driving ethics: from trolley problem to ethics of risk. Philos Technol 1–23

  26. Martin R, Kusev P, Teal J, Baranova V, Rigal B (2021) Moral decision making: from bentham to veil of ignorance via perspective taking accessibility. Behav Sci 11(66):1–14

    Google Scholar 

  27. Goldstein-Greenwood J, Conway P, Summerville A, Johnson BN (2020) (How) do you regret killing one to save five? Affective and cognitive regret differ after utilitarian and deontological decisions. Personal Soc Psychol Bull 46(9):1303–1317

    Article  Google Scholar 

  28. Conway P, Goldstein-Greenwood J, Polacek D, Green JD (2018) Sacrificial utilitarian judgements do not reflect concern for the greater good: clarification via process dissociation and the judgements of philosophers. Cognition 179:241–265

    Article  Google Scholar 

  29. Mirnig AG, Meschtscherjakov A (2019) Trolled by the trolley problem: on what matters for ethical decision making in automated vehicles. In CHI Conference on Human Factors in Computing Systems Proceedings, Glasgow, Scotland U.K 1–10

  30. Greene JD, Morelli SA, Lowenberg K, Nystrom LE, Cohen JD (2008) Cognitive load selectively interferes with utilitarian moral judgement. Cognition 107:1144–1154

    Article  Google Scholar 

  31. Borowsky A, Oron-Gilad T (2013) Exploring the effects of driving experience on hazard awareness and risk perception via real-time hazard identification, hazard classification, and rating tasks. Accid Anal Prev 59:548–565

    Article  Google Scholar 

  32. Cao F, Zhang J, Song L, Wang S, Miao D, Peng J (2017) Framing effect in the trolley problem and footbridge dilemma: number of saved lives matters. Psychol Rep 120(1):88–101

    Article  Google Scholar 

  33. Cochran WG (1950) The comparison of percentages in matched samples. Biometrika 37(3/4):256–266

    Article  MathSciNet  MATH  Google Scholar 

  34. Tate MW, Brown SM (1970) Note on the Cochran Q test. J Am Stat Assoc 65(329):155–160

    Article  Google Scholar 

  35. Christen M, Narvaez D, Zenk JD, Villano M, Crowell CR, Moore DR (2021) Trolley dilemma in the sky: context matters when civilians and cadets make remotely piloted aircraft decisions. PLoS One 16(3):247–273

    Article  Google Scholar 

  36. Lee J, Holyoak KJ (2020) “But he’s my brother”: the impact of family obligation on moral judgments and decisions. Mem Cognit 48(1):158–170

    Article  Google Scholar 

  37. Bleske-Rechek A, Nelson LA, Baker JP, Remiker MW, Brandt SJ (2010) Evolution and the trolley problem: people save five over one unless the one is young, genetically related, or a romantic partner. J Soc Evol Cult Psychol 4(3):115

    Article  Google Scholar 

  38. Sütfeld LR, Gast R, König P, Pipa G (2017) Using virtual reality to assess ethical decisions in road traffic scenarios: Applicability of value-of-life-based models and influences of time pressure. Front Behav Neurosci 11:122

    Article  Google Scholar 

  39. Tassy S, Oullier O, Mancini J, Wicker B (2013) Discrepancies between judgment and choice of action in moral dilemmas. Front Psychol 4:250

    Article  Google Scholar 

  40. Hevelke A, Nida-Rümelin J (2015) Responsibility for crashes of autonomous vehicles: an ethical analysis. Sci Eng Ethics 21:619–630

    Article  Google Scholar 

  41. Martinho A, Herber N, Kroesen M, Chorus C (2021) Ethical issues in focus by the autonomous vehicles industry. Transp Rev 1–22

  42. Gurney JK (2013) Sue my car not me: products liability and accidents involving autonomous vehicles. J Law Technol Policy 2013(2):247–277

    Google Scholar 

  43. Chugh D, Kern MC (2016) A dynamic and cyclical model of bounded ethicality. Res Organ Behav 36:85–100

    Google Scholar 

  44. McDuffie JP (2018) The policy trajectories of autonomous vehicles. Penn Wharton Public Policy Initiative 6(4):1–8

    Google Scholar 

  45. Waldrop M (2015) Autonomous vehicles: no drivers required. Nature 518(7537):20–23

    Article  Google Scholar 

Download references

Funding

This study is supported by the Research Incentive Fund (R20090) at the Zayed University, United Arab Emirates.

Author information

Authors and Affiliations

  1. Department of Management Sciences, Faculty of Engineering, University of Waterloo, 200 University Avenue West, Waterloo, ON, N2L 3G1, Canada

    Muhammad Umair Shah

  2. Department of User Experience Design, Faculty of Liberal Arts, Wilfrid Laurier University, 73 George Street, Brantford, ON, N3T 2Y3, Canada

    Umair Rehman

  3. College of Technological Innovation, Zayed University, P.O. Box 144534, Abu Dhabi Campus, United Arab Emirates

    Farkhund Iqbal

  4. Department of Psychology, Faculty of Arts, University of Waterloo, 200 University Avenue West, Waterloo, ON, N2L 3G1, Canada

    Hassan Ilahi

Authors
  1. Muhammad Umair Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Umair Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farkhund Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hassan Ilahi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Umair Shah.

Ethics declarations

Ethics approval

We received clearance from the Office of Research Ethics at the University of Waterloo and Zayed University for conducting studies with human participation.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shah, M.U., Rehman, U., Iqbal, F. et al. Exploring the human factors in moral dilemmas of autonomous vehicles. Pers Ubiquit Comput 26, 1321–1331 (2022). https://doi.org/10.1007/s00779-022-01685-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-022-01685-x

Keywords

Search

Navigation