Severin Engelmann
Jens Grossklags
ABSTRACT
Faces play an indispensable role in human social life. At present, computer vision artificial intelligence (AI) captures and interprets human faces for a variety of digital applications and services. The ambiguity of facial information has recently led to a debate among scholars in different fields about the types of inferences AI should make about people based on their facial looks. AI research often justifies facial AI inference-making by referring to how people form impressions in first-encounter scenarios. Critics raise concerns about bias and discrimination and warn that facial analysis AI resembles an automated version of physiognomy. What has been missing from this debate, however, is an understanding of how “non-experts” in AI ethically evaluate facial AI inference-making. In a two-scenario vignette study with 24 treatment groups, we show that non-experts (N = 3745) reject facial AI inferences such as trustworthiness and likability from portrait images in a low-stake advertising and a high-stake hiring context. In contrast, non-experts agree with facial AI inferences such as skin color or gender in the advertising but not the hiring decision context. For each AI inference, we ask non-experts to justify their evaluation in a written response. Analyzing 29,760 written justifications, we find that non-experts are either “evidentialists” or “pragmatists”: they assess the ethical status of a facial AI inference based on whether they think faces warrant sufficient or insufficient evidence for an inference (evidentialist justification) or whether making the inference results in beneficial or detrimental outcomes (pragmatist justification). Non-experts’ justifications underscore the normative complexity behind facial AI inference-making. AI inferences with insufficient evidence can be rationalized by considerations of relevance while irrelevant inferences can be justified by reference to sufficient evidence. We argue that participatory approaches contribute valuable insights for the development of ethical AI in an increasingly visual data culture.
Supplemental Material
Available for Download
pdf
facct22-11_Appendix.pdf (6.1 MB)
FAccT'22 conference - Appendix to the paper with the ID 11 and title 'What People Think AI Should Infer From Faces'
- Noura Al Moubayed, Yolanda Vazquez-Alvarez, Alex McKay, and Alessandro Vinciarelli. 2014. Face-based automatic personality perception. In Proceedings of the 22nd ACM International Conference on Multimedia. 1153–1156. https://doi.org/10.1145/2647868.2655014Google Scholar
Digital Library
- Kwame Anthony Appiah. 2008. Experiments in Ethics. Harvard University Press.Google Scholar
- Christiane Atzmüller and Peter M. Steiner. 2010. Experimental vignette studies in survey research. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 6, 3 (2010), 128–138. https://doi.org/10.1027/1614-2241/a000014Google ScholarCross Ref
- Danny Azucar, Davide Marengo, and Michele Settanni. 2018. Predicting the Big 5 personality traits from digital footprints on social media: A meta-analysis. Personality and Individual Differences 124 (2018), 150–159. https://doi.org/10.1016/j.paid.2017.12.018Google ScholarCross Ref
- Mitja Back, Juliane Stopfer, Simine Vazire, Sam Gaddis, Stefan Schmukle, Boris Egloff, and Samuel Gosling. 2010. Facebook profiles reflect actual personality, not self-idealization. Psychological Science 21, 3 (2010), 372–374. https://doi.org/10.1177/0956797609360756Google ScholarCross Ref
- Charles C. Ballew and Alexander Todorov. 2007. Predicting political elections from rapid and unreflective face judgments. Proceedings of the National Academy of Sciences 104, 46(2007), 17948–17953. https://doi.org/10.1073/pnas.0705435104Google ScholarCross Ref
- Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M. Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest 20, 1 (2019), 1–68. https://doi.org/10.1177/1529100619832930Google ScholarCross Ref
- Johannes M. Basch and Klaus G. Melchers. 2019. Fair and flexible?! Explanations can improve applicant reactions toward asynchronous video interviews. Personnel Assessment and Decisions 5, 3 (2019), Article 2. https://doi.org/10.25035/pad.2019.03.002Google ScholarCross Ref
- Adam Bear and Joshua Knobe. 2017. Normality: Part descriptive, part prescriptive. Cognition 167(2017), 25–37. https://doi.org/10.1016/j.cognition.2016.10.024Google ScholarCross Ref
- C. Fabian Benitez-Quiroz, Ramprakash Srinivasan, and Aleix M. Martinez. 2016. EmotioNet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5562–5570. https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Benitez-Quiroz_EmotioNet_An_Accurate_CVPR_2016_paper.htmlGoogle ScholarCross Ref
- Joan-Isaac Biel, Lucía Teijeiro-Mosquera, and Daniel Gatica-Perez. 2012. Facetube: Predicting personality from facial expressions of emotion in online conversational video. In Proceedings of the 14th ACM International Conference on Multimodal Interaction. 53–56. https://doi.org/10.1145/2388676.2388689Google ScholarDigital Library
- Elettra Bietti. 2020. From ethics washing to ethics bashing: A view on tech ethics from within moral philosophy. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 210–219. https://dl.acm.org/doi/abs/10.1145/3351095.3372860Google ScholarDigital Library
- Jean-François Bonnefon, Astrid Hopfensitz, Wim De Neys, 2015. Face-ism and kernels of truth in facial inferences. Trends in Cognitive Sciences 19, 8 (2015), 421–422. https://doi.org/10.1016/j.tics.2015.05.002Google ScholarCross Ref
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. PMLR, 77–91. https://proceedings.mlr.press/v81/buolamwini18a.htmlGoogle Scholar
- Filippo Cavallo, Francesco Semeraro, Laura Fiorini, Gergely Magyar, Peter Sinčák, and Paolo Dario. 2018. Emotion modelling for social robotics applications: A review. Journal of Bionic Engineering 15, 2 (2018), 185–203. https://doi.org/10.1007/s42235-018-0015-yGoogle ScholarCross Ref
- Fabio Celli, Elia Bruni, and Bruno Lepri. 2014. Automatic personality and interaction style recognition from Facebook profile pictures. In Proceedings of the 22nd ACM International Conference on Multimedia. 1101–1104. https://doi.org/10.1145/2647868.2654977Google ScholarDigital Library
- Dennis Chong and James N. Druckman. 2007. Framing Theory. Annual Review of Political Science 10, 1 (2007), 103–126. https://doi.org/10.1146/annurev.polisci.10.072805.103054Google ScholarCross Ref
- Cory J. Clark, Jamie B. Luguri, Peter H. Ditto, Joshua Knobe, Azim F. Shariff, and Roy F. Baumeister. 2014. Free to punish: A motivated account of free will belief. Journal of Personality and Social Psychology 106, 4(2014), 501–513. https://doi.org/10.1037/a0035880Google ScholarCross Ref
- Jeff F. Cohn and Fernando De la Torre. 2015. Automated face analysis for affective computing. In The Oxford Handbook of Affective Computing, Rafael A Calvo, Sidney D’Mello, Jonathan Matthew Gratch, and Arvid Kappas (Eds.). Oxford University Press, 131–150. https://doi.org/10.1093/oxfordhb/9780199942237.013.020Google ScholarCross Ref
- Kate Crawford. 2021. Time to regulate AI that interprets human emotions. Nature 592, 7853 (2021), 167–167. https://doi.org/10.1038/d41586-021-00868-5Google ScholarCross Ref
- Kate Crawford, Roel Dobbe, Theodora Dryer, Genevieve Fried, Ben Green, Elizabeth Kaziunas, Amba Kak, Varoon Mathur, Erin McElroy, A. Sánchez, 2019. AI Now 2019 report. Research Report. The AI Now Institute, NYU. https://ainowinstitute.org/AI_Now_2019_Report.pdfGoogle Scholar
- Kate Crawford and Trevor Paglen. 2019. Excavating AI: The politics of images in machine learning training sets. Research Report. The AI Now Institute, NYU. https://excavating.ai/Google Scholar
- Joanna Demaree-Cotton. 2016. Do framing effects make moral intuitions unreliable?Philosophical Psychology 29, 1 (2016), 1–22. https://doi.org/10.1080/09515089.2014.989967Google ScholarCross Ref
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv (2018). arxiv:1810.04805Google Scholar
- Abhinav Dhall, Oruganti V. Ramana Murthy, Roland Goecke, Jyoti Joshi, and Tom Gedeon. 2015. Video and image based emotion recognition challenges in the wild: Emotiw 2015. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 423–426. https://doi.org/10.1145/2818346.2829994Google ScholarDigital Library
- Rafaële Dumas and Benoît Testé. 2006. The influence of criminal facial stereotypes on juridic judgments. Swiss Journal of Psychology/Schweizerische Zeitschrift für Psychologie/Revue Suisse de Psychologie 65, 4 (2006), 237–244. https://doi.org/10.1024/1421-0185.65.4.237Google ScholarCross Ref
- Charles Efferson and Sonja Vogt. 2013. Viewing men’s faces does not lead to accurate predictions of trustworthiness. Scientific Reports 3(2013), Article 1047. https://doi.org/10.1038/srep01047Google ScholarCross Ref
- Paul Ekman and Wallace V. Friesen. 2003. Unmasking the face: A guide to recognizing emotions from facial clues. Malor Books.Google Scholar
- Severin Engelmann and Jens Grossklags. 2019. Setting the stage: Towards principles for reasonable image inferences. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization. 301–307. https://doi.org/10.1145/3314183.3323846Google ScholarDigital Library
- Bruce Ferwerda, Markus Schedl, and Marko Tkalcic. 2015. Predicting personality traits with Instagram pictures. In Proceedings of the Workshop on Emotions and Personality in Personalized Systems. 7–10. https://doi.org/10.1145/2809643.2809644Google ScholarDigital Library
- Bruce Ferwerda, Markus Schedl, and Marko Tkalcic. 2016. Using Instagram picture features to predict users’ personality. In Proceedings of the 22nd International Conference on Multimedia Modeling. 850–861. https://doi.org/10.1007/978-3-319-27671-7_71Google ScholarDigital Library
- Bruce Ferwerda and Marko Tkalcic. 2018. Predicting users’ personality from Instagram pictures: Using visual and/or content features?. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization. 157–161. https://doi.org/10.1145/3209219.3209248Google ScholarDigital Library
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Communications of the ACM 64, 12 (2021), 86–92. https://doi.org/10.1145/3458723Google ScholarDigital Library
- R. Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage in, garbage out? Do machine learning application papers in social computing report where human-labeled training data comes from?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 325–336. https://doi.org/10.1145/3351095.3372862Google ScholarDigital Library
- Jake Goldenfein. 2019. The profiling potential of computer vision and the challenge of computational empiricism. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 110–119. https://doi.org/10.1145/3287560.3287568Google ScholarDigital Library
- Armin Granulo, Christoph Fuchs, and Stefano Puntoni. 2019. Psychological reactions to human versus robotic job replacement. Nature Human Behaviour 3, 10 (2019), 1062–1069. https://doi.org/10.1038/s41562-019-0670-yGoogle ScholarCross Ref
- Igor Grossmann, Richard P. Eibach, Jacklyn Koyama, and Qaisar B. Sahi. 2020. Folk standards of sound judgment: Rationality versus reasonableness. Science Advances 6, 2 (2020), Article eaaz0289. https://doi.org/10.1126/sciadv.aaz0289Google ScholarCross Ref
- Yağmur Güçlütürk, Umut Güçlü, Xavier Baro, Hugo Jair Escalante, Isabelle Guyon, Sergio Escalera, Marcel A.J. Van Gerven, and Rob Van Lier. 2017. Multimodal first impression analysis with deep residual networks. IEEE Transactions on Affective Computing 9, 3 (2017), 316–329. https://doi.org/10.1109/TAFFC.2017.2751469Google ScholarDigital Library
- Sharath Chandra Guntuku, Weisi Lin, Jordan Carpenter, Wee Keong Ng, Lyle H. Ungar, and Daniel Preoţiuc-Pietro. 2017. Studying personality through the content of posted and liked images on Twitter. In Proceedings of the 2017 ACM on Web Science Conference. 223–227. https://doi.org/10.1145/3091478.3091522Google ScholarDigital Library
- Margot Hanley, Solon Barocas, Karen Levy, Shiri Azenkot, and Helen Nissenbaum. 2021. Computer vision and conflicting values: Describing people with automated alt text. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 543–554. https://doi.org/10.48550/arXiv.2105.12754Google ScholarDigital Library
- Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018. The dataset nutrition label: A framework to drive higher data quality standards. arXiv preprint (2018). https://doi.org/10.48550/arXiv.1805.03677Google ScholarCross Ref
- Michael R. Hyman and Susan D. Steiner. 1996. The vignette method in business ethics research: Current uses, limitations, and recommendations. In Proceedings of the Annual Meeting of the Southern Marketing Association. 261–265.Google Scholar
- Srirang K. Jha, Shweta Jha, and Manoj Kumar Gupta. 2020. Leveraging artificial intelligence for effective recruitment and selection processes. In Proceedings of the International Conference on Communication, Computing and Electronics Systems. 287–293. https://doi.org/10.1007/978-981-15-2612-1_27Google ScholarCross Ref
- Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature Machine Intelligence 1, 9 (2019), 389–399. https://doi.org/10.1038/s42256-019-0088-2Google ScholarCross Ref
- Alexander Kachur, Evgeny Osin, Denis Davydov, Konstantin Shutilov, and Alexey Novokshonov. 2020. Assessing the Big Five personality traits using real-life static facial images. Scientific Reports 10(2020), Article 8487. https://doi.org/10.1038/s41598-020-65358-6Google ScholarCross Ref
- Os Keyes. 2018. The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–22. https://doi.org/10.1145/3274357Google ScholarDigital Library
- Zaid Khan and Yun Fu. 2021. One label, one billion faces: Usage and consistency of racial categories in computer vision. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 587–597. https://doi.org/10.48550/arXiv.2102.02320Google ScholarDigital Library
- Kimon Kieslich, Marco Lünich, and Frank Marcinkowski. 2021. The threats of artificial intelligence scale (TAI). International Journal of Social Robotics 13 (2021), 1563–1577. https://doi.org/10.1007/s12369-020-00734-wGoogle ScholarCross Ref
- Owen C. King. 2019. Machine learning and irresponsible inference: Morally assessing the training data for image recognition systems. In On the Cognitive, Ethical, and Scientific Dimensions of Artificial Intelligence, Don Berkich and Matteo V. d’Alfonso (Eds.). Springer, 265–282. https://doi.org/10.1007/978-3-030-01800-9_14Google ScholarCross Ref
- Owen C. King. 2020. Presumptuous aim attribution, conformity, and the ethics of artificial social cognition. Ethics and Information Technology 22, 1 (2020), 25–37. https://doi.org/10.1007/s10676-019-09512-3Google ScholarCross Ref
- Karel Kleisner, Veronika Chvátalová, and Jaroslav Flegr. 2014. Perceived intelligence is associated with measured intelligence in men but not women. PloS ONE 9, 3 (2014), Article e81237. https://doi.org/10.1371/journal.pone.0081237Google ScholarCross Ref
- Joshua Knobe. 2003. Intentional action and side effects in ordinary language. Analysis 63, 3 (2003), 190–194. https://www.jstor.org/stable/3329308Google ScholarCross Ref
- Joshua Knobe and Shaun Nichols. 2017. Experimental Philosophy. In The Stanford Encyclopedia of Philosophy (Winter 2017 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University.Google Scholar
- Michal Kosinski. 2021. Facial recognition technology can expose political orientation from naturalistic facial images. Scientific Reports 11, 100 (2021). https://doi.org/10.1038/s41598-020-79310-1Google ScholarCross Ref
- Steven R. Kraaijeveld. 2021. Experimental philosophy of technology. Philosophy & Technology 34 (2021), 993–1012. https://doi.org/10.1007/s13347-021-00447-6Google ScholarCross Ref
- Robin S. Kramer and Robert Ward. 2010. Internal facial features are signals of personality and health. The Quarterly Journal of Experimental Psychology 63, 11(2010), 2273–2287. https://doi.org/10.1080/17470211003770912Google ScholarCross Ref
- Mucahid Kutlu, Tyler McDonnell, Yassmine Barkallah, Tamer Elsayed, and Matthew Lease. 2018. Crowd vs. expert: What can relevance judgment rationales teach us about assessor disagreement?. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 805–814. https://doi.org/10.1145/3209978.3210033Google ScholarDigital Library
- Mucahid Kutlu, Tyler McDonnell, Tamer Elsayed, and Matthew Lease. 2020. Annotator rationales for labeling tasks in crowdsourcing. Journal of Artificial Intelligence Research 69 (2020), 143–189. https://doi.org/10.1613/jair.1.12012Google ScholarCross Ref
- Gabriel S. Lenz and Chappell Lawson. 2011. Looking the part: Television leads less informed citizens to vote based on candidates’ appearance. American Journal of Political Science 55, 3 (2011), 574–589. https://doi.org/10.1111/j.1540-5907.2011.00511.xGoogle ScholarCross Ref
- John Leuner. 2019. A replication study: Machine learning models are capable of predicting sexual orientation from facial images. arXiv preprint arXiv:1902.10739(2019). https://doi.org/10.48550/arXiv.1902.10739Google ScholarCross Ref
- Lan Li, Tina Lassiter, Joohee Oh, and Min Kyung Lee. 2021. Algorithmic hiring in practice: Recruiter and HR professional’s perspectives on AI use in hiring. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 166–176. https://doi.org/10.1145/3461702.3462531Google ScholarDigital Library
- Anthony C. Little and David I. Perrett. 2007. Using composite images to assess accuracy in personality attribution to faces. British Journal of Psychology 98, 1 (2007), 111–126. https://doi.org/10.1348/000712606X109648Google ScholarCross Ref
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint (2019). arxiv:1907.11692Google Scholar
- Bertram F. Malle and Joshua Knobe. 1997. The folk concept of intentionality. Journal of Experimental Social Psychology 33, 2 (1997), 101–121. https://doi.org/10.1006/jesp.1996.1314Google ScholarCross Ref
- Julie M. McCarthy, Talya N. Bauer, Donald M. Truxillo, Neil R. Anderson, Ana Cristina Costa, and Sara M. Ahmed. 2017. Applicant perspectives during selection: A review addressing “So what?,”“What’s new?,” and “Where to next?”. Journal of Management 43, 6 (2017), 1693–1725. https://doi.org/10.1177/0149206316681846Google ScholarCross Ref
- David E. Melnikoff and Nina Strohminger. 2020. The automatic influence of advocacy on lawyers and novices. Nature Human Behaviour 4(2020), 1258–1264. https://doi.org/10.1038/s41562-020-00943-3Google ScholarCross Ref
- Milagros Miceli and Julian Posada. 2021. Wisdom for the crowd: Discoursive power in annotation instructions for computer vision. arXiv preprint (2021). https://doi.org/10.48550/arXiv.2105.10990Google ScholarCross Ref
- Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020. Between subjectivity and imposition: Power dynamics in data annotation for computer vision. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–25. https://doi.org/10.1145/3415186Google ScholarDigital Library
- Milagros Miceli, Tianling Yang, Laurens Naudts, Martin Schuessler, Diana Serbanescu, and Alex Hanna. 2021. Documenting computer vision datasets: An invitation to reflexive data practices. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 161–172. https://doi.org/10.1145/3442188.3445880Google ScholarDigital Library
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 220–229. https://doi.org/10.1145/3287560.3287596Google ScholarDigital Library
- Brent Mittelstadt. 2019. Principles alone cannot guarantee ethical AI. Nature Machine Intelligence 1, 11 (2019), 501–507. https://doi.org/10.1038/s42256-019-0114-4Google ScholarCross Ref
- Laura Naumann, Simine Vazire, Peter Rentfrow, and Samuel Gosling. 2009. Personality judgments based on physical appearance. Personality and Social Psychology Bulletin 35, 12 (2009), 1661–1671. https://doi.org/10.1177/0146167209346309Google ScholarCross Ref
- Shaun Nichols and Joshua Knobe. 2007. Moral responsibility and determinism: The cognitive science of folk intuitions. Noûs 41, 4 (2007), 663–685. https://www.jstor.org/stable/4494554Google ScholarCross Ref
- Stefanie Nowak and Stefan Rüger. 2010. How reliable are annotations via crowdsourcing: A study about inter-annotator agreement for multi-label image annotation. In Proceedings of the International Conference on Multimedia Information Retrieval. 557–566. https://doi.org/10.1145/1743384.1743478Google ScholarDigital Library
- Christopher Y. Olivola, Dawn L. Eubanks, and Jeffrey B. Lovelace. 2014. The many (distinctive) faces of leadership: Inferring leadership domain from facial appearance. The Leadership Quarterly 25, 5 (2014), 817–834. https://doi.org/10.1016/j.leaqua.2014.06.002Google ScholarCross Ref
- Christopher Y. Olivola, Friederike Funk, and Alexander Todorov. 2014. Social attributions from faces bias human choices. Trends in Cognitive Sciences 18, 11 (2014), 566–570. https://doi.org/10.1016/j.tics.2014.09.007Google ScholarCross Ref
- Christopher Y. Olivola, Abigail B. Sussman, Konstantinos Tsetsos, Olivia E. Kang, and Alexander Todorov. 2012. Republicans prefer Republican-looking leaders: Political facial stereotypes predict candidate electoral success among right-leaning voters. Social Psychological and Personality Science 3, 5 (2012), 605–613. https://doi.org/10.1177/1948550611432770Google ScholarCross Ref
- Harriet Over and Richard Cook. 2018. Where do spontaneous first impressions of faces come from?Cognition 170(2018), 190–200. https://doi.org/10.1016/j.cognition.2017.10.002Google ScholarCross Ref
- Harriet Over, Adam Eggleston, and Richard Cook. 2020. Ritual and the origins of first impressions. Philosophical Transactions of the Royal Society B 375, 1805(2020), Article 20190435. https://doi.org/10.1098/rstb.2019.0435Google ScholarCross Ref
- Ian S. Penton-Voak, Nicholas Pound, Anthony C. Little, and David I. Perrett. 2006. Personality judgments from natural and composite facial images: More evidence for a “kernel of truth” in social perception. Social Cognition 24, 5 (2006), 607–640. https://doi.org/10.1521/soco.2006.24.5.607Google ScholarCross Ref
- Lin Qiu, Jiahui Lu, Shanshan Yang, Weina Qu, and Tingshao Zhu. 2015. What does your selfie say about you?Computers in Human Behavior 52 (2015), 443–449. https://doi.org/10.1016/j.chb.2015.06.032Google ScholarDigital Library
- Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 469–481. https://doi.org/10.1145/3351095.3372828Google ScholarDigital Library
- Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, and Emily Denton. 2020. Saving face: Investigating the ethical concerns of facial recognition auditing. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 145–151. https://doi.org/10.1145/3375627.3375820Google ScholarDigital Library
- Nicholas O. Rule and Nalini Ambady. 2008. The face of success: Inferences from chief executive officers’ appearance predict company profits. Psychological Science 19, 2 (2008), 109–111. https://doi.org/10.1111/j.1467-9280.2008.02054.xGoogle ScholarCross Ref
- Morgan Klaus Scheuerman, Alex Hanna, and Emily Denton. 2021. Do datasets have politics? Disciplinary values in computer vision dataset development. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2(2021), 1–37. https://doi.org/10.1145/3476058Google ScholarDigital Library
- Morgan Klaus Scheuerman, Madeleine Pape, and Alex Hanna. 2021. Auto-essentialization: Gender in automated facial analysis as extended colonial project. Big Data & Society 8, 2 (2021), Article 20539517211053712. https://doi.org/10.1177/20539517211053712Google ScholarCross Ref
- Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R. Brubaker. 2020. How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1(2020), 1–35. https://doi.org/10.1145/3392866Google ScholarDigital Library
- Cristina Segalin, Dong Seon Cheng, and Marco Cristani. 2017. Social profiling through image understanding: Personality inference using convolutional neural networks. Computer Vision and Image Understanding 156 (2017), 34–50. https://doi.org/10.1016/j.cviu.2016.10.013Google ScholarDigital Library
- Crisitina Segalin, Alessandro Perina, Marco Cristani, and Alessandro Vinciarelli. 2016. The pictures we like are our image: Continuous mapping of favorite pictures into self-assessed and attributed personality traits. IEEE Transactions on Affective Computing 8, 2 (2016), 268–285. https://doi.org/10.1109/TAFFC.2016.2516994Google ScholarDigital Library
- Aaron Smith, Lee Rainie, Kenneth Olmstead, Jingjing Jiang, Andrew Perrin, Paul Hitlin, and Meg Hefferon. 2018. Public attitudes toward computer algorithms. Pew Research Center 16(2018). https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2018/11/PI_2018.11.19_algorithms_FINAL.pdfGoogle Scholar
- Alexander Sorokin and David Forsyth. 2008. Utility data annotation with Amazon Mechanical Turk. In Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 1–8. https://doi.org/10.1109/CVPRW.2008.4562953Google ScholarCross Ref
- Luke Stark. 2019. Facial recognition is the plutonium of AI. XRDS: Crossroads, The ACM Magazine for Students 25, 3 (2019), 50–55. https://doi.org/10.1145/3313129Google ScholarDigital Library
- Luke Stark and Jesse Hoey. 2021. The ethics of emotion in artificial intelligence systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 782–793. https://doi.org/10.1145/3442188.3445939Google ScholarDigital Library
- Luke Stark and Jevan Hutson. 2021. Physiognomic artificial intelligence. Available at SSRN (2021). https://doi.org/10.2139/ssrn.3927300Google ScholarCross Ref
- Hao Su, Jia Deng, and Li Fei-Fei. 2012. Crowdsourcing annotations for visual object detection. In Proceedings of the Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence. 40–46. https://www.aaai.org/ocs/index.php/WS/AAAIW12/paper/download/5350/5599Google Scholar
- Prasanna Tambe, Peter Cappelli, and Valery Yakubovich. 2019. Artificial intelligence in human resources management: Challenges and a path forward. California Management Review 61, 4 (2019), 15–42. https://doi.org/10.1177/0008125619867910Google ScholarCross Ref
- Adriana Tapus, Antonio Bandera, Ricardo Vazquez-Martin, and Luis V. Calderita. 2019. Perceiving the person and their interactions with the others for social robotics – A review. Pattern Recognition Letters 118 (2019), 3–13. https://doi.org/10.1016/j.patrec.2018.03.006Google ScholarCross Ref
- Thales Teixeira, Michel Wedel, and Rik Pieters. 2012. Emotion-induced engagement in internet video advertisements. Journal of Marketing Research 49, 2 (2012), 144–159. https://doi.org/10.1509/jmr.10.0207Google ScholarCross Ref
- Alexander Todorov. 2017. Face Value: The Irresistible Influence of First Impressions. Princeton University Press.Google Scholar
- Alexander Todorov, Sean G. Baron, and Nikolaas N. Oosterhof. 2008. Evaluating face trustworthiness: A model based approach. Social Cognitive and Affective Neuroscience 3, 2 (2008), 119–127. https://doi.org/10.1093/scan/nsn009Google ScholarCross Ref
- Alexander Todorov, Christopher Y. Olivola, Ron Dotsch, and Peter Mende-Siedlecki. 2015. Social attributions from faces: Determinants, consequences, accuracy, and functional significance. Annual Review of Psychology 66 (2015), 519–545. https://doi.org/10.1016/j.tics.2014.09.007Google ScholarCross Ref
- Richard J.W. Vernon, Clare A.M. Sutherland, Andrew W. Young, and Tom Hartley. 2014. Modeling first impressions from highly variable facial images. Proceedings of the National Academy of Sciences 111, 32(2014), E3353–E3361. https://doi.org/10.1073/pnas.1409860111Google ScholarCross Ref
- Yilun Wang and Michal Kosinski. 2018. Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology 114, 2(2018), 246–257. https://doi.org/10.1037/pspa0000098Google ScholarCross Ref
- Vanessa Williamson. 2016. On the ethics of crowdsourced research. PS: Political Science & Politics 49, 1 (2016), 77–81. https://doi.org/10.1017/S104909651500116XGoogle ScholarCross Ref
- John Paul Wilson and Nicholas O. Rule. 2015. Facial Trustworthiness Predicts Extreme Criminal-Sentencing Outcomes. Psychological Science 26, 8 (2015), 1325–1331. https://doi.org/10.1177/0956797615590992Google ScholarCross Ref
- Gerhard Wohlgenannt. 2016. A comparison of domain experts and crowdsourcing regarding concept relevance evaluation in ontology learning. In International Workshop on Multi-disciplinary Trends in Artificial Intelligence. Springer, 243–254. https://doi.org/10.1007/978-3-319-49397-8 21Google ScholarCross Ref
- Nan Xi, Di Ma, Marcus Liou, Zachary C. Steinert-Threlkeld, Jason Anastasopoulos, and Jungseock Joo. 2020. Understanding the political ideology of legislators from social media images. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media. 726–737. https://ojs.aaai.org/index.php/ICWSM/article/view/7338Google ScholarCross Ref
- Yan Yan, Jie Nie, Lei Huang, Zhen Li, Qinglei Cao, and Zhiqiang Wei. 2015. Is your first impression reliable? Trustworthy analysis using facial traits in portraits. In Proceedings of the 21st International Conference on Multimedia Modeling. 148–158. https://doi.org/10.1007/978-3-319-14442-9_13Google ScholarCross Ref
- Leslie A. Zebrowitz and Susan M. McDonald. 1991. The impact of litigants’ baby-facedness and attractiveness on adjudications in small claims courts. Law and Human Behavior 15, 6 (1991), 603–623. https://doi.org/10.1007/BF01065855Google ScholarCross Ref
- Leslie A. Zebrowitz and Joann M. Montepare. 2008. Social psychological face perception: Why appearance matters. Social and Personality Psychology Compass 2, 3 (2008), 1497–1517. https://doi.org/10.1111/j.1751-9004.2008.00109.xGoogle ScholarCross Ref
Index Terms
- What People Think AI Should Infer From Faces
Index terms have been assigned to the content through auto-classification.
Recommendations
AI-Competent Individuals and Laypeople Tend to Oppose Facial Analysis AI
EAAMO '22: Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and OptimizationRecent advances in computer vision analysis have led to a debate about the kinds of conclusions artificial intelligence (AI) should make about people based on their faces. Some scholars have argued for supposedly “common sense” facial inferences that ...
Reconciliation between factions focused on near-term and long-term artificial intelligence
Artificial intelligence (AI) experts are currently divided into "presentist" and "futurist" factions that call for attention to near-term and long-term AI, respectively. This paper argues that the presentist---futurist dispute is not the best focus of ...
Of, for, and by the people: the legal lacuna of synthetic persons
Conferring legal personhood on purely synthetic entities is a very real legal possibility, one under consideration presently by the European Union. We show here that such legislative action would be morally unnecessary and legally troublesome. While AI ...
Comments