Sina Mohseni
Skip Abstract Section
Abstract
The need for interpretable and accountable intelligent systems grows along with the prevalence of artificial intelligence (AI) applications used in everyday life. Explainable AI (XAI) systems are intended to self-explain the reasoning behind system decisions and predictions. Researchers from different disciplines work together to define, design, and evaluate explainable systems. However, scholars from different disciplines focus on different objectives and fairly independent topics of XAI research, which poses challenges for identifying appropriate design and evaluation methodology and consolidating knowledge across efforts. To this end, this article presents a survey and framework intended to share knowledge and experiences of XAI design and evaluation methods across multiple disciplines. Aiming to support diverse design goals and evaluation methods in XAI research, after a thorough review of XAI related papers in the fields of machine learning, visualization, and human-computer interaction, we present a categorization of XAI design goals and evaluation methods. Our categorization presents the mapping between design goals for different XAI user groups and their evaluation methods. From our findings, we develop a framework with step-by-step design guidelines paired with evaluation methods to close the iterative design and evaluation cycles in multidisciplinary XAI teams. Further, we provide summarized ready-to-use tables of evaluation methods and recommendations for different goals in XAI research.
- Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 582.Google Scholar
Digital Library
- Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138–52160.Google ScholarCross Ref
- Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems. 9505–9515.Google Scholar
- Yongsu Ahn and Yu-Ru Lin. 2019. FairSight: Visual analytics for fairness in decision making. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 1086–1095.Google Scholar
- Eric Alexander and Michael Gleicher. 2015. Task-driven comparison of topic models. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2015), 320–329.Google ScholarDigital Library
- Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI Magazine 35, 4 (2014), 105–120.Google ScholarDigital Library
- Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 3.Google ScholarDigital Library
- Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in AI safety. arxiv:1606.06565. http://arxiv.org/abs/1606.06565.Google Scholar
- Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20, 3 (2018), 973–989.Google ScholarCross Ref
- Stavros Antifakos, Nicky Kern, Bernt Schiele, and Adrian Schwaninger. 2005. Towards improving trust in context-aware systems by displaying system confidence. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services. ACM, 9–14.Google ScholarDigital Library
- Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82–115.Google ScholarDigital Library
- Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. 2014. Multiple object recognition with visual attention. arXiv:1412.7755.Google Scholar
- Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One 10, 7 (2015), e0130140.Google ScholarCross Ref
- David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller. 2010. How to explain individual classification decisions. Journal of Machine Learning Research 11, (June 2010), 1803–1831.Google Scholar
- Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in Human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2–11.Google ScholarCross Ref
- Victoria Bellotti and Keith Edwards. 2001. Intelligibility and accountability: Human considerations in context-aware systems. Human–Computer Interaction 16, 2-4 (2001), 193–212.Google ScholarDigital Library
- Shlomo Berkovsky, Ronnie Taib, and Dan Conway. 2017. How to recommend?: User trust factors in movie recommender systems. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (IUI’17). ACM, New York, NY, 287–300. DOI:https://doi.org/10.1145/3025171.3025209Google ScholarDigital Library
- Daniel M. Best, Alex Endert, and Daniel Kidwell. 2014. 7 key challenges for visualization in cyber network defense. In Proceedings of the 11th Workshop on Visualization for Cyber Security. ACM, 33–40.Google ScholarDigital Library
- Mustafa Bilgic and Raymond J. Mooney. 2005. Explaining recommendations: Satisfaction vs. promotion. In Beyond Personalization Workshop, IUI, Vol. 5. 153.Google Scholar
- Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. “It’s reducing a human being to a percentage”: Perceptions of justice in algorithmic decisions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 377.Google ScholarDigital Library
- Philip Bobko, Alex J. Barelka, and Leanne M. Hirshfield. 2014. The construct of state-level suspicion: A model and research agenda for automated and information technology (IT) contexts. Human Factors 56, 3 (2014), 489–508.Google ScholarCross Ref
- Mariusz Bojarski, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Larry J. Ackel, Urs Muller, Phil Yeres, and Karol Zieba. 2018. Visualbackprop: Efficient visualization of CNNs for autonomous driving. In 2018 IEEE International Conference on Robotics and Automation (ICRA’18). IEEE, 1–8.Google ScholarCross Ref
- Engin Bozdag and Jeroen van den Hoven. 2015. Breaking the filter bubble: Democracy and design. Ethics and Information Technology 17, 4 (2015), 249–265.Google ScholarDigital Library
- Nicholas Bryan and Gautham Mysore. 2013. An efficient posterior regularized latent variable model for interactive sound source separation. In International Conference on Machine Learning. 208–216.Google Scholar
- Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are explanations always important?: A study of deployed, low-cost intelligent interactive systems. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces. ACM, 169–178.Google ScholarDigital Library
- Adrian Bussone, Simone Stumpf, and Dympna O’Sullivan. 2015. The role of explanations on trust and reliance in clinical decision support systems. In International Conference on Healthcare Informatics (ICHI’15). IEEE, 160–169.Google ScholarDigital Library
- Angel Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, and Duen Horng Chau. 2019. FairVis: Visual analytics for discovering intersectional bias in machine learning. IEEE Conference on Visual Analytics Science and Technology (VAST’19).Google ScholarCross Ref
- Béatrice Cahour and Jean-François Forzy. 2009. Does projection into use improve trust and exploration? An example with a cruise control system. Safety Science 47, 9 (2009), 1260–1270.Google ScholarCross Ref
- Carrie J. Cai, Jonas Jongejan, and Jess Holbrook. 2019. The effects of example-based explanations in a machine learning interface. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 258–262.Google ScholarDigital Library
- Carrie J. Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S Corrado, Martin C. Stumpe, et al. 2019. Human-centered tools for coping with imperfect algorithms during medical decision-making. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarDigital Library
- Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1721–1730.Google ScholarDigital Library
- Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. 2019. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 339–348.Google ScholarDigital Library
- Jaegul Choo, Hanseung Lee, Jaeyeon Kihm, and Haesun Park. 2010. iVisClassifier: An interactive visual analytics system for classification based on supervised dimension reduction. In 2010 IEEE Symposium on Visual Analytics Science and Technology (VAST’10). IEEE, 27–34.Google ScholarCross Ref
- Jaegul Choo and Shixia Liu. 2018. Visual analytics for explainable deep learning. IEEE Computer Graphics and Applications 38, 4 (2018), 84–92.Google ScholarCross Ref
- Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5, 2 (2017), 153–163.Google ScholarCross Ref
- Michael Chromik, Malin Eiband, Sarah Theres Völkel, and Daniel Buschek. 2019. Dark patterns of explainability, transparency, and user control for intelligent systems.. In IUI Workshops.Google Scholar
- Lingyang Chu, Xia Hu, Juhua Hu, Lanjun Wang, and Jian Pei. 2018. Exact and consistent interpretation for piecewise linear neural networks: A closed form solution. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1244–1253.Google ScholarDigital Library
- Miruna-Adriana Clinciu and Helen Hastie. 2019. A survey of explainable AI terminology. In Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI’19). 8–13.Google ScholarCross Ref
- Sven Coppers, Jan Van den Bergh, Kris Luyten, Karin Coninx, Iulianna Van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. Intellingo: An intelligible translation environment. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 524.Google ScholarDigital Library
- Enrico Costanza, Joel E. Fischer, James A. Colley, Tom Rodden, Sarvapali D. Ramchurn, and Nicholas R. Jennings. 2014. Doing the laundry with agents: A field trial of a future smart energy system in the home. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 813–822.Google Scholar
- William Curran, Travis Moore, Todd Kulesza, Weng-Keen Wong, Sinisa Todorovic, Simone Stumpf, Rachel White, and Margaret Burnett. 2012. Towards recognizing cool: Can end users help computer vision recognize subjective attributes of objects in images?. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces. ACM, 285–288.Google ScholarDigital Library
- Abhishek Das, Harsh Agrawal, C. Lawrence Zitnick, Devi Parikh, and Dhruv Batra. 2016. Human attention in visual question answering: Do humans and deep networks look at the same regions?. In Conference on Empirical Methods in Natural Language Processing (EMNLP’16). https://computing.ece.vt.edu/ abhshkdz/vqa-hat/Google ScholarCross Ref
- Abhishek Das, Harsh Agrawal, Larry Zitnick, Devi Parikh, and Dhruv Batra. 2017. Human attention in visual question answering: Do humans and deep networks look at the same regions?Computer Vision and Image Understanding 163 (2017), 90–100.Google Scholar
- Amit Datta, Michael Carl Tschantz, and Anupam Datta. 2015. Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies 2015, 1 (2015), 92–112.Google ScholarCross Ref
- Nicholas Diakopoulos. 2014. Algorithmic-accountability: The investigation of black boxes. Tow Center for Digital Journalism (2014).Google Scholar
- Nicholas Diakopoulos. 2017. Enabling accountability of algorithmic media: Transparency as a constructive and critical lens. In Transparent Data Mining for Big and Small Data. Springer, 25–43.Google Scholar
- Jonathan Dodge, Sean Penney, Andrew Anderson, and Margaret M. Burnett. 2018. What should be in an XAI explanation? What IFT reveals. In IUI Workshops.Google Scholar
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arxiv:1702.08608. http://arxiv.org/abs/1702.08608.Google Scholar
- Finale Doshi-Velez, Mason Kortz, Ryan Budish, Christopher Bavitz, Samuel J. Gershman, David O’Brien, Stuart Shieber, Jim Waldo, David Weinberger, and Alexandra Wood. 2017. Accountability of AI under the law: The role of explanation. Berkman Center Research Publication Forthcoming (2017), 18–07.Google Scholar
- James K. Doyle, Michael J. Radzicki, and W. Scott Trees. 2008. Measuring change in mental models of complex dynamic systems. In Complex Decision Making. Springer, 269–294.Google Scholar
- Fan Du, Catherine Plaisant, Neil Spring, Kenyon Crowley, and Ben Shneiderman. 2019. EventAction: A visual analytics approach to explainable recommendation for event sequences. ACM Transactions on Interactive Intelligent Systems (TiiS) 9, 4 (2019), 1–31.Google ScholarDigital Library
- Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu. 2018. Towards explanation of DNN-based prediction with guided feature inversion. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1358–1367.Google ScholarDigital Library
- M. Du, N. Liu, F. Yang, and X. Hu. 2019. Learning credible deep neural networks with rationale regularization. In 2019 IEEE International Conference on Data Mining (ICDM’19). 150–159.Google Scholar
- John J. Dudley and Per Ola Kristensson. 2018. A review of user interface design for interactive machine learning. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 2 (2018), 8.Google ScholarDigital Library
- Malin Eiband, Daniel Buschek, Alexander Kremer, and Heinrich Hussmann. 2019. The impact of placebic explanations on trust in intelligent systems. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW0243.Google ScholarDigital Library
- Malin Eiband, Hanna Schneider, Mark Bilandzic, Julian Fazekas-Con, Mareike Haug, and Heinrich Hussmann. 2018. Bringing transparency design into practice. In 23rd International Conference on Intelligent User Interfaces (IUI’18). ACM, New York, NY, 211–223. DOI:https://doi.org/10.1145/3172944.3172961Google ScholarDigital Library
- A. Endert, W. Ribarsky, C. Turkay, B. L. Wong, Ian Nabney, I. Díaz Blanco, and F. Rossi. 2017. The state of the art in integrating machine learning into visual analytics. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 458–486.Google Scholar
- Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. I always assumed that I wasn’t really that close to [her]: Reasoning about Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 153–162.Google ScholarDigital Library
- Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. “Be careful; things can be worse than they appear”: Understanding biased algorithms and users’ behavior around them in rating platforms. In 11th International AAAI Conference on Web and Social Media.Google Scholar
- Raquel Florez-Lopez and Juan Manuel Ramon-Jeronimo. 2015. Enhancing accuracy and interpretability of ensemble strategies in credit risk assessment. A correlated-adjusted decision forest proposal. Expert Systems with Applications 42, 13 (2015), 5737–5753.Google ScholarDigital Library
- Ruth C. Fong and Andrea Vedaldi. 2017. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision. 3429–3437.Google Scholar
- Fatih Gedikli, Dietmar Jannach, and Mouzhi Ge. 2014. How should I explain? A comparison of different explanation types for recommender systems. International Journal of Human-Computer Studies 72, 4 (2014), 367–382.Google ScholarDigital Library
- Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. 2019. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems. 9273–9282.Google Scholar
- Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA’18). IEEE, 80–89.Google ScholarCross Ref
- Alyssa Glass, Deborah L. McGuinness, and Michael Wolverton. 2008. Toward establishing trust in adaptive agents. In Proceedings of the 13th International Conference on Intelligent User Interfaces. ACM, 227–236.Google ScholarDigital Library
- John Goodall, Eric D. Ragan, Chad A. Steed, Joel W. Reed, G. David Richardson, Kelly M. T. Huffer, Robert A. Bridges, and Jason A. Laska. 2018. Situ: Identifying and explaining suspicious behavior in networks. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 204–214.Google ScholarDigital Library
- Bryce Goodman and Seth Flaxman. 2017. European union regulations on algorithmic decision-making and a “right to explanation.”AI Magazine 38, 3 (2017), 50–57.Google ScholarDigital Library
- Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs. 2018. The dark (patterns) side of UX design. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 534.Google Scholar
- Shirley Gregor and Izak Benbasat. 1999. Explanations from intelligent systems: Theoretical foundations and implications for practice. Management Information Systems Quarterly 23, 4 (1999), 2.Google ScholarDigital Library
- Alex Groce, Todd Kulesza, Chaoqiang Zhang, Shalini Shamasunder, Margaret Burnett, Weng-Keen Wong, Simone Stumpf, Shubhomoy Das, Amber Shinsel, Forrest Bice, et al. 2014. You are the only possible oracle: Effective test selection for end users of interactive machine learning systems. IEEE Transactions on Software Engineering 40, 3 (2014), 307–323.Google ScholarDigital Library
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM Computing Surveys (CSUR) 51, 5 (2018), 93.Google ScholarDigital Library
- David Gunning. 2017. Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA) (2017).Google Scholar
- Aniko Hannak, Piotr Sapiezynski, Arash Molavi Kakhki, Balachander Krishnamurthy, David Lazer, Alan Mislove, and Christo Wilson. 2013. Measuring personalization of web search. In Proceedings of the 22nd International Conference on World Wide Web. ACM, 527–538.Google ScholarDigital Library
- Steven R. Haynes, Mark A. Cohen, and Frank E. Ritter. 2009. Designs for explaining intelligent agents. International Journal of Human-Computer Studies 67, 1 (2009), 90–110.Google ScholarDigital Library
- Jeffrey Heer. 2019. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences U S A 116, 6 (2019), 1844–1850.Google ScholarCross Ref
- Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018. Women also snowboard: Overcoming bias in captioning models. In European Conference on Computer Vision. Springer, 793–811.Google ScholarCross Ref
- Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work. ACM, 241–250.Google ScholarDigital Library
- Bernease Herman. 2017. The promise and peril of human evaluation for model interpretability. arxiv:1711.07414. https://arxiv.org/abs/1711.07414.Google Scholar
- Robert Hoffman, Tim Miller, Shane T. Mueller, Gary Klein, and William J. Clancey. 2018. Explaining explanation, part 4: A deep dive on deep nets. IEEE Intelligent Systems 33, 3 (2018), 87–95.Google ScholarCross Ref
- Robert R. Hoffman. 2017. Theory concepts measures but policies metrics. In Macrocognition Metrics and Scenarios. CRC Press, 35–42.Google Scholar
- Robert R. Hoffman, John K. Hawley, and Jeffrey M. Bradshaw. 2014. Myths of automation, part 2: Some very human consequences. IEEE Intelligent Systems 29, 2 (2014), 82–85.Google ScholarCross Ref
- Robert R. Hoffman, Matthew Johnson, Jeffrey M. Bradshaw, and Al Underbrink. 2013. Trust in automation. IEEE Intelligent Systems 28, 1 (2013), 84–88.Google ScholarDigital Library
- Robert R. Hoffman and Gary Klein. 2017. Explaining explanation, part 1: Theoretical foundations. IEEE Intelligent Systems 32, 3 (2017), 68–73.Google ScholarDigital Library
- Robert R. Hoffman, Shane T. Mueller, and Gary Klein. 2017. Explaining explanation, part 2: Empirical foundations. IEEE Intelligent Systems 32, 4 (2017), 78–86.Google ScholarDigital Library
- Robert R. Hoffman, Shane T. Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arxiv:1812.04608. https://arxiv.org/abs/1812.04608.Google Scholar
- Fred Hohman, Haekyu Park, Caleb Robinson, and Duen Horng Polo Chau. 2019. Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 1096–1106.Google ScholarDigital Library
- Fred Hohman, Arjun Srinivasan, and Steven M. Drucker. 2019. TeleGam: Combining visualization and verbalization for interpretable machine learning. IEEE Visualization Conference (VIS’19).Google Scholar
- Fred Matthew Hohman, Minsuk Kahng, Robert Pienta, and Duen Horng Chau. 2018. Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics 25, 8 (Aug. 2018), 2674–2693.Google Scholar
- Daniel Holliday, Stephanie Wilson, and Simone Stumpf. 2016. User trust in intelligent systems: A journey over time. In Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, 164–168.Google ScholarDigital Library
- Kristina Höök. 2000. Steps to take before intelligent user interfaces become real. Interacting with Computers 12, 4 (2000), 409–426.Google ScholarCross Ref
- Philip N. Howard and Bence Kollanyi. 2016. Bots, #StrongerIn, and #Brexit: Computational propaganda during the UK-EU referendum.Google Scholar
- Yuening Hu, Jordan Boyd-Graber, Brianna Satinoff, and Alison Smith. 2014. Interactive topic modeling. Machine Learning 95, 3 (2014), 423–469.Google ScholarDigital Library
- Shagun Jhaver, Yoni Karpfen, and Judd Antin. 2018. Algorithmic anxiety and coping strategies of Airbnb hosts. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 421.Google ScholarDigital Library
- Jiun-Yin Jian, Ann M. Bisantz, and Colin G. Drury. 2000. Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics 4, 1 (2000), 53–71.Google ScholarCross Ref
- Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, and Duen Horng Polo Chau. 2018. ActiVis: Visual exploration of industry-scale deep neural network models. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 88–97.Google ScholarCross Ref
- Matthew Kay, Tara Kola, Jessica R. Hullman, and Sean A. Munson. 2016. When (ish) is my bus?: User-centered visualizations of uncertainty in everyday, mobile predictive systems. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5092–5103.Google Scholar
- Frank C. Keil. 2006. Explanation and understanding. Annual Review of Psychology 57 (2006), 227–254.Google ScholarCross Ref
- Been Kim, Rajiv Khanna, and Oluwasanmi O. Koyejo. 2016. Examples are not enough, learn to criticize! criticism for interpretability. In Advances in Neural Information Processing Systems. 2280–2288.Google Scholar
- Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning. 2673–2682.Google Scholar
- Jaedeok Kim and Jingoo Seo. 2017. Human understandable explanation extraction for black-box classification models based on matrix factorization. arxiv:1709.06201. https://arxiv.org/abs/1709.06201.Google Scholar
- Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019. The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 267–280.Google Scholar
- Gary Klein. 2018. Explaining explanation, part 3: The causal landscape. IEEE Intelligent Systems 33, 2 (2018), 83–88.Google ScholarDigital Library
- Rafal Kocielnik, Saleema Amershi, and Paul N. Bennett. 2019. Will you accept an imperfect AI? Exploring designs for adjusting end-user expectations of AI systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.Google Scholar
- Johannes Kraus, David Scholz, Dina Stiegemeier, and Martin Baumann. 2019. The more you know: Trust dynamics and calibration in highly automated driving and the effects of take-overs, system malfunction, and system transparency. Human Factors (2019), 0018720819853686.Google Scholar
- Josua Krause, Aritra Dasgupta, Jordan Swartz, Yindalon Aphinyanaphongs, and Enrico Bertini. 2017. A workflow for visual diagnostics of binary classifiers using instance-level explanations. In 2017 IEEE Conference on Visual Analytics Science and Technology (VAST’17). IEEE, 162–172.Google ScholarCross Ref
- Josua Krause, Adam Perer, and Enrico Bertini. 2014. INFUSE: Interactive feature selection for predictive modeling of high dimensional data. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 1614–1623.Google ScholarCross Ref
- Josua Krause, Adam Perer, and Kenney Ng. 2016. Interacting with predictions: Visual inspection of black-box machine learning models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5686–5697.Google ScholarDigital Library
- Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces. ACM, 126–137.Google ScholarDigital Library
- Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: The effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 1–10. Google ScholarDigital Library
- Todd Kulesza, Simone Stumpf, Margaret Burnett, Weng-Keen Wong, Yann Riche, Travis Moore, Ian Oberst, Amber Shinsel, and Kevin McIntosh. 2010. Explanatory debugging: Supporting end-user debugging of machine-learned programs. In ,2010 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC’10). IEEE, 41–48.Google ScholarDigital Library
- Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, and Weng-Keen Wong. 2013. Too much, too little, or just right? Ways explanations impact end users’ mental models. In 2013 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC’13). IEEE, 3–10.Google ScholarCross Ref
- Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Samuel J. Gershman, and Finale Doshi-Velez. 2019. Human evaluation of models built for interpretability. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 59–67.Google ScholarCross Ref
- Himabindu Lakkaraju, Stephen H. Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1675–1684.Google ScholarDigital Library
- Ellen J. Langer, Arthur Blank, and Benzion Chanowitz. 1978. The mindlessness of ostensibly thoughtful action: The role of “placebic” information in interpersonal interaction.Journal of Personality and Social Psychology 36, 6 (1978), 635.Google Scholar
- Min Kyung Lee, Anuraag Jain, Hea Jin Cha, Shashank Ojha, and Daniel Kusbit. 2019. Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic mediation. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 182.Google ScholarDigital Library
- Min Kyung Lee, Daniel Kusbit, Evan Metsky, and Laura Dabbish. 2015. Working with machines: The impact of algorithmic and data-driven management on human workers. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 1603–1612.Google ScholarDigital Library
- Bruno Lepri, Nuria Oliver, Emmanuel Letouzé, Alex Pentland, and Patrick Vinck. 2017. Fair, transparent, and accountable algorithmic decision-making processes. Philosophy & Technology (2017), 1–17.Google Scholar
- Piyawat Lertvittayakumjorn and Francesca Toni. 2019. Human-grounded evaluations of explanation methods for text classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 5198–5208.Google ScholarCross Ref
- Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, David Madigan, et al. 2015. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350–1371.Google ScholarCross Ref
- Alexander Lex, Marc Streit, H.-J. Schulz, Christian Partl, Dieter Schmalstieg, Peter J. Park, and Nils Gehlenborg. 2012. StratomeX: Visual analysis of large-scale heterogeneous genomics data for cancer subtype characterization. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 1175–1184.Google Scholar
- Kunpeng Li, Ziyan Wu, Kuan-Chuan Peng, Jan Ernst, and Yun Fu. 2018. Tell me where to look: Guided attention inference network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9215–9223.Google ScholarCross Ref
- Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu. 2019. Attention bridging network for knowledge transfer. In Proceedings of the IEEE International Conference on Computer Vision. 5198–5207.Google ScholarCross Ref
- Brian Lim. 2011. Improving understanding, trust, and control with intelligibility in context-aware applications. Carnegie Mellon University.Google Scholar
- Brian Y. Lim and Anind K. Dey. 2009. Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th International Conference on Ubiquitous Computing. ACM, 195–204.Google Scholar
- Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2119–2128.Google ScholarDigital Library
- Brian Y. Lim, Qian Yang, Ashraf M. Abdul, and Danding Wang. 2019. Why these explanations? Selecting intelligibility types for explanation Goals. In IUI Workshops.Google Scholar
- Zachary C. Lipton. 2016. The mythos of model interpretability. arxiv:1606.03490. https://arxiv.org/abs/1606.03490.Google Scholar
- Mengchen Liu, Shixia Liu, Xizhou Zhu, Qinying Liao, Furu Wei, and Shimei Pan. 2016. An uncertainty-aware approach for exploratory microblog retrieval. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2016), 250–259.Google ScholarDigital Library
- Mengchen Liu, Jiaxin Shi, Kelei Cao, Jun Zhu, and Shixia Liu. 2018. Analyzing the training processes of deep generative models. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 77–87.Google ScholarCross Ref
- Mengchen Liu, Jiaxin Shi, Zhen Li, Chongxuan Li, Jun Zhu, and Shixia Liu. 2017. Towards better analysis of deep convolutional neural networks. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 91–100.Google ScholarDigital Library
- Shixia Liu, Xiting Wang, Jianfei Chen, Jim Zhu, and Baining Guo. 2014. TopicPanorama: A full picture of relevant topics. In 2014 IEEE Conference on Visual Analytics Science and Technology (VAST’14). IEEE, 183–192.Google ScholarCross Ref
- Tania Lombrozo. 2006. The structure and function of explanations. Trends in Cognitive Sciences 10, 10 (2006), 464–470.Google ScholarCross Ref
- Tania Lombrozo. 2009. Explanation and categorization: How “why?” informs “what?”. Cognition 110, 2 (2009), 248–253.Google ScholarCross Ref
- Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765–4774.Google ScholarDigital Library
- Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, (Nov. 2008), 2579–2605.Google Scholar
- Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th Australasian Conference on Information Systems, Vol. 53. Citeseer, 6–8.Google Scholar
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arxiv:1908.09635. https://arxiv.org/abs/1908.09635.Google Scholar
- Sarah Mennicken, Jo Vermeulen, and Elaine M. Huang. 2014. From today’s augmented houses to tomorrow’s smart homes: New directions for home automation research. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 105–115.Google Scholar
- Stephanie M. Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but I don’t know why: Effects of implicit attitudes toward automation on trust in an automated system. Human Factors 55, 3 (2013), 520–534.Google ScholarCross Ref
- Miriah Meyer, Michael Sedlmair, P. Samuel Quinan, and Tamara Munzner. 2015. The nested blocks and guidelines model. Information Visualization 14, 3 (2015), 234–249.Google ScholarCross Ref
- Debra Meyerson, Karl E. Weick, and Roderick M. Kramer. 1996. Swift trust and temporary groups. Trust in Organizations: Frontiers of Theory and Research 166 (1996), 195.Google Scholar
- Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (2019), 1–38.Google ScholarCross Ref
- Yao Ming, Shaozu Cao, Ruixiang Zhang, Zhen Li, Yuanzhe Chen, Yangqiu Song, and Huamin Qu. 2017. Understanding hidden memories of recurrent neural networks. In 2017 IEEE Conference on Visual Analytics Science and Technology (VAST’17). IEEE, 13–24.Google ScholarCross Ref
- Yao Ming, Huamin Qu, and Enrico Bertini. 2018. Rulematrix: Visualizing and understanding classifiers with rules. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 342–352.Google ScholarDigital Library
- Brent Mittelstadt. 2016. Automation, algorithms, and politics: Auditing for transparency in content personalization systems. International Journal of Communication 10 (2016), 12.Google Scholar
- Sina Mohseni, Akshay Jagadeesh, and Zhangyang Wang. 2019. Predicting model failure using saliency maps in autonomous driving systems. ICML Workshop on Uncertainty & Robustness in Deep Learning.Google Scholar
- Sina Mohseni, Mandar Pitale, Vasu Singh, and Zhangyang Wang. 2020. Practical solutions for machine learning safety in autonomous vehicles. In The AAAI Workshop on Artificial Intelligence Safety (Safe AI’20).Google Scholar
- Sina Mohseni, Eric Ragan, and Xia Hu. 2019. Open issues in combating fake news: Interpretability as an opportunity. arxiv:1904.03016. https://arxiv.org/abs/1904.03016.Google Scholar
- Sina Mohseni and Eric D. Ragan. 2018. A human-grounded evaluation benchmark for local explanations of machine learning. arxiv:1801.05075. https://arxiv.org/abs/1801.05075.Google Scholar
- Sina Mohseni, Fan Yang, Shiva Pentyala, Mengnan Du, Yi Liu, Nic Lupfer, Xia Hu, Shuiwang Ji, and Eric Ragan. 2020. Trust evolution over time in explainable AI for fake news detection. Fair & Responsible AI Workshop at CHI 2020.Google Scholar
- Christoph Molnar. 2019. Interpretable Machine Learning. Lulu.com.Google Scholar
- Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. 2017. Methods for interpreting and understanding deep neural networks. Digital Signal Processing 73 (Feb. 2018), 1–15.Google Scholar
- Shane T. Mueller and Gary Klein. 2011. Improving users’ mental models of intelligent software tools. IEEE Intelligent Systems 26, 2 (2011), 77–83.Google ScholarDigital Library
- Bonnie M. Muir. 1987. Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies 27, 5-6 (1987), 527–539.Google ScholarDigital Library
- Tamara Munzner. 2009. A nested process model for visualization design and validation. IEEE Transactions on Visualization and Computer Graphics6 (2009), 921–928.Google Scholar
- Brad A. Myers, David A. Weitzman, Andrew J. Ko, and Duen H. Chau. 2006. Answering why and why not questions in user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 397–406.Google Scholar
- Andrew P. Norton and Yanjun Qi. 2017. Adversarial-playground: A visualization suite showing how adversarial examples fool deep learning. In 2017 IEEE Symposium on Visualization for Cyber Security (VizSec’17). IEEE, 1–4.Google Scholar
- Florian Nothdurft, Felix Richter, and Wolfgang Minker. 2014. Probabilistic human-computer trust handling. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL’14). 51–59.Google ScholarCross Ref
- Mahsan Nourani, Dondald Honeycutt, Jeremy Block, Chiradeep Roy, Tahrima Rahman, Eric D. Ragan, and Vibhav Gogate. 2020. Investigating the importance of first impressions and explainable AI with interactive video analysis. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM.Google Scholar
- Mahsan Nourani, Samia Kabir, Sina Mohseni, and Eric D. Ragan. 2019. The effects of meaningful and meaningless explanations on trust and perceived system accuracy in intelligent systems. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 97–105.Google Scholar
- Besmira Nushi, Ece Kamar, and Eric Horvitz. 2018. Towards accountable AI: Hybrid human-machine analyses for characterizing system failure. In 6th AAAI Conference on Human Computation and Crowdsourcing.Google Scholar
- Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev. 2018. The building blocks of interpretability. Distill (2018). DOI:https://doi.org/10.23915/distill.00010 https://distill.pub/2018/building-blocks.Google Scholar
- Cathy O’Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books.Google ScholarDigital Library
- Sean Penney, Jonathan Dodge, Claudia Hilderbrand, Andrew Anderson, Logan Simpson, and Margaret Burnett. 2018. Toward foraging for understanding of StarCraft agents: An empirical study. In 23rd International Conference on Intelligent User Interfaces (IUI’18). ACM, New York, NY, 225–237. DOI:https://doi.org/10.1145/3172944.3172946Google ScholarDigital Library
- Nicola Pezzotti, Thomas Höllt, Jan Van Gemert, Boudewijn P. F. Lelieveldt, Elmar Eisemann, and Anna Vilanova. 2018. DeepEyes: Progressive visual analytics for designing deep neural networks. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 98–108.Google ScholarCross Ref
- Nina Poerner, Hinrich Schütze, and Benjamin Roth. 2018. Evaluating neural network explanation methods using hybrid documents and morphological prediction. In 56th Annual Meeting of the Association for Computational Linguistics (ACL’18).Google Scholar
- Brett Poulin, Roman Eisner, Duane Szafron, Paul Lu, Russell Greiner, David S. Wishart, Alona Fyshe, Brandon Pearcy, Cam MacDonell, and John Anvik. 2006. Visual explanation of evidence with additive classifiers. In Proceedings of the 18th Conference on Innovative Applications of Artificial Intelligence, Vol. 2. AAAI Press, 1822–1829.Google Scholar
- Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arxiv:1802.07810. https://arxiv.org/abs/1802.07810.Google Scholar
- Pearl Pu and Li Chen. 2006. Trust building with explanation interfaces. In Proceedings of the 11th International Conference on Intelligent User Interfaces. ACM, 93–100.Google ScholarDigital Library
- Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations as mechanisms for supporting algorithmic transparency. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 103.Google ScholarDigital Library
- Emilee Rader and Rebecca Gray. 2015. Understanding user beliefs about algorithmic curation in the Facebook news feed. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 173–182.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1135–1144.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- Caleb Robinson, Fred Hohman, and Bistra Dilkina. 2017. A deep learning approach for population estimation from satellite imagery. In Proceedings of the 1st ACM SIGSPATIAL Workshop on Geospatial Humanities. ACM, 47–54.Google ScholarDigital Library
- Marko Robnik-Šikonja and Marko Bohanec. 2018. Perturbation-based explanations of prediction models. In Human and Machine Learning. Springer, 159–175.Google Scholar
- Stephanie Rosenthal, Sai P. Selvaraj, and Manuela Veloso. 2016. Verbalization: Narration of autonomous robot experience. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 862–868.Google Scholar
- Andrew Slavin Ross and Finale Doshi-Velez. 2018. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In 32nd AAAI Conference on Artificial Intelligence.Google Scholar
- Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. 2017. Right for the right reasons: Training differentiable models by constraining their explanations. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI’17). 2662–2670. DOI:https://doi.org/10.24963/ijcai.2017/371Google ScholarCross Ref
- Stephen Rudolph, Anya Savikhin, and David S. Ebert. 2009. FinVis: Applied visual analytics for personal financial planning. In IEEE Symposium on Visual Analytics Science and Technology, 2009. Citeseer, 195–202.Google Scholar
- Dominik Sacha, Michael Sedlmair, Leishi Zhang, John Aldo Lee, Daniel Weiskopf, Stephen North, and Daniel Keim. 2016. Human-centered machine learning through interactive visualization. In 24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. 641–646.Google Scholar
- Dominik Sacha, Hansi Senaratne, Bum Chul Kwon, Geoffrey Ellis, and Daniel A. Keim. 2016. The role of uncertainty, awareness, and trust in visual analytics. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2016), 240–249.Google ScholarDigital Library
- Bahador Saket, Arjun Srinivasan, Eric D. Ragan, and Alex Endert. 2017. Evaluating interactive graphical encodings for data visualization. IEEE Transactions on Visualization and Computer Graphics 24, 3 (2017), 1316–1330.Google ScholarCross Ref
- Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. 2017. Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems 28, 11 (2017), 2660–2673.Google ScholarCross Ref
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns Into Productive Inquiry (2014), 1–23.Google Scholar
- Martin Schaffernicht and Stefan N. Groesser. 2011. A comprehensive method for comparing mental models of dynamic systems. European Journal of Operational Research 210, 1 (2011), 57–67.Google ScholarCross Ref
- Ute Schmid, Christina Zeller, Tarek Besold, Alireza Tamaddoni-Nezhad, and Stephen Muggleton. 2016. How does predicate invention affect human comprehensibility?. In International Conference on Inductive Logic Programming. Springer, 52–67.Google Scholar
- Philipp Schmidt and Felix Biessmann. 2019. Quantifying interpretability and trust in machine learning systems. arxiv:1901.08558. https://arxiv.org/abs/1901.08558.Google Scholar
- Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618–626.Google ScholarCross Ref
- Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 3145–3153.Google ScholarDigital Library
- Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arxiv:1312.6034. https://arxiv.org/abs/1312.6034.Google Scholar
- Daniel Smilkov, Shan Carter, D. Sculley, Fernanda B. Viégas, and Martin Wattenberg. 2017. Direct-manipulation visualization of deep networks. arxiv:1708.03788. http://arxiv.org/abs/1708.03788.Google Scholar
- Thilo Spinner, Udo Schlegel, Hanna Schäfer, and Mennatallah El-Assady. 2019. explAIner: A visual analytics framework for interactive and explainable machine learning. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 1064–1074.Google Scholar
- Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, and Alexander M. Rush. 2018. LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 667–676.Google ScholarCross Ref
- Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker. 2009. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies 67, 8 (2009), 639–662.Google ScholarDigital Library
- Simone Stumpf, Simonas Skrebe, Graeme Aymer, and Julie Hobson. 2018. Explaining smart heating systems to discourage fiddling with optimized behavior.Google Scholar
- Latanya Sweeney. 2013. Discrimination in online ad delivery. Communications of the ACM 56, 5 (2013), 44–54.Google ScholarDigital Library
- Jiliang Tang, Huiji Gao, Huan Liu, and Atish Das Sarma. 2012. eTrust: Understanding trust evolution in an online world. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 253–261.Google ScholarDigital Library
- Christina Tikkinen-Piri, Anna Rohunen, and Jouni Markkula. 2018. EU general data protection regulation: Changes and implications for personal data collecting companies. Computer Law & Security Review 34, 1 (2018), 134–153.Google ScholarCross Ref
- Nava Tintarev and Judith Masthoff. 2011. Designing and evaluating explanations for recommender systems. In Recommender Systems Handbook. Springer, 479–510.Google Scholar
- Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arxiv:1806.07552. https://arxiv.org/abs/1806.07552.Google Scholar
- Matteo Turilli and Luciano Floridi. 2009. The ethics of information transparency. Ethics and Information Technology 11, 2 (2009), 105–112.Google ScholarCross Ref
- Jo Vermeulen, Geert Vanderhulst, Kris Luyten, and Karin Coninx. 2010. PervasiveCrystal: Asking and answering why and why not questions about pervasive computing applications. In 2010 Sixth International Conference on Intelligent Environments (IE’10). IEEE, 271–276.Google ScholarDigital Library
- Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology 31 (2017), 841.Google Scholar
- Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). ACM, New York, NY, Article 601, 15 pages. Google Scholar
- Fulton Wang and Cynthia Rudin. 2015. Falling rule lists. In Artificial Intelligence and Statistics. 1013–1022.Google Scholar
- Qianwen Wang, Jun Yuan, Shuxin Chen, Hang Su, Huamin Qu, and Shixia Liu. 2019. Visual genealogy of deep neural networks. IEEE Transactions on Visualization and Computer Graphics 26, 11 (2020), 3340–3352.Google ScholarCross Ref
- Daniel S. Weld and Gagan Bansal. 2019. The challenge of crafting intelligible intelligence. Communications of the ACM 62, 6 (May 2019), 70–79. Google ScholarDigital Library
- Adrian Weller. 2017. Challenges for transparency. arxiv:1708.01870. https://arxiv.org/abs/1708.01870.Google Scholar
- Gesa Wiegand, Matthias Schmidmaier, Thomas Weber, Yuanting Liu, and Heinrich Hussmann. 2019. I drive-you trust: Explaining driving behavior of autonomous cars. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW0163.Google ScholarDigital Library
- James A. Wise, James J. Thomas, Kelly Pennock, David Lantrip, Marc Pottier, Anne Schur, and Vern Crow. 1995. Visualizing the non-visual: Spatial analysis and interaction with information from text documents. In Proceedings of Information Visualization, 1995. IEEE, 51–58.Google ScholarCross Ref
- Kanit Wongsuphasawat, Daniel Smilkov, James Wexler, Jimbo Wilson, Dandelion Mane, Doug Fritz, Dilip Krishnan, Fernanda B. Viégas, and Martin Wattenberg. 2017. Visualizing dataflow graphs of deep learning models in tensorflow. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 1–12.Google ScholarCross Ref
- Samuel C. Woolley. 2016. Automating power: Social bot interference in global politics. First Monday 21, 4 (2016).Google Scholar
- Mike Wu, Michael C. Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, and Finale Doshi-Velez. 2018. Beyond sparsity: Tree regularization of deep models for interpretability. In 32nd AAAI Conference on Artificial Intelligence.Google Scholar
- Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. In ICML Deep Learning Workshop 2015.Google Scholar
- Rulei Yu and Lei Shi. 2018. A user-based taxonomy for deep learning visualization. Visual Informatics 2, 3 (2018), 147–154.Google ScholarCross Ref
- Tom Zahavy, Nir Ben-Zrihem, and Shie Mannor. 2016. Graying the black box: Understanding DQNs. In International Conference on Machine Learning. 1899–1908.Google Scholar
- Tal Zarsky. 2016. The trouble with algorithmic decisions: An analytic road map to examine efficiency and fairness in automated and opaque decision making. Science, Technology, & Human Values 41, 1 (2016), 118–132.Google ScholarCross Ref
- Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European Conference on Computer Vision. Springer, 818–833.Google Scholar
- Quanshi Zhang, Wenguan Wang, and Song-Chun Zhu. 2018. Examining CNN representations with respect to dataset bias. In 32nd AAAI Conference on Artificial Intelligence.Google Scholar
- Quan-shi Zhang and Song-Chun Zhu. 2018. Visual interpretability for deep learning: A survey. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 27–39.Google ScholarCross Ref
- Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT*’20).Google Scholar
- Zijian Zhang, Jaspreet Singh, Ujwal Gadiraju, and Avishek Anand. 2019. Dissonance between human and machine understanding. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 56.Google ScholarDigital Library
- Wen Zhong, Cong Xie, Yuan Zhong, Yang Wang, Wei Xu, Shenghui Cheng, and Klaus Mueller. 2017. Evolutionary visual analysis of deep neural networks. In ICML Workshop on Visualization for Deep Learning.Google Scholar
- Jichen Zhu, Antonios Liapis, Sebastian Risi, Rafael Bidarra, and G. Michael Youngblood. 2018. Explainable AI for designers: A human-centered perspective on mixed-initiative co-creation. In 2018 IEEE Conference on Computational Intelligence and Games (CIG’18). IEEE, 1–8.Google Scholar
- Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling. 2017. Visualizing deep neural network decisions: Prediction difference analysis. arxiv:1702.04595. http://arxiv.org/abs/1702.04595.Google Scholar
Index Terms
- A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Recommendations
Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsBiases in Artificial Intelligence (AI) systems or their results are one important issue that demands AI explainability. Despite the prevalence of AI applications, the general public are not necessarily equipped with the ability to understand how the ...
DARPA's explainable artificial intelligence (XAI) program
IUI '19: Proceedings of the 24th International Conference on Intelligent User InterfacesThe DARPA's Explainable Artificial Intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. This talk will summarize the XAI program and present highlights ...
Comments