Emanuele Albini
ABSTRACT
Feature attributions are a common paradigm for model explanations due to their simplicity in assigning a single numeric score for each input feature to a model. In the actionable recourse setting, wherein the goal of the explanations is to improve outcomes for model consumers, it is often unclear how feature attributions should be correctly used. With this work, we aim to strengthen and clarify the link between actionable recourse and feature attributions. Concretely, we propose a variant of SHAP, Counterfactual SHAP (CF-SHAP), that incorporates counterfactual information to produce a background dataset for use within the marginal (a.k.a. interventional) Shapley value framework. We motivate the need within the actionable recourse setting for careful consideration of background datasets when using Shapley values for feature attributions with numerous synthetic examples. Moreover, we demonstrate the efficacy of CF-SHAP by proposing and justifying a quantitative score for feature attributions, counterfactual-ability, showing that as measured by this metric, CF-SHAP is superior to existing methods when evaluated on public datasets using tree ensembles.
- Kjersti Aas, Martin Jullum, and Anders Løland. 2021. Explaining Individual Predictions When Features Are Dependent: More Accurate Approximations to Shapley Values. Artificial Intelligence 298 (2021), 103502.Google Scholar
Digital Library
- Amina Adadi and Mohammed Berrada. 2018. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (9 2018), 52138–52160.Google Scholar
- Emanuele Albini, Antonio Rago, Pietro Baroni, and Francesca Toni. 2020. Relation-Based Counterfactual Explanations for Bayesian Network Classifiers. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI. 451–457.Google ScholarCross Ref
- Emanuele Albini, Antonio Rago, Pietro Baroni, and Francesca Toni. 2021. Influence-Driven Explanations for Bayesian Network Classifiers. In PRICAI 2021: Trends in Artificial Intelligence. 88–100.Google Scholar
- Solon Barocas, Andrew D Selbst, and Manish Raghavan. 2020. The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAccT. 80–89.Google ScholarDigital Library
- Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58(2020), 82–115.Google ScholarDigital Library
- James Bergstra, Daniel Yamins, and David Cox. 2013. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. In Proceedings of the 30th International Conference on International Conference on Machine Learning, ICML. I–115–I–123.Google Scholar
- Matt Chapman-Rounds, Umang Bhatt, Erik Pazos, Marc-Andre Schulz, and Konstantinos Georgatzis. 2021. FIMAP: Feature Importance by Minimal Adversarial Perturbation. Proceedings of the 35th AAAI Conference on Artificial Intelligence 35, 13 (5 2021), 11433–11441.Google ScholarCross Ref
- Hugh Chen, Joseph D Janizek, Scott Lundberg, and Su-In Lee. 2020. True to the Model or True to the Data?. In ICML ’20 Workshop on Human Interpretability. arxiv:2006.16234Google Scholar
- Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD. 785–794.Google ScholarDigital Library
- Paulo Cortez, António Cerdeira, Fernando Almeida, Telmo Matos, and José Reis. 2009. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems 47, 4 (11 2009), 547–553.Google Scholar
- Kristijonas Čyras, Antonio Rago, Emanuele Albini, Pietro Baroni, and Francesca Toni. 2021. Argumentative XAI: A Survey. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI, Vol. 5. 4392–4399.Google Scholar
- Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. 2020. Multi-objective counterfactual explanations. In Proceedings of the 16th International Conference on Parallel Problem Solving from Nature, Vol. 12269 LNCS. 448–469.Google ScholarDigital Library
- High-Level Expert Group on Artificial Intelligence European Commission. 2019. Ethics Guidelines for Trustworthy AI. Technical Report.Google Scholar
- Xiaoli Fern and Quintin Pope. 2021. Text Counterfactuals via Latent Optimization and Shapley-Guided Search. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 5578–5593.Google ScholarCross Ref
- Carlos Fernández-Loría, Foster Provost, and Xintian Han. 2021. Explaining Data-Driven Decisions Explaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach. arxiv:2001.07417Google Scholar
- FICO Community. 2019. Explainable Machine Learning Challenge. https://community.fico.com/s/explainable-machine-learning-challengeGoogle Scholar
- Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. 2021. Shapley Explainability on the Data Manifold. In Proceedings of the 9th International Conference on Learning Representations (ICLR). 14.Google Scholar
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. 2018. Local Rule-Based Explanations of Black Box Decision Systems. arxiv:1805.10820Google Scholar
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A Survey of Methods for Explaining Black Box Models. Comput. Surveys 51, 5 (2019), 1–42.Google ScholarDigital Library
- Masoud Hashemi and Ali Fathi. 2020. PermuteAttack: Counterfactual Explanation of Machine Learning Credit Scorecards. arxiv:2008.10138Google Scholar
- GitHub Issues. 2018. Interpretation of Kernel SHAP and Its Hyperparameters - Issue #23 https://github.com/slundberg/shap.Google Scholar
- GitHub Issues. 2019. Choosing the Background Set · Issue #391 · https://github.com/slundberg/shap.Google Scholar
- GitHub Issues. 2019. Interpretation of SHAP Values Away from the Mean · Issue #435 · https://github.com/slundberg/shap.Google Scholar
- GitHub Issues. 2019. ZestFinance Writeup on SHAP and Why It Shouldn’t Be Used on Its Own · Issue #624 · https://github.com/slundberg/shap.Google Scholar
- Dominik Janzing, Lenon Minorics, and Patrick Bloebaum. 2020. Feature Relevance Quantification in Explainable AI: A Causal Problem. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics. 2907–2916.Google Scholar
- Kaggle. 2019. Lending Club Loan Data. https://www.kaggle.com/wordsforthewise/lending-clubGoogle Scholar
- Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Hiroki Arimura. 2020. DACE: Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI. 2855–2862.Google ScholarCross Ref
- Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. 2020. Model-Agnostic Counterfactual Explanations for Consequential Decisions. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, AISTATS. 895–905.Google Scholar
- Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. 2020. Algorithmic Recourse under Imperfect Causal Knowledge: A Probabilistic Approach. In Proceedings of the 34th Conference on Neural Information Processing Systems, NeurIPS.Google ScholarDigital Library
- Amir-Hossein Karimi, Eth Zürich, Switzerland Bernhard Schölkopf, and Isabel Valera. 2021. Algorithmic Recourse: from Counterfactual Explanations to Interventions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT. 353––362.Google ScholarDigital Library
- Mark T Keane, Eoin M Kenny, Eoin Delaney, and Barry Smyth. 2021. If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques. In Proceeding of the 30th International Joint Conference on Artificial Intelligence, IJCAI. 4466–4474.Google ScholarCross Ref
- I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle A Friedler. 2020. Problems with Shapley-value-based explanations as feature importance measures. In Proceedings of the 37th International Conference on Machine Learning, ICML. 5491–5500.Google Scholar
- Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2019. Faithful and Customizable Explanations of Black Box Models. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, FAccT. 131–138.Google ScholarDigital Library
- Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2019. The Dangers of Post-Hoc Interpretability: Unjustified Counterfactual Explanations. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI. 2801–2807.Google ScholarCross Ref
- Scott Lundberg. 2017. Supplementary Material to a Unified Approach to Interpreting Model Predictions: The Monotonicity Axiom Implies the Symmetry Axiom for Shapley Values.Google Scholar
- Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence 2, 1 (1 2020), 56–67.Google Scholar
- Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems, NeurIPS. 4768––4777.Google Scholar
- Luke Merrick and Ankur Taly. 2020. The Explanation Game: Explaining Machine Learning Models Using Shapley Values. In International Cross-Domain Conference for Machine Learning and Knowledge Extraction, CD-MAKE. 17–38.Google Scholar
- John W L Merrill, Geoff M Ward, Sean J Kamkar, Jay Budzik, and Douglas C Merrill. 2019. Generalized Integrated Gradients: A practical method for explaining diverse ensembles. Journal of Machine Learning Research - Under Review (2019). arxiv:1909.01869Google Scholar
- Tim Miller. 2019. Explanation in Artificial Intelligence: Insights from the Social Sciences. Artificial Intelligence 267 (6 2019), 1–38.Google Scholar
- R. K. Mothilal, Divyat Mahajan, Chenhao Tan, and Amit Sharma. 2021. Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES. 652–663.Google ScholarDigital Library
- Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. 2020. Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAccT. 607–617.Google ScholarDigital Library
- Martin Pawelczyk, Sascha Bielawski, Johannes van den Heuvel, Tobias Richter, and Gjergji Kasneci. 2021. CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms. In Benchmark & Data Sets Track at the 36th Conference on Neural Information Processing Systems, NeurIPS.Google Scholar
- Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. 2020. Learning Model-Agnostic Counterfactual Explanations for Tabular Data. In Proceedings of The Web Conference 2020, WWW. 3126––3132.Google ScholarDigital Library
- Gregory Plumb, Denali Molitor, and Ameet S Talwalkar. 2018. Model Agnostic Supervised Local Explanations. In Advances in Neural Information Processing Systems, NeurIPS. 2520––2529.Google Scholar
- Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. FACE: Feasible and Actionable Counterfactual Explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES. 344–350.Google ScholarDigital Library
- Yanou Ramon, David Martens, Foster Provost, and Theodoros Evgeniou. 2020. A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: SEDC, LIME-C and SHAP-C. Advances in Data Analysis and Classification 14 (2020), 801–819.Google ScholarDigital Library
- Shubham Rathi. 2019. Generating Counterfactual and Contrastive Explanations using SHAP. In 2nd Workshop on Humanizing AI (HAI) at IJCAI ’19. arxiv:1906.09293Google Scholar
- Kaivalya Rawal and Himabindu Lakkaraju. 2020. Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses. In Advances in Neural Information Processing Systems, NeurIPS. 12187–12198.Google Scholar
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ”Why Should I Trust You?”. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD. 1135–1144.Google ScholarDigital Library
- Alvin E. Roth. 1988. The Shapley Value: Essays in Honor of Lloyd S. Shapley.Google Scholar
- Chris Russell. 2019. Efficient Search for Diverse Coherent Explanations. In Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency, FAccT. 20–28.Google ScholarDigital Library
- Lloyd Stowell Shapley. 1951. Notes on the n-Person Game-II: The Value of an n-Person Game.Google Scholar
- Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2020. CERTIFAI: A Common Framework to Provide Explanations and Analyse the Fairness and Robustness of Black-box Models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES. 166–172.Google ScholarDigital Library
- Ravid Shwartz-Ziv and Amitai Armon. 2021. Tabular Data: Deep Learning Is Not All You Need. arxiv:2106.03253Google Scholar
- Barry Smyth and Mark T. Keane. 2021. A Few Good Counterfactuals: Generating Interpretable, Plausible and Diverse Counterfactual Explanations. arxiv:2101.09056Google Scholar
- Thomas Spooner, Danial Dervovic, Jason Long, Jon Shepard, Jiahao Chen, and Daniele Magazzeni. 2021. Counterfactual Explanations for Arbitrary Regression Models. In ICML’21 Workshop on Algorithmic Recourse. arxiv:2106.15212Google Scholar
- Ilia Stepin, Jose M. Alonso, Alejandro Catala, and Martin Pereira-Farina. 2021. A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence. IEEE Access 9(2021), 11974–12001.Google ScholarCross Ref
- Erik Strumbelj and Igor Kononenko. 2010. An Efficient Explanation of Individual Classifications using Game Theory. Journal of Machine Learning Research 11 (2010), 1–18.Google ScholarDigital Library
- Agus Sudjianto and Scott Zoldi. 2021. The Case for Interpretable Models in Credit Underwriting. https://soundcloud.com/finreglab/agus-sudjiantoscott-zoldi-the-case-for-interpretable-models-in-credit-underwritingGoogle Scholar
- Mukund Sundararajan and Amir Najmi. 2020. The Many Shapley Values for Model Explanation. In Proceedings of the 37th International Conference on Machine Learning. 9269–9278.Google Scholar
- U.S. Congress. 2018. 12 CFR Part 1002 - Equal Credit Opportunity Act (Regulation B). https://www.consumerfinance.gov/rules-policy/regulations/1002/9/Google Scholar
- Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable Recourse in Linear Classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAccT. 10–19.Google ScholarDigital Library
- Sahil Verma, Arthur Ai, John Dickerson, and Keegan Hines. 2020. Counterfactual Explanations for Machine Learning: A Review. arxiv:2010.10596Google Scholar
- Julius von Kügelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard Schölkopf. 2022. On the Fairness of Causal Algorithmic Recourse. In Proceedings of the 36th AAAI Conference on Artificial Intelligence.Google Scholar
- Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2018. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology 31 (2018), 1–52.Google Scholar
- Jiaxuan Wang, Jenna Wiens, and Scott Lundberg. 2021. Shapley Flow: A Graph-based Approach to Interpreting Model Predictions. In Proocedings of the 24th International Conference on Artificial Intelligence and Statistics, AISTATS.Google Scholar
- David S. Watson. 2022. Rational Shapley Values. In Proceedings of the 2022 Conference on Fairness, Accountability, and Transparency, FAccT.Google ScholarDigital Library
- Adam White and Artur d’Avila Garcez. 2019. Measurable Counterfactual Local Explanations for Any Classifier. In Proceedings of the 24th European Conference on Artificial Intelligence, ECAI. 2529–2535.Google Scholar
Index Terms
- Counterfactual Shapley Additive Explanations
Index terms have been assigned to the content through auto-classification.
Recommendations
On the Connection between Game-Theoretic Feature Attributions and Counterfactual Explanations
AIES '23: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and SocietyExplainable Artificial Intelligence (XAI) has received widespread interest in recent years, and two of the most popular types of explanations are feature attributions, and counterfactual explanations. These classes of approaches have been largely ...
Explaining anomalies detected by autoencoders using Shapley Additive Explanations▪
AbstractDeep learning algorithms for anomaly detection, such as autoencoders, point out the outliers, saving experts the time-consuming task of examining normal cases in order to find anomalies. Most outlier detection algorithms output a score for each ...
Highlights- Explaining anomalies identified by autoencoder using shapley values.
- Explain features with high reconstruction error.
- Evaluated correctness and robustness of explanations.
- Explanations can assist in reducing anomaly score.
- ...
Achieving Diversity in Counterfactual Explanations: a Review and Discussion
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and TransparencyIn the field of Explainable Artificial Intelligence (XAI), counterfactual examples explain to a user the predictions of a trained decision model by indicating the modifications to be made to the instance so as to change its associated prediction. These ...
Comments