Abstract
Current research in Explainable AI includes post-hoc explanation methods that focus on building transparent explaining agents able to emulate opaque ones. Such agents are naturally required to be accurate and trustworthy. However, what it means for an explaining agent to be accurate and trustworthy is far from being clear. We characterize accuracy and trustworthiness as measures of the distance between the formal properties of a given opaque system and those of its transparent explanantes. To this aim, we extend Probabilistic Computation Tree Logic with operators to specify degrees of accuracy and trustworthiness of explaining agents. We also provide a semantics for this logic, based on a multi-agent structure and relative model-checking algorithms. The paper concludes with a simple example of a possible application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
As in standard PCTL, the CTL existential and universal quantifiers, expressing quantification over paths satisfying a given formula \(\psi \), here are omitted. It is easy to prove that they correspond to special cases of probabilistic quantification. In particular, \(\exists \psi \iff P_{>0}\psi \) and \(\forall \psi \iff P_{=1}\psi \). For the details, see [1].
- 3.
Notice that path-formulas \(\psi \) are usually not considered in a typical probabilistic model-checking workflow. For the details of the procedure, see [1].
- 4.
Notice that, this must not be intended as a conditional probability.
- 5.
Here, minimality is defined as for Eq. (2).
- 6.
This represents a typical example of a stochastic machine learning model. According to the classification we propose in the introduction, it can be classified as an opaque but comprehensible model. For more details, see [2].
- 7.
Remeber that an explanans is an agent able to (locally) emulate the behaviour of the target-system and usually consider more transparent than this one.
- 8.
A Python implementation of Algorithm 1 is available at https://github.com/dasaro/ATCTL together with details on how to reproduce the results from the example.
References
Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)
Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, 5th edn. Springer, New York (2007). https://www.worldcat.org/oclc/71008143
Chen, T., Primiero, G., Raimondi, F., Rungta, N.: A computationally grounded, weighted doxastic logic. Studia Logica 104(4), 679–703 (2016)
D’Asaro, F.A., Primiero, G.: Probabilistic typed natural deduction for trustworthy computations. In: Proceedings of the 22nd International Workshop on Trust in Agent Societies (TRUST2021 @ AAMAS) (2021)
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 93:1–93:42 (2019). https://doi.org/10.1145/3236009
Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects Comput. 6(5), 512–535 (1994)
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy 23(1), 18 (2021). https://doi.org/10.3390/e23010018
Lomuscio, A., Raimondi, F.: mcmas: a model checker for multi-agent systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 450–454. Springer, Heidelberg (2006). https://doi.org/10.1007/11691372_31
Revuz, D.: Markov Chains. Elsevier, New York (2008)
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., Zhong, C.: Interpretable machine learning: Fundamental principles and 10 grand challenges. CoRR abs/2103.11251 (2021). https://arxiv.org/abs/2103.11251
Acknowledgments
This research has been funded by the Department of Philosophy “Piero Martinetti” of the University of Milan under the Project “Departments of Excellence 2018–2022” awarded by the Ministry of Education, University and Research (MIUR). The authors also thankfully acknowledge the support of the Italian Ministry of University and Research (PRIN 2017 project n. 20173YP4N3).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Termine, A., Primiero, G., D’Asaro, F.A. (2021). Modelling Accuracy and Trustworthiness of Explaining Agents. In: Ghosh, S., Icard, T. (eds) Logic, Rationality, and Interaction. LORI 2021. Lecture Notes in Computer Science(), vol 13039. Springer, Cham. https://doi.org/10.1007/978-3-030-88708-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-88708-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88707-0
Online ISBN: 978-3-030-88708-7
eBook Packages: Religion and PhilosophyPhilosophy and Religion (R0)
