IFME: Influence Function Based Model Explanation for Black Box Decision Systems

Abstract

Due to the high precision and the huge prediction needs, machine learning models based decision systems has been widely adopted in all works of life. They were usually constructed as a black box based on sophisticated, opaque learning models. The lack of human understandable explanation to the inner logic of these models and the reason behind the predictions from such systems causes a serious trust issue. Interpretable Machine Learning methods can be used to relieve this problem by providing an explanation for the models or predictions. In this work, we focus on the model explanation problem, which study how to explain the black box prediction model globally through human understandable explanation. We propose the Influence Function based Model Explanation (IFME) method to provide interpretable model explanation based on key training points selected through influence function. First, our method introduces a novel local prediction interpreter, which also utilizes the key training points for local prediction. Then it finds the key training points to the learning models via influence function globally. Finally, we provide the influence function based model agnostic explanation to the model used. We also show the efficiency of our method through both theoretical analysis and simulated experiments.