Ablation Path Saliency

Abstract

Various types of saliency methods have been proposed for explaining black-box classification. In image applications, this means highlighting the part of the image that is most relevant for the current decision.

Unfortunately, the different methods may disagree and it can be hard to quantify how representative and faithful the explanation really is. We observe however that several of these methods can be seen as edge cases of a single, more general procedure based on finding a particular path through the classifier’s domain. This offers additional geometric interpretation to the existing methods.

We demonstrate furthermore that ablation paths can be directly used as a technique of its own right. This is able to compete with literature methods on existing benchmarks, while giving more fine-grained information and better opportunities for validation of the explanations’ faithfulness.

Appendices

Appendix A Canonical Time Reparametrisation

The function $m :[0,1] \rightarrow \mathbb {R}$ defined by $m(t) {:}{=}\int _{\varOmega } \varphi (t)$ is increasing and goes from zero to one (since we assume that $\int _{\varOmega } 1 = 1$).

Note first that if $m(t_1) = m(t_2)$, then $\varphi (t_1) = \varphi (t_2)$ from the monotonicity property. Indeed, supposing for instance that $t_1 \le t_2$, and defining the element $\theta {:}{=}\varphi (t_2) - \varphi (t_1)$ we see that on the one hand $\int _{\varOmega } \theta = 0$, on the other hand, $\theta \ge 0$, so $\theta = 0$ and thus $\varphi (t_1) = \varphi (t_2)$.

Now, define . Pick $s \in [0,1]$.

If $s \in \textsf{M}$ we define $\psi (s) {:}{=}\varphi (t)$ where $m(t) = s$ (and this does not depend on which $t$ fulfills $m(t) = s$ from what we said above). We remark that $\int _{\varOmega }\psi (s) = \int _{\varOmega }\varphi (t) = m(t) = s$.

Now suppose that $s \not \in \textsf{M}$. Define $s_1 {:}{=}\sup (\textsf{M}\cap [0,s])$ and $s_2 {:}{=}\inf (\textsf{M}\cap [s,1])$ (neither set are empty since $0\in \textsf{M}$ and $1 \in \textsf{M}$). Since $s_1\in \textsf{M}$ and $s_2\in \textsf{M}$, there are $t_1\in [0,1]$ and $t_2\in [0,1]$ such that $m(t_1) = s_1$ and $m(t_2) = s_2$. Finally define $\psi (s) {:}{=}\varphi (t_1) + (s - s_1)\frac{\varphi (t_2) - \varphi (t_1)}{s_2 - s_1}$. In this case, $\int _{\varOmega }\psi (s) = m(t_1) + (s-s_1) \frac{m(t_2) - m(t_1)}{s_2-s_1} = s$. The path $\psi$ constructed this way is still monotone, and it has the constant speed property, so it is an ablation path.

Appendix B $\mathcal {L}^{\infty }$-Optimal Monotonicity Projection

The algorithm proposed in Appendix C for optimising monotone paths uses updates that can locally introduce nonmonotonicity in the candidate $\hat{\varphi }_1$, so that it is necessary to project back onto a monotone path $\varphi _1$. The following routine⁸ performs such a projection in a way that is optimal in the sense of minimising the $\mathcal {L}^{\infty }$-distance⁹, i.e.,

$$ \sup _{t}\bigl |\varphi _1(t,\textbf{r}) - \hat{\varphi }_1(t,\textbf{r})\bigr | \le \sup _{t}\bigl |\vartheta (t,\textbf{r}) - \hat{\varphi }_1(t,\textbf{r})\bigr | $$

for all $\textbf{r}\in \varOmega$ and any other monotone path $\vartheta$.

The algorithm works separately for each $\textbf{r}$, i.e., we express it as operating simply on continuous functions $p: [0,1]\rightarrow \mathbb {R}$.

Algorithm 1

. Make a function $[0,1] \rightarrow \mathbb {R}$ nondecreasing

The final step effectively flattens out, in a minimal way, any region in which the function was decreasing.

In practice, this algorithm is executed not on continuous functions but on a PCM-discretised representation; this changes nothing about the algorithm except that instead as real numbers, $l,r$ and $t$ are represented by integral indices.

Appendix C Path Optimisation Algorithm

As said in Sect. 5, our optimisation algorithm is essentially gradient descent of a path $\varphi$: it repeatedly seeks the direction within the space of all paths that (first ignoring the monotonicity constraint) would affect the largest increase to $P(\varphi )$ as per Sect. 4, for any of the defined score functions. Algorithm 2 shows the details of how this is done in presence of our constraints. In case of ${P_{\uparrow \downarrow }}$, the state $\varphi$ is understood to consist of the two paths $\varphi _\uparrow$ and $\varphi _\downarrow$.

As discussed before, the use of a gradient requires a metric to obtain a vector from the covector-differential, which could be either the implicit $\ell ^2$ metric on the discretised representation (pixels), or a more physical kernel/filter-based metric. In the present work, we base this on the regularisation filter.

Fig. 6.

Example view of the monotonisation algorithm in practice. (a) contains decreasing intervals, which have been localised in (b). For each interval, the centerline is then extended to meet the original path non-decreasingly (c). In some cases, this will cause intervals overlapping; in this case merge them to a single interval and re-grow from the corresponding centerline (d). Finally, replace the path on the intervals with their centerline (e).

Unlike with the monotonisation condition, the update can easily be made to preserve speed-constness by construction, by projecting for each $t$ the gradient $\textbf{g}$ on the sub-tangent-space of zero change to $\int _\varOmega \varphi (t)$, by subtracting the constant function times $\int _\varOmega \textbf{g}(t)$. Note this requires the measure of $\varOmega$ to be normalised, or else considered at this point.

Then we apply these gradients time-wise as updates to the path, using a scalar product in the channel-space to obtain the best direction for $\varphi$ itself (as opposed to the corresponding image composite $x_{\varphi ,t}$).

The learning rate $\gamma$ can be chosen in different ways. What we found to work best is to normalise the step size in a $\mathcal {L}^{\infty }$ sense, such that the strongest-affected pixel in the mask experiences a change of at most 0.7 per step. This is small enough to avoid excessively violating the constraint, but not so small to make the algorithm unnecessarily slow (Fig. 6).

Appendix D Baseline Choice

The baseline image is prominently present in the input for much of the ablation path, and it is therefore evident that it will have a significant impact on the saliency. In line with previous work, we opted for a blurred baseline for the examples in the main paper, but even then there is still considerable freedom in the choice of blurring filter. Figure 7 shows two examples, where the result is not fundamentally, but still notably different.

Algorithm 2

. Projected Gradient Descent

Fig. 7.

An example of paths obtained with different-size blur baselines.