Looking at the posterior: accuracy and uncertainty of neural-network predictions

An artificial neural network f with parameters $\theta \in \mathbb{R}$ can be seen as a map from input space X to the output space Y [29], where we take Y to be the space of distributions over possible outcomes so that the likelihood is given by $p(y| x, \theta) = f(y, x ; \theta)$. Bayesian inference [30] allows us to reason about uncertainty in terms of posterior distributions for the parameters of a model. With a training dataset $\mathcal{D} \in X \times Y$ corresponding to observations $(x_t, y_t)$ with $t\in\{1\ldots N\}$, the posterior distribution for the neural-network parameters $p(\theta | \mathcal{D})$ can be calculated using Bayes theorem in terms of the likelihood as

$$\begin{equation} p\left(\theta | \mathcal{D}\right) = \frac{p\left(\mathcal{D} | \theta\right)p\left(\theta\right)}{p\left(\mathcal{D}\right)}, \end{equation} \tag{ 2 }$$

where $p(\mathcal{D} | \theta)$ (assuming independent samples in the dataset $\mathcal{D}$) can be expressed as $p(\mathcal{D}|\theta) = \prod_{i = 1}^{N}$ $f(y_{i}, x_{i}; \theta)$, $p(\theta)$ the parameter prior and the evidence $p(\mathcal{D})$ is the marginalization over the parameter prior.

A prediction, or more generally a distribution over possible outputs, is computed using the posterior predictive distribution in equation (1) by marginalizing over the parameters. Each parameter configuration is weighted by its posterior probability given the training data. For large neural-network architectures, it is in general very challenging to evaluate the high-dimensional integral over θ [31]. To implement Bayesian inference for neural networks, the posterior needs to be approximated.

There are a number of different methods available that range from computationally expensive Monte-Carlo simulations [10] to more efficient dropout approximations [32, 33], as well as simpler ensembling methods [34]. The most accurate approximations to the true posterior rely on the Hamiltonian Monte-Carlo (HMC) methods [35–37]. Through intensive computational efforts, HMC methods have recently been used to approximate the posteriors of larger convolutional neural networks such as a 20-layer ResNet [31]. The HMC computations show that simpler approximation schemes such as ensembling and variational inference can fail to accurately describe the true posterior, but that ensembles often provide more accurate posteriors than more advanced methods. Here we use deep ensembles, as described in appendix A, to approximate the Bayesian posteriors of the neural networks [38].

For classification over a discrete set of M classes, the entropy [39] of the predictive distribution

$$\begin{equation} H\left(p\right) = -\sum_{c = 1}^{M}p\left( {c} \right)\log \left(p\left( {c} \right)\right), \end{equation} \tag{ 3 }$$

provides a measure of the information content and thus its uncertainty.

In terms of the ensemble approximations of the posterior predictive distribution, see appendix equation (A.1) for details, this takes the form

$$\begin{equation} H\left(p\left(y| x, \mathcal{D}\right)\right) = -\sum_{c = 1}^{M}\left(\frac{1}{N}\sum_{i = 1 }^N f_{c}\left(x; \theta^{\left(i\right)}\right)\right)\log \left(\frac{1}{N}\sum_{i = 1 }^N f_{c}\left(x; \theta^{\left(i\right)}\right)\right), \end{equation} \tag{ 4 }$$

where $H(p(y|x, \mathcal{D}))$ is the entropy of the posterior predictive distribution, referred to here as predictive uncertainty. The entropy of $p(y|x, \mathcal{D})$ in equation (4), as a measure of uncertainty, contains contributions that are both epistemic and aleatoric.

Epistemic uncertainty stems from uncertainty in model parameters. This is captured by the shape of the posterior distribution of these parameters. To quantify the epistemic uncertainty associated with a single data sample x, we can ask how this shape changes when the dataset is extended to include x [11]. The epistemic uncertainty associated with a data sample x can be quantified by the expected change in entropy of the model parameter posterior distribution $p(\theta|\mathcal{D})$ when x is added to the observations [11],

$$\begin{align} I\left(x\right) & = H \left(p\left(\theta | \mathcal{D}\right) \right) - E_{p\left(y | x , \mathcal{D}\right)} \left[H \left(p\left(\theta | \mathcal{D}, \left(x, y\right)\right)\right)\right]. \end{align} \tag{ 5 }$$

The entropy difference in equation (5) is the conditional mutual information $I(\theta; y | x, \mathcal{D})$ between the parameter posterior and the predictive posterior distribution for the new sample point (x, y). Since mutual information is symmetric, this can be written in terms of the entropy of the predictive distribution instead:

$$\begin{align} I\left(x\right) & = H \left(p\left(y | x, \mathcal{D}\right) \right) - E_{p \left(\theta | \mathcal{D} \right)} \left[H \left(p\left(y | \theta, x\right) \right)\right], \end{align} \tag{ 6 }$$

where $H (p(y | \theta, x ))$ is the entropy of the likelihood $p(y | \theta, x)$. See appendix B for an example of how this relation manifests in a toy model. The first term in equation (6) is the entropy of the posterior predictive distribution given the dataset $\mathcal{D}$, whereas the second term is the expected value of the likelihood entropy over the model posterior distribution.

Using the approximate posteriors from the ensemble method, the first term in equation (6) is given by equation (4) and the second term is given by

$$\begin{equation} E_{p\left(\theta | \mathcal{D} \right)} \left[H \left(p\left(y | \theta, x\right) \right)\right] = \frac{1}{N} \sum_{i = 1}^{N}\sum_{c = 1}^{M}f_{c}\left(x; \theta^{\left(i\right)}\right)\log \left(f_{c}\left(x; \theta^{\left(i\right)}\right)\right). \end{equation} \tag{ 7 }$$

Together, equations (4) and (7) provide a concrete way to evaluate epistemic uncertainty as defined by equation (6) in practice.

The epistemic uncertainty in equation (6) is large when the posterior predictive entropy is large and the mean likelihood entropy is small. In terms of the ensemble members this corresponds to the situation where each member has a sharp distribution but they disagree about the mean. Large aleatoric uncertainty is ascribed to broad output distributions from the individual members $f(\cdot; \theta^{(i)})$, also implying a large posterior predictive entropy. Since entropy is positive, both the term $H(p(y|x, \mathcal{D}))$ and the expected entropy of the likelihood in equation (6) are positive, and so the entropy difference in equation (6) is bounded by the posterior entropy in equation (4), resulting in the inequality

$$\begin{equation} I\left(x\right) \unicode{x2A7D} H\left(x\right), \end{equation} \tag{ 8 }$$

i.e. a large epistemic uncertainty implies a large posterior predictive entropy. This can be seen by noting that a collection of sharp member distributions that disagree about the mean necessarily adds up to a broad mean distribution. One way to describe a sample with large epistemic uncertainty is that the model might fit the data well, but in many different ways. The parameters with high posterior probability could all result in sharp distributions, whereas the full posterior can be broad [40].

Since the epistemic uncertainty in equation (6) measures the change in posterior entropy, a model can have irrelevant parameters that contribute to a broad posterior distribution, but the change in posterior entropy can still be small when adding a given sample to the training set. If the parameter posterior for some irrelevant parameter is equally broad after we add sample x, then we do not want to consider this as a point of high epistemic uncertainty.

Even though entropic measures are a natural starting point for quantifying uncertainty of predictions, they have been shown to possess a number of unwanted properties such as ataining maximal values in un-intuitive scenarios [41]. In this work we take a pragmatic view and consider uncertainty quantification based on entropy as a flawed, but still practically useful tool to understand neural network predictions.

Two datasets are used for all numerical experiments, MNIST [24] and a grayscale version of CIFAR10 [25]. We choose these simple datasets to be able to train ensembles efficiently and still be in the domain of realistic images with CIFAR10. To be able to compare relative shifts in uncertainty measures between the two datasets we choose to work with a grayscale version of CIFAR denoted CIFAR10G so that the data-distributional shift acts in exactly the same way for both MNIST and CIFAR10G. MNIST is an example of an image classification dataset with minimal complexity, whereas CIFAR10 provides a more realistic data distribution for image classification. The grayscale conversion for the RGB data from CIFAR10 is given by the standard BT.601 luminance $Y = 0.2989 r + 0.5870 g + 0.1140 b$. We apply impulse noise [23], a common corruption present in digital images, to the original datasets (MNIST, CIFAR10G) where the strength of the perturbation is controlled by a noise parameter α. For 'salted' noise with parameter α > 0 a random sample of pixels of size $\alpha N_{{\operatorname{pixels}}}$ are given the maximum value 1.0, and for 'peppered' noise with α < 0 a corresponding amount of pixels are set to 0. We pick N_α distinct values between $\alpha_{\text{min}} = -0.3$ and $\alpha_{\text{max}} = 0.3$.

MNIST consists of 60 000 grayscale images with resolution $28\,\,{\times}\,\,28$. The original CIFAR10 dataset consists of 60 000 RGB images with resolution $32{\times}32$ that we resample to a single grayscale channel. See figure 1 for examples of the different noise levels.

Zoom In Zoom Out Reset image size

In order to quantify how the architecture affects the uncertainty estimates, we use three neural-network architectures: fully connected (Dense), convolutional (CNN) and attention-based (Swin) neural networks. The fully connected neural network has a three-layer architecture with two 128-neuron hidden layers using ReLU activations. The convolutional neural network is identical to model A in [42], a simple fully convolutional model with five layers using max-pooling for spatial down-sampling and ReLU activations. Finally, we also use a small version of a shifted windows transformer (Swin) [43], a popular attention-based model for computer vision. All architectures use a softmax output layer. See table 1 for a summary of the model sizes and baseline accuracy on the target datasets.

Table 1. Summary of the three different neural-network architectures used for comparisons. The accuracy for MNIST and CIFAR10G is over the validation datasets with standard deviation calculated for ten separate trainings. The dense model uses three layers (128, 128, 10) with ReLU activations, CNN is model A in [42] and Swin is a shifted window transformer [43] based on the KERAS vision implementation [44].

Model	Parameters	MNIST	CIFAR10G
Dense	118k	$99.5\% \pm 0.07\%$	$45.8\% \pm 0.5\%\,\,\,\,\,$
CNN	836k	$99.9\% \pm 0.06\,\,\,\,\,\,\,$	$74.1\% \pm 0.95\%$
Swin	147k	$97.3\% \pm 0.1\%\,\,\,\,\,$	$66.2\% \pm 1.3\%\,\,\,\,\,$

One way of improving the accuracy of a neural-network model is to extend the training set. Active learning [45–48] makes use of the network predictions to choose inputs that contribute the most to increased accuracy. Which inputs to choose can be phrased in terms of a so-called acquisition function [49], that determines how samples are picked from the pool of unlabelled data. Common acquisition functions include BALD scoring [50] that picks samples with highest epistemic uncertainty as measured by mutual information I(x) in equation (6). Another common acquisition function is or to choose samples with highest predictive uncertainty [51] quantified by $H(y|x, \mathcal{D})$ in equation (4). We refer to these two methods as BALD and max entropy respectively. Based on our results for the correlation between accuracy and the joint distribution (H, I) we propose a novel acquisition function that picks samples with lowest expected accuracy in section 6.2.

Figure 2 shows the accuracy of predictions of three neural networks for MNIST and CIFAR10G as a function of predictive and epistemic uncertainty, evaluated on the union of all noise levels including the uncorrupted test set.

Zoom In Zoom Out Reset image size

For CIFAR10G in figure 2, the simple fully connected neural network in the first panel has a pronounced correlation between the predictive uncertainty and model accuracy. In the center panel, the convolutional model shows a different pattern where both epistemic and predictive uncertainty correlates equally with accuracy. For the attention-based model in the last panel, there is a more intricate relation. Decreasing epistemic uncertainty correlates with higher accuracy, but predictive uncertainty for a fixed epistemic value does not. In general, lower epistemic and aleatoric uncertainty does not always imply higher accuracy.

Turning to MNIST in figure 2, the first panel shows that the correlation between accuracy and predictive uncertainty is not as pronounced as in the first panel in figure 2. In the domain of MNIST where the dense model has higher accuracy there is instead a stronger correlation between decreasing epistemic uncertainty and prediction accurac. The center panel shows that for the convolutional neural network, for a fixed moderate predictive uncertainty, the accuracy does not monotonically increase with epistemic uncertainty. On the other hand, the attention-based Swin architecture in the right panel shows a monotonically increasing accuracy with decreasing epistemic uncertainty with similar structure as for CIFAR10G.

The moments of the joint uncertainty distribution for CIFAR10G at a fixed distributional shift are summarized in table 2 together with average prediction accuracies. For MNIST, the corresponding joint uncertainty distribution at fixed data-distributional shift is given in table 3. For reference, the moments and accuracy on the unshifted test set is shown in table 4. Comparing the average accuracy over the shifted dataset in table 2 with the accuracy on the validation set in table 4, we see that the dense model retains its accuracy better than the more complex architectures. For each individual architecture, a decrease in average predictive and epistemic uncertainty is accompanied by an increase in average accuracy.

Table 2. Mean and standard deviation for predictive and epistemic uncertainty for fully connected (Dense), convolutional (CNN) and attention-based (Swin) models with posterior approximations using ensembles evaluated on CIFAR10G at fixed data-distributional shift α = 0.09.

Architecture	Acc.	H	I
Dense	$41\%$	$1.66 \pm 0.39$	$0.10 \pm 0.04$
CNN	$20\%$	$1.92 \pm 0.22$	$0.39 \pm 0.11$
Swin	$20\%$	$1.12 \pm 0.47$	$0.27 \pm 0.14$

Table 3. Mean and standard deviation for predictive and epistemic uncertainty for fully connected (Dense), convolutional (CNN) and attention-based (Swin) models with posterior approximations using ensembles for MNIST at fixed data-distributional shift α = 0.09.

Architecture	Accuracy	H	I
Dense	$65\%$	$0.54 \pm 0.47$	$0.35 \pm 0.32$
CNN	$73\%$	$0.67 \pm 0.54$	$0.25 \pm 0.22$
Swin	$58\%$	$0.69 \pm 0.44$	$0.44 \pm 0.30$

Table 4. Mean and standard deviation for predictive and epistemic uncertainty and accuracy for fully connected (Dense), convolutional (CNN) and attention-based (Swin) models with posterior approximations using deep ensembles evaluated on CIFAR10G without data-distributional shift.

Architecture	Acc.	H	I
Dense	$45\%$	$1.58 \pm 0.41$	$0.07 \pm 0.04$
CNN	$75\%$	$1.07 \pm 0.64$	$0.14 \pm 0.10$
Swin	$65\%$	$0.90 \pm 0.55$	$0.08 \pm 0.06$

Given the patterns in the accuracy distributions in the (H, I) plane shown in figure 2, is it possible to use information on where the model is less accurate to improve training? According to the results in section 6.1, both the predictive uncertainty and the epistemic uncertainty are needed to parameterize the accuracy. Since standard acquisition functions, such as BALD [50] and max entropy [51], only refer to the marginal uncertainty distributions, we propose here an acquisition function that uses H as well as I to pick inputs from the pool of unlabelled data D_pool corresponding to low-accuracy regions in the (H, I) plane.

The goal is to increase the prediction accuracy of the model by choosing low-accuracy inputs from $D_\textrm{pool}$ to label and train on. The problem is of course that the samples from $D_\textrm{pool}$ are not labeled, therefore there is no direct way of determining the accuracy. The idea is to create a look-up table $\mathrm{EA}(H,I)$ that parameterizes the relation between expected accuracy and (H, I). This is done using a labeled calibration dataset $D_\textrm{calibration}$, and discretising H as well as I to obtain the look-up table.

Using this look-up table an active-learning loop is constructed in the following way. For each iteration, the values of H and I are computed for all input samples in the unlabeled pool $D_\textrm{pool}$, and the expected accuracy $\mathrm{EA}(H,I)$ is evaluated. The input samples are then ordered by their expected accuracy, and the 50 samples with lowest expected accuracy are labeled and added to the training dataset. A new ensemble of neural networks is trained using the updated training dataset, and a new look-up table is created. This loop is iterated 20 times such that a total of 1000 inputs from $D_\textrm{pool}$ have been acquired.

To evaluate the proposed acquisition function LEA, we use a simple active learning setup [50] where an ensemble of neural networks is first trained on a small class-balanced subset of 20 inputs from the CIFAR10 training set. For the accuracy calibration dataset, we use a random subset of 10 000 samples from the CIFAR10 training dataset, corresponding to about 17% of the training data. We discretise the (H, I) plane into $16\times16$ regular bins limited by the minimum and maximum values of H and I. As the pool of unlabelled data we use the remaining CIFAR10 training dataset after the initial inputs and accuracy calibration dataset have been removed. We evaluate the active-learning scheme for three different acquisition functions: BALD (maximum mutual information), max entropy (maximum predictive entropy) and our proposed lowest expected accuracy function, LEA, described above.

Figure 3 shows the results of this procedure. The left panel shows the evolution of the accuracy on the validation set for the dense and convolution architectures, using the three different acquisition functions. LEA consistently outperforms BALD, as well as active learning based solely on the maximum predictive uncertainty. To visualize the acquisition process for a particular active-learning iteration, we also show (right panel) the accuracy on the calibration dataset (background tiles) for an example active-learning iteration, together with the pool of unlabelled data (red crosses) and the inputs acquired by the acquisition function (green circles). The panel shows that the acquired inputs are from regions in the (H, I) plane that neither maximise epistemic uncertainty nor predictive uncertainty, thus acquiring different inputs than using BALD or max entropy as acquisition function. This explains why LEA outperforms the other two acquisition functions: neither H nor I alone suffice to estimate the expected accuracy.

Zoom In Zoom Out Reset image size

To understand how the uncertainty distribution in figure 2 correlates with noise level, we now turn to the uncertainty density distributions at a fixed noise level. Figure 4 shows, for different neural-network architectures, the joint distribution of the predictive uncertainty defined by equation (4) and epistemic uncertainty as defined by equation (6) in the (H, I) plane. The uncertainty measures are calculated using approximations to the Bayesian posterior provided by the ensemble method as explained in section 4.2. The joint distributions in figure 4 are evaluated on the test set of CIFAR10G shifted by impulse noise with noise parameter α = 0.09, corresponding to $9\%$ of pixels set to the maximum grayscale intensity, see figure 1. The joint distributions in figure 4 allow us to separate the origin of uncertainty: a sample with large value of I on the vertical axis has higher epistemic uncertainty, whereas a sample on the horizontal axis has no epistemic uncertainty. The dashed red diagonal line indicates the bound on epistemic uncertainty imposed by equation (8). In particular this means that a sample close to the epistemic bound is dominated by epistemic uncertainty.

Zoom In Zoom Out Reset image size

The first panel in the top row of figure 4 shows the joint distribution for the dense model. The joint distribution is shifted towards higher predictive uncertainty and lower epistemic uncertainty. The top-center panel in figure 4 shows the joint uncertainty distribution for the convolutional neural network. Here, the perceived epistemic uncertainty is larger compared to the other models. The right-most panel in figure 4 shows that the attention-based model exhibits similar epistemic uncertainty to the convolutional neural network in the center panel. The attention-based model does however perceive a lower aleatoric uncertainty, although this is not accompanied by a significant increase in accuracy.

Turning to MNIST, figure 4 shows the corresponding joint distributions for the three architectures at noise level α = 0.09. The first panel shows joint distributions of predictive and epistemic uncertainty for the dense model. In contrast to the dense model on CIFAR10G, the epistemic uncertainty is larger than the other models and the predictive uncertainty is smaller. The center panel shows the joint distribution for the convolutional neural network, and the right-most panel the attention-based Swin model.

To evaluate the dependence of the uncertainty quantification on the training dataset we calculate the induced shift in joint distribution of predictive and epistemic uncertainty when we apply the same data-distributional shift to neural networks trained on MNIST and CIFAR10G. The shift from α = 0 to α = 0.09 in mean predictive and epistemic uncertainty is quantified in table 5. For the dense and Swin model we observe a significantly smaller shift in both H and I on CIFAR10G compared to MNIST, whereas the convolutional neural network perceives a larger shift on CIFAR10G compared to MNIST.

Table 5. Change in mean predictive and epistemic uncertainty, $\Delta \bar{H}$ and $\Delta \bar{I}$, for fully connected (Dense), convolutional (CNN) and attention-based (Swin) models from α = 0 to α = 0.09. Posterior approximation by ensembling.

Architecture	Dataset	α	$\Delta \bar{H}$	$\Delta \bar{I}$
Dense	MNIST	0.09	0.49	0.33
Dense	CIFAR	0.09	0.08	0.03
CNN	MNIST	0.09	0.65	0.24
CNN	CIFAR	0.09	0.85	0.25
Swin	MNIST	0.09	0.63	0.41
Swin	CIFAR	0.09	0.22	0.20

The dense and Swin model both contain more parameters for CIFAR10G compared to MNIST stemming from the four extra pixels in both spatial directions. One possible method to keep the parameter count constant would be to resample the resolution of the images, however this would also introduce some dependence on the resampling method. Here we keep the original resolution for simplicity and leave the possibility of resampling for future work.

Our results allow us to draw three conclusions regarding the connection between the joint distribution of epistemic and predictive uncertainty, and prediction accuracy.

First, looking at the joint distribution helps to identify samples where a given model is more accurate. One important goal of uncertainty quantification is to assess when the predictions of a model can be trusted. It is natural to expect that a prediction is more accurate when the predictive uncertainty is low. We show that the joint distribution of epistemic and predictive uncertainty can identify accurate predictions while predictive uncertainty or epistemic uncertainty alone cannot. This can be seen in figure 2, where the joint distribution clearly resolves where the models are accurate, and where they are not. Figure 2 show that the projection of the joint distribution on either the axis of predictive uncertainty or the axis of epistemic uncertainty mixes samples with high and low accuracy. As a consequence, selecting for predictive uncertainty or epistemic uncertainty alone, is not as effective in identifying where a given model is more accurate. This explains the findings in e.g. [20, 52], regarding the clustering of incorrect predictions at high uncertainty, and also explains why it is difficult to achieve higher precision using the marginalized measures of epistemic and predictive uncertainty, as observed in [22]: samples with small values for the marginalized distributions can still have accurate model predictions.

Second, for the attention-based model it is difficult to find a threshold of the projected distribution on either predictive uncertainty or epistemic uncertainty that results in high accuracy. The top-right panels in figure 2 show a distinct structure for the accuracy conditioned on the joint distribution where the marginalized distribution on either axis mixes samples of high and low accuracy. This has implications for active learning. In the present example, choosing samples based on e.g. posterior predictive uncertainty alone is inefficient, because it results in training on data regions where the model is already accurate.

Third, by training on data regions where the accuracy is low, there is potential to increase overall prediction accuracy. The joint distribution of epistemic and predictive uncertainty can be used to identify such data regions where the model accuracy can be improved, also for data that is not yet labeled. Figure 2 show that the regions are specific to each network architecture, and to the dataset used for training. The mixing of high and low accuracy prediction when projecting the distributions in figure 2 also explain the difficulty in achieving better sample efficiency by using the decomposition into aleatoric and epistemic uncertainty observed in [13]: the projected distributions will contain regions of high prediction accuracy for small uncertainties. An active learning scheme that makes use of uncertainty quantification could therefore benefit from selection using the joint distribution.

Since prediction accuracy depends on both epistemic and predictive uncertainty, we proposed an acquisition function for active learning that uses a parameterization of expected accuracy in terms of both conditional mutual information and predictive entropy. Our proposed LEA (lowest expected accuracy) acquisition function, defined in section 5.3, outperforms acquisition functions using marginal uncertainty distributions. The left panel in figure 3 shows that our calibrated accuracy acquisition produce samples that enable the neural networks achieve higher validation accuracy with less training samples. The right panel in figure 3 shows that LEA picks inputs from regions that would have been missed by BALD (acquired inputs do not maximise mutual information) and max entropy (acquired inputs do not maximise predictive entropy). We stress that the proposed acquisition function only considers single sample uncertainty, we expect that e.g accuracy gain per iteration can be further improved by incorporating expected accuracy in state of the art acquisition functions such as BatchBALD [40] or EPIG [53]. Also note that the comparison with BALD and max entropy in figure 3 can be argued to be unfair in the sense that LEA uses information about the targets from the calibration dataset. This could be amended by calibrating on the current training dataset instead of a held-out calibration dataset.

We can draw two conclusions from our results regarding how uncertainty quantification depends on model architecture.

First, the origin of uncertainty is not objective. Table 4 shows that there are significant differences in the perceived origin of uncertainty between the different model architectures for in-domain data. For CIFAR10G, the fully connected neural network perceives a higher degree of aleatoric uncertainty, and low epistemic uncertainty compared to the convolutional and attention-based models. This shows that the origin of uncertainty depends on the model architecture: the fully connected neural network struggles to express the relationship between inputs and classes of CIFAR10G, and thus perceives a higher degree of aleatoric uncertainty. Even though the model has low accuracy, the ensemble posterior approximation show a low epistemic uncertainty. Thus we do not expect to be able to salvage the accuracy of the fully connected neural network by increasing the size of the dataset. For CIFAR10G, the top row of figure 4 and table 2 show that for a moderate data-distributional shift, the fully connected neural network perceives a higher degree of aleatoric uncertainty and low epistemic uncertainty compared to the convolutional and attention-based models. Here the dense architecture is already perceiving the validation data as aleatoric noise and thus continues to do so for further shifts.

Second, the different model posteriors agree about the origin of uncertainty when the dataset complexity is low. In the case of MNIST, the bottom row of figure 4 and table 3 show that all neural-network architectures considered here have similar joint distributions of predictive and epistemic uncertainty. In other words, the MNIST data has relatively low complexity, as evident by the higher average accuracies in table 1, and therefore all three architectures succeed in capturing the important features. As a consequence, all three architectures agree about the origin of uncertainty.

In addition, there are two conclusions we can draw from our results regarding how predictive and epistemic uncertainty depend on the dataset. First, the relative change in perceived uncertainty under a fixed data-distributional shift depends on the dataset, and varies with model architecture. Comparing the change in the joint distribution under a fixed data-distributional shift in table 5, we observe that a given model architecture understands the same data-distributional shift in different ways, depending on the underlying training dataset. There is an asymmetry in this shift sensitivity: only the convolutional model shows a stronger sensitivity of the predictive and epistemic uncertainty when evaluated on CIFAR10G compared to MNIST. Furthermore, the fully connected model perceives the impulse noise on CIFAR10G very differently from impulse noise on MNIST. The distributional shift for MNIST induces a large change in both epistemic and predictive uncertainty, whereas for CIFAR10G the induced shift in uncertainty is significantly smaller. This can be explained by CIFAR10G containing realistic digital images, hence we expect impulse noise to be more in-domain than for MNIST. The converse is true for the convolutional model, where the induced shift is larger for CIFAR10G. In summary, the sensitivity to a particular data-distributional shift depends on the data domain and on neural-network architecture. Even though this difference in perceived relative change of uncertainty is present for widely different data, it also has implications for different domains in the same training dataset, something that would be interesting to quantify in more detail.

Second, robustness of prediction accuracy under data-distributional shifts for a given model depends on the dataset. For CIFAR10G, the higher accuracy of the fully connected neural network on noised data in table 2 shows that this architecture is more robust against this particular distributional shift, even though the model is less accurate close to the training domain as seen in table 4. For MNIST, the convolutional model is instead more robust than both the dense and attention-based architectures as seen in table 3.