Regularized Bagged Canonical Component Analysis for Multiclass Learning in Brain Imaging

Abstract

A fundamental problem of supervised learning algorithms for brain imaging applications is that the number of features far exceeds the number of subjects. In this paper, we propose a combined feature selection and extraction approach for multiclass problems. This method starts with a bagging procedure which calculates the sign consistency of the multivariate analysis (MVA) projection matrix feature-wise to determine the relevance of each feature. This relevance measure provides a parsimonious matrix, which is combined with a hypothesis test to automatically determine the number of selected features. Then, a novel MVA regularized with the sign and magnitude consistency of the features is used to generate a reduced set of summary components providing a compact data description. We evaluated the proposed method with two multiclass brain imaging problems: 1) the classification of the elderly subjects in four classes (cognitively normal, stable mild cognitive impairment (MCI), MCI converting to AD in 3 years, and Alzheimer’s disease) based on structural brain imaging data from the ADNI cohort; 2) the classification of children in 3 classes (typically developing, and 2 types of Attention Deficit/Hyperactivity Disorder (ADHD)) based on functional connectivity. Experimental results confirmed that each brain image (defined by 29.852 features in the ADNI database and 61.425 in the ADHD) could be represented with only 30 − 45% of the original features. Furthermore, this information could be redefined into two or three summary components, providing not only a gain of interpretability but also classification rate improvements when compared to state-of-art reference methods.

Appendices

Appendix A: Hypothesis Test

Considering the success probability $p_{jcr} = (1/P){\sum }_{p=1}^{P}$${({{U^{p}_{c}}} > 0) }$, we can formulate the following hypothesis test:

$$ \begin{cases} H_{0}: & p_{jcr} = 0.5, ~j \text{~ is~ not~ relevant~ for~ c-th~ class~ and~ r-th~ eigenvector. }\\ H_{1}: & p_{jcr} \neq 0.5, ~j \text{~ is~ relevant~ for~ c-th~ class~ and~ r-th~ eigenvector. } \end{cases} $$

(18)

To be able to statically evaluate if p_jcr differs from 0.5, we define the following statistic:

$$ t_{jcr} = \frac{p_{jcr} - 0.5}{\sigma_{jcr} }, $$

(19)

where σ_jcr is a scaling factor proportional to the standard error of p_jcr. We now derive this scaling factor. The term ${\sum }_{p=1}^{P}$${({{U^{p}_{c}}} > 0) }$ counts of the number of times that a feature is positive over P bagging iterations. Thus, assuming that the bagging iterations are independent, it can be modelled as a rescaled Binomial distribution with parameters P (number of experiments) and p_jcr (success probability). Further, since the number of bagging iterations is very large, the binomial distribution can be approximated by a Normal distribution with mean P ⋅ p_jcr and variance P ⋅ p_jcr(1 − p_jcr). So, under the independence assumption, we can define σ_jcr as the standard deviation of the term $\frac {1}{P} {\sum }_{p=1}^{P}$${({{U^{p}_{c}}} > 0) }$, which is straightforwardly computed by rescaling the variance of the Normal distribution:

$$ \sigma_{jcr} = \sqrt{\frac{1}{P} \cdot p_{jcr}(1-p_{jcr})}. $$

(20)

However, we need to take into account that the observations are coming from a bagging process and independence can not be assumed. To address this problem, the standard deviation is computed with an unbiased estimator (Nadeau and Bengio 2000) which, applied to our scenario, provides the following corrected estimator for the standard deviation:

$$ \begin{array}{@{}rcl@{}} \tilde{\sigma}_{jcr}^{\text{corr}} &=& \sqrt{\frac{1}{P} \left( 1 + P\frac{M}{1-M}\right) {p_{jcr}(1-p_{jcr})}} \\ &\simeq& \sqrt{\frac{M}{1-M} {p_{jcr}(1-p_{jcr})}} \end{array} $$

(21)

and, therefore, the statistic t_j becomes:

$$ t_{jcr} = \frac{p_{jcr} - 0.5}{\sqrt[]{\frac{M}{1-M} {p_{jcr}(1-p_{jcr})} }}. $$

(22)

The statistic t_jcr is distributed according to the t-distribution with P − 1 degrees of freedom. Since P is very large, one can safely approximate the t-distribution by the standard normal distribution.

Once this statistic is calculated, the class-wise feature selection can be carried out by majority-voting r. This means that for each class we select the features that are considered as relevant by the majority of the eigenvectors.

Thanks to the inclusion of the statistical test, the cross-validation (CV) of the optimum amount of selected features is not needed, therefore reducing the computational time. Furthermore, this efficient approach allows the selection of features in a class-wise manner, improving the interpretability of the results and posing an advantage over the approach presented in Muñoz-Romero et al. (2017).

Appendix B: Unbalanced Method Results

In this appendix further results obtained with different versions of the method are depicted. In particular, here we present the results obtained in both databases when not using the balanced version of the method.

Table 9 ADNI - Accuracy results with the different versions of the method, considering the usage of the proposed selection and extraction methods in their unbalanced version

Regarding Table 9, the results are similar to the ones obtained in Table 5 in terms of accuracy and slightly worse considering the AUC. The main advantage of using the balanced version in this database is the improvement in the classification of the most critical class, sMCI.

In Table 10 we can see the results obtained using the unbalanced version of the method in the ADHD database. In this highly unbalanced database, the results are worse than the ones obtained in Table 6 in terms of accuracy, having that without the usage of the balanced version the method overfits to the most populated class. Therefore, it is critical in this database to use the balanced version.

Table 10 ADHD - Accuracy results with the different versions of the method, considering the usage of the proposed selection and extraction methods in their unbalanced version