Canonical Measure of Correlation (CMC) and Canonical Measure of Distance (CMD) between sets of data

Abstract

This paper proposes a new method for determining the subset of variables that reproduce as well as possible the main structural features of the complete data set. This method can be useful for pre-treatment of large data sets since it allows discarding variables that contain redundant information. Reducing the number of variables often allows one to better investigate data structure and obtain more stable results from multivariate modelling methods.

The novel method is based on the recently proposed canonical measure of correlation (CMC index) between two sets of variables [R. Todeschini, V. Consonni, A. Manganaro, D. Ballabio, A. Mauri, Canonical Measure of Correlation (CMC) and Canonical Measure of Distance (CMD) between sets of data. Part 1. Theory and simple chemometric applications, Anal. Chim. Acta submitted for publication (2009)]. Following a stepwise procedure (backward elimination), each variable in turn is compared to all the other variables and the most correlated is definitively discarded. Finally, a key subset of variables being as orthogonal as possible are selected. The performance was evaluated on both simulated and real data sets. The effectiveness of the novel method is discussed by comparison with results of other well known methods for variable reduction, such as Jolliffe techniques, McCabe criteria, Krzanowski approach and its modification based on genetic algorithms, loadings of the first principal component, Key Set Factor Analysis (KSFA), Variable Inflation Factor (VIF), pairwise correlation approach, and K correlation analysis (KIF). The obtained results are consistent with those of the other considered methods; moreover, the advantage of the proposed CMC method is that calculation is very quick and can be easily implemented in any software application.

Introduction

In multivariate data analysis a very important problem to be faced is the correlation within the set of data, which greatly influences results of several statistical and chemometric methods. Examples of these are methods for regression analysis, the search for optimal informative subsets of objects in experimental design, and similarity/diversity analysis. In such methods it is essential to estimate how much data variability is related to systematic and potentially useful information rather than random noise and chance correlation. Random noise or chance correlation are always potentially present, making models both unstable and unreliable and giving undesired and sudden bias in data exploration. Especially in multivariate regression analysis, data correlation greatly influences regression results due to well known troubles like model instability, overfitting, errors in the regression coefficients, related to the presence of a strong correlation structure in the predictor variables.

The problem of data correlation is relevant in methods of chemoinformatics, which produce thousands of variables such as the interaction energy fields obtained from CoMFA-like approaches [2], the molecular fingerprints used to screen libraries of molecules [3], and the large collection of topological indices derived from graph–theoretical matrices. Also in analytical chemistry, highly dimensional spectral data are affected by correlation and hence need specific pre-treatment before analysis. This is the case, for instance, of multivariate data generated through coupled chromatographic methods such as liquid chromatography coupled with diode array detector (LC–DAD), gas chromatography and mass spectrometry (GC–MS), and liquid chromatography coupled with nuclear magnetic resonance (LC–NMR), which are increasingly common methods for analytical analysis.

Variable reduction refers to the procedure that aims at selecting a subset of variables able to preserve as much information of the original data as possible, but eliminating redundancy, noise and linearly or near-linearly dependent variables. When the variables to be used in a model are chosen on the basis of general principles and not accounting for a specific goal (i.e. some experimental property to model), the term variable reduction should be more properly used than variable selection. By variable reduction techniques, variables are chosen by some comparison among the variables themselves regardless of the specific property which needs to be modelled. For instance, in QSAR studies, molecular descriptors can be selected on the basis of their information content: descriptors with high information content are more effective in discriminating different molecules and, thus, are expected to be more effective in modelling any property of molecules.

In this paper some methods for variable reduction will be reviewed and compared to a new method: this is a backward elimination procedure based on the recently proposed canonical measure of correlation between two sets of variables [1]. The first part of the paper introduces some of the well known methods for variable reduction purposes and the new procedure with its theoretical fundamentals. Then, performance of the proposed method is evaluated on both simulated and real data sets and its effectiveness is discussed by comparison with results of the other methods for variable reduction.