A common framework for deriving preference values from pairwise comparison matrices

Abstract

Pairwise comparison is commonly used to estimate preference values of finite alternatives with respect to a given criterion. We discuss 18 estimating methods for deriving preference values from pairwise judgment matrices under a common framework of effectiveness: distance minimization and correctness in error free cases. We point out the importance of commensurate scales when aggregating all the columns of a judgment matrix and the desirability of weighting the columns according to the preference values. The common framework is useful in differentiating the strength and weakness of the estimated methods. Some comparison results of these 18 methods on two sets of judgment matrices with small and large errors are presented. We also give insight regarding the underlying mathematical structure of some of the methods.

Scope and purpose

Pairwise comparison is commonly used to estimate preference values of finite alternatives with respect to a given criterion. This is part of the model structure of the analytical hierarchy process, a widely used multicriteria decision-making methodology. The main difficulty is to reconcile the inevitable inconsistency of the pairwise comparison matrix elicited from the decision makers in real-world applications. We discuss 18 estimating methods for deriving preference values from pairwise judgment matrices under a common framework of effectiveness: the common concepts of minimizing aggregated deviation and correctness in error free cases. The common framework is useful in differentiating the strength and weakness of these methods. For each of these methods, we point out their individual strength in decisional effectiveness. Some comparison results of these 18 methods on two sets of judgment matrices with small and large errors are presented. We also give insight regarding the underlying mathematical structure of some of the methods. We recommend the simple geometric mean method with the stronger feature of distance minimization and the simple normalized column sum method that is based on the simple ideas of commensurate unit and column sum. These two methods have closed-form formulas for easy calculation and good performance on both sets of judgment matrices with small and large errors.

Introduction

In evaluating n competing alternatives A₁,…,A_n under a given criterion, it is natural to use the framework of pairwise comparisons represented by an n×n square matrix from which a set of preference values for the alternatives is derived. Many methods for estimating the preference values from the pairwise comparison matrix have been proposed and their effectiveness comparatively evaluated. Some of the proposed estimating methods presume interval-scaled preference values [1], [2]. But most of the estimating methods proposed and studied are within the paradigm of the analytic hierarchy process that presumes ratio-scaled preference values [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16]. The main challenge is how to reconcile the inevitable inconsistency of the pairwise comparison matrix elicited from the decision makers in real-world applications.

When the decision maker is unable to rank the alternatives holistically and directly with respect to a criterion, pairwise comparisons are often used as intermediate decision support. The approach taken herein is to specify a common way of measuring effectiveness of the solutions based upon what the estimating procedures are attempting to achieve. The stronger form of effectiveness is how close the ratio of derived preferences (w_i/w_j) come to the pairwise ratios (t_ij) in the comparison matrix. Those estimating techniques that rely on minimizing the aggregated distance between w_i/w_j and t_ij are called the “distance minimization” methods.

A weaker form of effectiveness relates to different ways of calculating preference values from the column of an error free comparison matrix. Since these techniques are known to give the correct preference values in error free cases, they may possess some efficacy when used under imperfect conditions with some errors. We call them “correctness in error free cases” methods. Although the distance minimization methods achieve the correctness in every error free case, we consider them under their stronger features of distance minimization.

In the paper, we are interested in the ability of the methods to estimate ratio-scaled preference values from pairwise comparison matrices. As mentioned above, our approach is to use the two effectiveness concepts as a common framework for the many methods of deriving preference values. Some comparison results of these 18 methods on two sets of matrices with small and large errors are presented. We also give insight regarding the underlying mathematical structure of some of the methods.

Problem structure is given in Section 2 and notations used are given in the appendix. A common framework based on two general concepts of effectiveness is introduced and discussed in Section 3. Some estimating methods based on distance minimization are introduced in Section 4. In Section 5, we describe several estimating methods that satisfy “correctness in error free cases.” Some numerical comparisons are given in Section 6 to illustrate the strength of these 18 estimating methods on two sets of judgment matrices with small and large errors. Insight from the underlying mathematical structure of some of the methods is discussed. Finally we highlight the importance of commensurate scales and provide some concluding remarks.