Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Large-scale group-hierarchical DEMATEL method for complex systems

  • Wenyu Chen ,

    Roles Investigation, Methodology

    lorilouto_cwy@163.com

    Affiliations Graduate Collage, Air Force Engineering University, Xi’an, China, Air and Missile Defense College, Air Force Engineering University, Xi’an, China

    ⨯
  • Weimin Li ,

    Contributed equally to this work with: Weimin Li, Lei Shao, Tao Zhang, Xi Wang

    Roles Conceptualization

    Affiliation Air and Missile Defense College, Air Force Engineering University, Xi’an, China

    ⨯
  • Lei Shao ,

    Contributed equally to this work with: Weimin Li, Lei Shao, Tao Zhang, Xi Wang

    Roles Funding acquisition, Methodology

    Affiliation Air and Missile Defense College, Air Force Engineering University, Xi’an, China

    ⨯
  • Tao Zhang ,

    Contributed equally to this work with: Weimin Li, Lei Shao, Tao Zhang, Xi Wang

    Roles Data curation, Formal analysis

    Affiliation Unit 94907 of PLA, Nanchang, Jiangxi, China

    ⨯
  • Xi Wang

    Contributed equally to this work with: Weimin Li, Lei Shao, Tao Zhang, Xi Wang

    Roles Writing – review & editing

    Affiliation Xi’an Satellite Control Center, Xi’an, China

    ⨯

Abstract

Existing Decision-Making Trial and Evaluation Laboratory (DEMATEL) methods are mostly suitable for simple systems with fewer factors, and lack effective integration of expert knowledge and experience from large-scale group populations, resulting in a potential compromise of the quality of the initial direct relation (IDR) matrix. To make DEMATEL better suited for the identification of critical factors in complex systems, this paper proposes a hierarchical DEMATEL method for large-scale group decision-making. Considering the limitations of expert knowledge and experience, a method based on expert consistency network for constructing the expert weight matrix is designed. The expert consistency network is constructed for different elements, and the weights of experts in different elements are determined using the clustering coefficient. Following the principles of the classic DEMATEL method, the steps for identifying key elements in complex systems using the large-scale group-hierarchical DEMATEL method are summarized. To objectively test the effectiveness and superiority of the decision algorithm, the robustness of the algorithm is analyzed in an interference environment. Finally, the superiority of the proposed method and algorithm is verified through a case study, which demonstrating that the proposed decision-making method is suitable for group decision-making in complex systems, with high algorithm stability and low algorithm deviation.

Citation: Chen W, Li W, Shao L, Zhang T, Wang X (2023) Large-scale group-hierarchical DEMATEL method for complex systems. PLoS ONE 18(12): e0288326. https://doi.org/10.1371/journal.pone.0288326

Editor: Mehdi Keshavarz-Ghorabaee, Gonbad Kavous University, ISLAMIC REPUBLIC OF IRAN

Received: March 27, 2023; Accepted: June 25, 2023; Published: December 4, 2023

Copyright: © 2023 Chen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Detailed data information in Appendix A-D is provided on https://osf.io/gxtj5.

Funding: This research was funded by National Natural Science Foundation of China, grant number 62173339, 61873278. The funders had no role in study design, data analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

In the knowledge economy era, the number of influence factors for complex organizations and systems is increasing, and the complex interrelationships between factors lead to complexity and uncertainty in management of complex organizations and systems. Accurately identifying key factors and clarifying the importance between factors have become important research topics in organizational management and decision-making, leading to the emergence of a series of evaluation and decision-making methods [1–3].

Many Multi-Criteria Decision Making (MCDM) methods are applied to identify influence factors by determining the weights of the criteria, including commonly used ones such as: CRiteria Importance Through Intercriteria Correlation (CRITIC) [4–6], BWM(Best-Worst Method) [7], Complex Proportional Assessment (COPRAS) [8], Simultaneous Evaluation of Criteria and Alternatives(SECA) [9], Combinative Distance-based Assessment (CODAS) [10], Stepwise Weight Assessment Ratio Analysis (SWARA) [11], Method based on the Removal Effects of Criteria (MEREC) [12] and Evaluation based on Distance from Average Solution (EDAS) [13].

Professors Gabus and Fontela proposed the Decision-Making Trial and Evaluation Laboratory (DEMATEL) algorithm [14] in the 1970s. The algorithm identifies key elements by constructing a complex interrelationship diagram between factors and is used for analyzing complex systems.

Compared to the other methods, the DEMATEL method, based on graph theory, provides a more intuitive representation of the complex interrelationships between factors and its calculation process is more simple and straightforward. It has been widely promoted and applied in many fields such as system engineering, and management science. Costa Federica et al. investigated the role of human factors in promoting the establishment of sustainable continuous improvement (SCI) environment [15] by the DEMATEL, Huang et al. analyzed the key elements of circular supply chain management (CSCM) [16], Shahriar et al. analyzed the complex influence factors in the Covid-19 vaccine supply chain [17]. In addition, there are many application cases, which are not listed one by one here. Furthermore, many scholars have improved the traditional DEMATEL method. Some researchers are committed to combining the DEMATEL with other methods. Mohammad et al. improved DEMATEL by combining it with the best-worst method (BWM) and Bayesian network (BN), and applied it to safety management [18]. Sirous et al. studied the problem of selecting technical suppliers by constructing the Delphi-DEMATEL-ELECTRE method [19]. In addition to considering the combination with other methods, some researchers have improved the DEMATEL scale method and expanded the expression of expert judgment information, mainly about point estimation judgment information [20, 21], fuzzy number estimation information [22–25], and grey number estimation information [26–28]. In addition, some researchers have studied the normalization processing of the decision matrix in DEMATEL to solve the problem that the normalized matrix does not converge in some cases. Michnik et al. proposed a new DIM normalization processing method to solve the problem of the unsolvable TIM matrix [29], and Chen et al. expanded the normalization method for the influence matrix in DEMATEL [30].

However, both DEMATEL method and its improved version require experts to analyze each factor in pairs. In a complex system with n factors, experts need to make n(n−1) judgments. When n is larger than 10, this will require a significant workload, leading to emotional fatigue and boredom that may affect the decision-making results. This limitation restricted the application of DEMATEL in complex systems with numerous factors, and it was not given enough attention by researchers. In 2021, Du proposed a hierarchical DEMATEL method [31], which adopts hierarchical decomposition to divide complex systems into several subsystems. Experts only need to make judgments on subsystems, and then use the proposed method for data integration. This method greatly simplifies the DEMATEL calculation steps and processes in complex systems, reduces the workload of experts, and has attracted the attention of many scholars. Moreover, considering the limitations of individual expert knowledge and experience, Du further introduced group decision-making based on the hierarchical DEMATEL [32], however, the group decision-making method is only suitable for small expert groups. When dealing with complex system identification, a large-scale expert group with not less than 20 people is usually needed [33, 34]. In addition, when a large-scale expert group makes decisions, the rationality of expert weights directly influences the accuracy of decision-making results. However, the weight of experts in Du’s study is subjective, lacking persuasive expert weight calculation. The references mentioned above were subjected to comparative analysis, as shown in Table 1.

thumbnail
Download:
Table 1. Application of the DEMATEL method.

https://doi.org/10.1371/journal.pone.0288326.t001

In summary, although hierarchical DEMATEL has improved the shortcomings of traditional DEMATEL method and can effectively analyze factors in complex systems, how to integrate and utilize the wisdom of large-scale expert groups and improve the scientificity of the IDR matrix of hierarchical DEMATEL method is a new and urgent problem to be solved. This article believes that as the number of factors in complex systems increases and the relationships between factors become more complex, decision making by a small group of experts may not be sufficient to cope with such complexity. It is necessary to determine the direct influence matrix in the hierarchical DEMATEL method by a large group of experts. Decision making by a large group of experts presents the following characteristics:

  1. The group size is relatively large, usually consisting of no fewer than 20 decision-making experts;
  2. The decision-making problem exhibits multidimensional, complex, and stochastic attributes;
  3. High consistency requirements need to be met among the group.

When solving problems of large-scale group decision-making, the main difficulties are as follows [35]:

  1. There are significant differences among decision-makers. It is necessary to identify the status of each decision-maker and assign corresponding weights to achieve scientific evaluation results.
  2. Due to the large size of the decision-makers, it is important to use effective methods to gather the opinions of large groups to avoid leverage effects caused by intentional praise or criticism during the evaluation process.
  3. When group opinions are relatively scattered, it is necessary to effectively coordinate the differences in preferences among decision-makers to maximize the satisfaction of large-scale group decisions.

In the problem of large-scale group decision-making, how to objectively determine the weight of each expert is a key issue. However, this issue is often overlooked by researchers. Chen pointed out that only 41% of the cited papers in group decision-making problems mentioned the determination of expert weights [36].

There are three main methods for determining expert weights in existing research: subjective method, objective method, and comprehensive method. For the subjective method, expert weights are calculated based on factors such as age, attitude, and experience, and the mutual evaluation of experts [37]. Multiplicative Analytic Hierarchy Process (MAHP) [38], Simple Multi-Attribute Rating Technique (SMART) [39], and Delphi [40] are key methods for subjective expert weight determination. The objective method is based on the evaluation performance of experts, using individual decision matrices (IDMs) proposed by each expert as the main basis for judgment, and assigning different weights, which is usually more objective [41]. The weight is mainly determined based on the degree of closeness between the expert’s individual decision and the group decision [42–44]. In addition, some researchers have constructed expert opinion adjustment mechanisms, attempting to achieve consistency of group opinions by adjusting experts’ weights or decisions as much as possible. Pang developed a nonlinear programming model and determined expert weights by maximizing group consensus [45], where expert weights were adaptively adjusted based on their decisions. Yang used a fixed-point iteration method to adjust expert weights multiple times [46].

However, from the relevant research on expert weights, it can be seen that when the research problem is a multi-attribute decision-making problem, almost all methods use a weight value to represent the expert’s evaluation status for all attributes. For example, literatures [46–48] all use a weight to represent the expert’s performance in all fields.

In reality, each expert has limited knowledge and abilities in their own expertise. Using a single weight to represent an expert’s status in all fields is unreasonable, which cannot fully reflect the expert’s professionalism to highlight their important position in their research field. Repeatedly iterating to solve expert weights to achieve consensus of group opinions often puts pressure on experts who adhere to their own opinions, forcing them to give up their decisions and ideas.

Finally, since decision-making problems are subjective progresses, the results of decisions do not have a correct answer, only subjective judgments of "reasonable" or "unreasonable". It’s hard to judge whether one’s method is superior to others. Most studies usually compare the decision results of their proposed method with other methods through numerical calculation examples. When inconsistencies occur, researchers often use subjective analysis to explain the rationality of their method, which often lacks persuasiveness. How to use a verification experiment to demonstrate the superiority of the proposed method instead of subjective analysis and judgment is an aspect that almost no researchers have studied.

Therefore, based on the above analysis, we find that the existing research on using DEMATEL method to identify factors in complex systems has the following shortcomings:

  1. The traditional DEMATEL method can only be applied to situations with fewer elements. When the number of elements n>10, it will significantly increase the number of expert judgments and workload.
  2. Although Du attempted to solve problem (1) by using group hierarchical DEMATEL method, the expert group was small and could not better reflect the wisdom of the expert group. In addition, the expert weight was subjectively given and lacked convincing objective calculation.
  3. When calculating expert weights using the objective method, the importance of experts in all attribute fields is often measured by a single weight value. In reality, experts often have a certain disciplinary background, and the degree of specialization of their judgment may be higher for some factors in complex systems, but lower for other factors. Using a single weight to determine the expert’s judgment status for all factors is not scientific, and a more targeted method should be adopted.
  4. There is a lack of convincing means to test the effectiveness and superiority of decision-making methods. Almost all studies subjectively analyze the differences in decision results between different methods when demonstrating the superiority of their methods. This inevitably leads authors to analyze in a way that favors their own methods and lacks persuasiveness.

Based on this, the author proposes a hierarchical DEMATEL method for large-scale group decision-making to identify key factors in complex systems and address the following issues:

  1. In response to the heavy workload for experts in identifying complex systems using traditional DEMATEL methods, the hierarchical DEMATEL method proposed by Du is used to reduce the workload of experts in identifying elements of complex systems. The hierarchical DEMATEL method is combined with the large-scale group decision-making, and the expert number is not less than 20, which improves the quality of decision-making.
  2. In response to the problem of expert weight solving, considering the potential influences of experts’ knowledge, background, and profession, the factor setting weights are distinguished to construct the weight matrix of expert decision-making. Based on the IDR matrix of experts, the expert consistency network for a certain factor is constructed based on the performance of experts in scoring this factor, combined with the clustering coefficient of the weighted network to represent the consistency of experts. The weight of experts in scoring this factor is determined by their contribution to the clustering coefficient of the weighted network, and the weight matrix of expert decision-making is formed for all factors. This method avoids measuring the performance of experts in scoring in all fields with a single weight value, and can well represent the consistency of the group.
  3. In order to objectively demonstrate the effectiveness of the proposed method, interference scenarios are set to analyze the robustness of the decision-making algorithm, and subjective analysis of the decision-making results is avoided. The stability and deviation of the decision-making method after interference are analyzed. The interference scenario refers to the implementation of interference on the original expert data of a certain scale to simulate the judgment deviation of experts. The stability index refers to the degree of change in the decision-making result after random interference, and the deviation index refers to the degree of deviation of the decision-making result after interference from the true value. Obviously, when the stability of the algorithm is high, it indicates that the decision-making algorithm will not easily change the decision-making result due to the influence of disturbance. When the deviation of the algorithm is low, it indicates that the decision-making algorithm can ensure a result closer to the true decision even if it is interfered with.

The innovations of this article are:

  1. Introducing a new method for identifying the weights of experts in large-scale groups. This method assigns different weights to different indicators, abandoning the practice of using a single weight value to represent the decision-making status of experts under all indicators, in order to address the unique characteristics of each expert in terms of knowledge, skills, experience, and personality.
  2. Using the network clustering coefficient to describe the consistency of expert groups in scoring the same indicator, and calculating expert weights through the consistency between experts and the group, to maximize the requirement of opinion consistency in large-scale group decision-making.
  3. The methods involved are more suitable for analyzing the correlation between various factors within complex systems and identifying key factors. It can not only reduce the workload of experts but also improve the scientificity of decision-making results.
  4. Instead of analyzing the effectiveness of decision-making algorithms through subjective methods as in other studies, this article constructs interference scenarios to analyze the stability and bias of algorithm results when expert decision-making data is interfered with, which is more convincing.

The rest of this study is organized as follows. Section Preliminaries introduces the basic knowledge, mainly including the introduction of traditional DEMATEL method and hierarchical DEMATEL method. Section The Proposed method mainly including the construction of expert consistency network, calculation of weighted network clustering coefficient, calculation of expert weight matrix, collection of expert opinions, and overall calculation steps. In Section Case presentation and Methodology analysis, stability and deviation indicators of the algorithm are introduced, and numerical calculations and comparative analysis of different methods are performed. In Section Conclusion, our conclusions, contributions and innovations are explained.

Preliminaries

This section introduces the DEMATLE method and the hierarchical DEMATEL method. The hierarchical DEMATLEL method is a new method based on the DEMATEL method proposed in [31], which decomposes the complex system into several subsystems, invites experts to score the degree of influence between elements within each subsystem, and finally turn the set of IDR matrices of all subsystems into a super IDR matrix. This method can effectively reduce the workload of experts and consider the hierarchical characteristics of complex systems. Each of these two methods is described below.

DEMATEL method

The DEMATEL method is a structural model expansion method used to establish and analyze the interactions between complex criteria and oriented to factor analysis of complex systems, the basic elements of the DEMATEL method are as follows.

  1. Determine the IDR matrix between the elements
    Suppose a system F contains N elements, denoted as F = {f1,f2,⋯fN}, and experts are invited to judge the degree of direct influence among these N elements using a scale of {0,1,2,3,4}, representing "no influence", "low influence", "medium influence", "strong influence", and "very strong influence", respectively. The degree of influence of element fi on element fj is recorded as , the IDR matrix X = [xij]N×N is constructed according to xij, and when i = j, xij = 0, it represents no influence of the element itself.
  2. IDR matrix normalization
    Normalize the IDR matrix constructed by experts [49] (1) (2)
    ε is a non-Archimedean infinitesimal, the role of ε is to ensure that the infinite powers of the normalized IDR matrix can converge to zero in order to satisfy the conditions for the subsequent third step of constructing the comprehensive influence matrix.
  3. Constructing the comprehensive influence matrix
    The comprehensive influence matrix T is (3)
  4. Calculation of causality and centrality of each factor
    Calculate ri = ∑jtij and dj = ∑itij based on the comprehensive influence matrix T. The sum of each row element of matrix T, denoted by ri, represents the sum of the influence of factor fi on other factors, which is called the influence degree of factor fi. The sum of each column, denoted by dj, represents the sum of the influence of other factors on factor fi, which is called the being influenced degree of factor fi.
    Let ri+di be the centrality of fi, which characterizes the relative importance of factor fi in the system. Let ri−di be the causality of factor fi. If ri−di>0, then fi is a causal factor; if ri−di<0, then fi is a receive factor.
  5. Calculate the weights of each factor
    The weight of factor fi is (4)
    wi satisfies wi∈[0,1], .

In summary, the traditional DEMATEL method can be summarized in the following steps.

Step 1, experts are invited to make decisions on the system elements and construct the IDR matrix X;.

Step 2, the IDR matrix is normalized, and the normalization matrix H is obtained by Eq (2);

Step 3, the comprehensive influence matrix T is constructed by Eq (3);

Step 4, the centrality and causality of the elements are calculated from the comprehensive influence matrix T.

Step 5, the relative importance of each factor is calculated by Eq (4).

Hierarchical DEMATEL method

The traditional DEMATEL method requires the expert to compare each element pairwise, which is suitable for simple systems with few elements. However, when there are many elements in the system, determining the IDR matrix requires a huge amount of work (if there are n elements in the system, experts need to make n(n−1) judgments). This can easily lead to expert mental fatigue and boredom. In addition, complex systems generally have hierarchical characteristics, which cannot be reflected in the traditional DEMATEL method. To address these issues, Du proposed the hierarchical DEMATEL method in reference [31], which is suitable for identifying key elements in complex systems that contain many system factors and have hierarchical characteristics among them. The method mainly includes the following contents:

  1. Hierarchical decomposition of the system
    Hierarchical decomposition mainly includes vertical decomposition and horizontal decomposition. Horizontal decomposition focuses on dividing the critical factor identification problem of complex systems into several simple problems and vertical decomposition focuses on dividing the complex system into multi-level subsystems under a specific rule. Horizontal decomposition provides the rules for making vertical decomposition [31].
    The complex system F is decomposed into subsystems according to the horizontal and vertical decomposition. As shown in Fig 1, the complex system F can be decomposed into subsystems F1~FQ, and F1~FQ can be decomposed into subsystems , , etc. If F = {f1,f2,⋯fN}, the decomposition stops when the system F is decomposed downward to the factors fi∈{f1,f2,⋯fN}, which is a component of F.
  2. The IDR matrix of subsystem
    The experts are arranged to score all the subsystems, such as F, F1~FQ and , to obtain the IDR matrix of each system. For example, when system F contains Q subsystems, the IDR matrix of system F is denoted as X = [xqq′]Q×Q, xqq′ refers to the degree of direct influence of the q subsystem in F on the q′ subsystem; similarly, the IDR matrix of system Fq containing Nq subsystems can be denoted as , and the superscript q of is used to denote the system to which it belongs.
  3. Calculate the super IDR matrix of the total system
    The super IDR matrix of the total system F is obtained by integrating all the subsystem direct influence matrices after correction, which indicates the degree of direct influence among all elements. The integration rules are (5) where the calculation rules for the elements are (6)
    When q = q′, which means that the two subsystems are identical, , the elements can be obtained directly from the expert scoring matrix for that system;
    When q≠q′, it means that the two subsystems are different, xqq′ represents the degree of direct influence of the subsystems Fq and Fq′, the subscript number represents the order of the subsystems Fq and Fq′ in the IDR matrix of their superior system.
    represents the centrality of factor in subsystem Fq, i.e., (7)
    and are the elements in the comprehensive influence matrix Tq of the subsystem Fq with and Tq is normalized according to the IDR matrix Xq of the subsystem Fq using steps (2) to (3).
    For the convenience of example, the above describes the simple case of two levels. When the level decomposition of a complex system involves multiple levels, the modified IDR matrix of the subsystem needs to be derived sequentially from the bottom to the top level, and the recursive integration from low to high forms the super IDR matrix, and the specific process is detailed in the literature [31].
  4. Calculation of elemental importance and centrality
    is brought into the DEMATEL method as the IDR matrix, as in steps (1) to (5) in Section DEMATEL method, and the importance and centrality of each element are calculated.
thumbnail
Download:
Fig 1. Schematic diagram of subsystem division, this figure is from Du’s literature.

https://doi.org/10.1371/journal.pone.0288326.g001

The proposed method

In this section, we introduce the hierarchical DEMATEL method to large-scale group expert decision making. We will construct an expert consistency network based on the performance of the large-scale group experts when scoring the same factors, express the consistency of the experts through the weighted network clustering coefficients, determine the expert weights using the contribution of each expert to the consistency, each expert will have different weights when scoring different factors, express the different abilities shown by the experts in different fields through the weight matrix, and finally the resulting weighted IDR matrix of each system. The weighted IDR matrix will be integrated using the hierarchical DEMATEL method to obtain the centrality and the causality of each factor, and realize the large-scale group hierarchical DEMATEL decision.

On the one hand, this paper combines large-scale group experts with the hierarchical DEMATEL, proposes a new method of pooling group wisdom, and improves the authority and scientificity of the IDR matrix of subsystems; on the other hand, it integrates the ability of different experts in different fields, and highlights the professional status of experts in a certain field, because even some authoritative experts, who make judgments in certain fields, are not as scientific as experts who are good at that field.

Problem description and hypothesis

The hierarchical DEMATEL method is used to make decisions on the importance of various elements within a complex system. Firstly, the complex system is divided into several subsystems according to levels and categories, and then m experts are organized to judge the degree of influence among the subsystem factors to derive the IDR matrix. Due to the large number of levels and subsystems, the information of the parent system at each level is used to name a certain subsystem in the form of , and q1…qp−1 represents the information of the parent system at each level.

For example, a subsystem with subscript q1⊃q2, q2 means it’s serial number in its parent system, q1 is the serial number for its parent system at the higher level, and so on. In Fig 2, system F1⊃1 represents the subsystem of the second level, and its parent system is the first system of the first level. The IDR matrix given by the nth expert to the subsystem is denoted as , which includes pairwise comparisons of the degree of influence between the factors involved, using a scale of {0,1,2,3,4}, representing "no influence", "low influence", "medium influence", "strong influence", and "very strong influence". The interrelationships between the systems are shown in the Fig 2.

thumbnail
Download:
Fig 2. Representation of multi-layer system with IDR matrix.

https://doi.org/10.1371/journal.pone.0288326.g002

The problem to be solved in this paper is: How to fully explore the IDR matrix of m experts on the subsystem analysis, determine the weight of each expert in scoring different factors, and obtain the weight matrix of experts to calculate the weighted IDR matrix of the subsystem, and finally achieve the identification of the importance of each element for all factors based on the hierarchical DEMATEL method.

Building expert consistency networks

Suppose that m experts are organized to score the subsystem , which has K elements, where the IDR matrix given by the nth expert is denoted as , (8)

The element represents the influence degree of factor i relative to factor j in system made by the nth expert, the process of constructing the IDR matrix by the organization experts is shown in Fig 3.

thumbnail
Download:
Fig 3. Expert decision-making process.

https://doi.org/10.1371/journal.pone.0288326.g003

Due to the differences in professional background and competence knowledge of experts, certain subjectivity and deviation will occur in scoring decisions, and different weights are needed to measure the importance of experts’ decisions. In this paper, we use a weight matrix to determine the importance of experts for different factors instead of a single weight value, and establish an expert consistency network through the scoring performance of each expert to calculate the weight of experts for that factor.

Definition 1: Consistency of experts

Existing experts a and b make judgments on the degree of influence of factor i on factor j in system . is the decision value of expert a and is the decision value of expert b. Define the degree of agreement between experts a and b as (9)

Where a,b∈{1,2,⋯m}, TN is the difference between the maximum and minimum values in the DEMATLE evaluation scale, and in scales {0,1,2,3,4} of this paper, TN = 4.

The rationality and nature of the above definition is discussed as follows:

Property 1:

σab∈[0,1], when rating the importance of factor i compared to factor j, if expert a and b give equal ratings, i.e., when , σab = 1, indicating that the two experts’ agreement degree to reach the maximum value; when the difference between the ratings is the largest, i.e., when , σab = 0, indicating that the two experts’ agreement degree reach the minimum value.

Property 2:

As the ratings given by the two experts become closer, the value of σab will be bigger, indicating that the agreement degree between two experts is also greater, and σab = σba, i.e., the relationship between the two decision makers is symmetric.

When scoring element xij in the system , an undirected weighted network is constructed based on the consistency exhibited by m experts. The m experts are the nodes in the complex network, and the agreement degree among experts is the weight of the edges. Assuming that m experts score element xij as , the consistency matrix for element xij formed by m experts is (10)

The matrix can be considered as the consistency network adjacency matrix when m experts score the element xij in the system . The network can be denoted as . is an undirected weighted network, represents the set of m experts as nodes, represents the set of edges, and the weights of the edges are the agreement degree σab among the experts.

Similarly, the expert consistency network for all other elements can be constructed based on the expert scoring performance. Obviously, for subsystem , since it contains K elements, K×K consistency networks can be formed, and each network corresponds to an adjacency matrix, and the adjacency matrices of all networks will form the Super Consistency Matrix. The process of constructing an expert consistency network is shown in Fig 4.

thumbnail
Download:
Fig 4. Expert consistency network construction process.

https://doi.org/10.1371/journal.pone.0288326.g004

The later section will effectively reveal the intrinsic connections among the expert members by further analyzing the consistency network of experts in order to clarify the weight of the experts’ scores for each element.

Solving the weighted IDR matrix based on the weighted network clustering coefficient

After constructing experts consistency networks, we can study the relationship between experts based on the network features. The Clustering Coefficient of a network is the ratio of the connectivity between any two nodes in the network to the connectivity between the neighboring nodes they share.

In general, suppose a node vi has ki edges connecting it to other nodes, and these ki nodes are called neighbors of node vi. Obviously, there are at most possible edges between these ki nodes. The ratio of the actual number of edges Ei and the total number of possible edges between ki neighboring nodes of node vi is defined as the clustering coefficient Ci of node vi, i.e.

(11)

In simple terms, it is the ratio of the actual number of connections around a node to the theoretical maximum number of connections. The clustering coefficient C of the whole network is the average of the clustering coefficients Ci of all nodes vi. That is (12)

Clearly, when all nodes are isolated and there are no connecting edges, C = 0; and C = 1 only when the network is globally coupled, with any two nodes directly connected.

The clustering coefficient represents the degree of closeness and stability of groups formed in the network. When the clustering coefficient is higher, it indicates that the neighbors of the node are closer and the resulting clustering groups are more stable. Since this paper constructs a network based on the scores given by experts for a certain factor, it is only necessary to determine whether the clustering groups for the same factor judgment scenario are stable. The property of the clustering coefficient can just reflect the consistency of the expert group. The expert consistency network in this paper is a weighted network. Onnela [50] studied the clustering coefficient of weighted networks, and defined the clustering coefficient of nodes vi in a weighted network as: (13)

In this equation, wij represents the edge weight between node vi and vj, is the normalized weight, and ki is the degree of node vi. Combining Eqs (10) and (13), when all experts score the elements xij in subsystem , the clustering coefficient vector of m experts in weighted network can be obtained based on the consistency matrix : (14) Where (15) (16)

We can perform exponential normalization on Eq (14) with the exponent m being the number of experts, to obtain the weight value of the nth expert in network .

Normalize Formula (14) by exponentiation, where the exponent m is the number of experts, to obtain the weight value wn of the nth expert in network , there is (17)

To distinguish the subsystem information and elements targeted by the weight values, we add a superscript q1⊃⋯⊃qp−1:ij to the weight values in Eq (17), so that the weight value of the nth expert when judging the degree of influence of factor i on factor j in system can be expressed as , where (18)

Repeating Eqs (11) to (18), we can obtain the weight matrix of experts for all factors in the system . The weight matrix of the nth expert for the system is (19)

By combining the weight matrix and the IDR matrix, the weighted IDR matrix for subsystem is (20)

Based on Eqs (8) to (20), we can obtain the weighted IDR matrix for each subsystem by constructing an expert consensus network, calculating expert weights based on the weighted network clustering coefficient, and synthesizing the weighted IDR matrices for each subsystem. By using the weighted IDR matrices for each subsystem as the IDR matrix of the hierarchical DEMATEL method, we can analyze the causality and centrality of all elements in a complex system by the hierarchical DEMATEL method.

In summary, the main process of the proposed method in this article is shown in Fig 5.

thumbnail
Download:
Fig 5. Flow chart of the decision algorithm.

https://doi.org/10.1371/journal.pone.0288326.g005

Calculation steps of the proposed method

Based on the previous description, the computational steps of the large-scale group decision hierarchical DEMATEL method proposed in this paper are sorted out as follows.

Step 1: To classify a complex system hierarchically

For a system F = {fi|i = 1,2,…,y}, the influence factors fi are divided into subsystems by hierarchy and by attributes, and the subsystems are denoted as ;

Step 2: Constructing the IDR matrix for each subsystem.

The IDR matrix obtained from the judgment of subsystem by the nth expert is ;

Step 3: Constructing expert consistency networks for each factor

The expert consistency network is formed when the group experts judge the influence degree of factor i on factor j in the subsystem . The network adjacency matrix is calculated as Eqs (9) to (10);

Step 4: Calculation of expert weight matrix by weighted network clustering coefficients

According to Eqs (13) to (16), the clustering coefficient vector of all experts for the network is calculated, and the weights of all experts in this network are obtained by normalizing Eq (18); the consistency network for all factors is traversed, and the weight matrix of the experts is solved;

Step 5: Calculate the weighted IDR matrix

Calculate the weighted IDR matrix for subsystem from Eq (20);

Step 6: Solving the super IDR matrix by hierarchical DEMATEL method

Using Eqs (5) to (6), the super IDR matrix is solved through weighted IDR matrix ;

Step 7: Using the traditional DEMATEL method to calculate the centrality and causality of each factor

According to Eqs (1) to (4), the centrality ri+di of factor fi is calculated, which characterizes the relative importance of factor fi in the system; the causality ri−di of factor fi is calculated, and if ri−di>0, then fi is the cause factor, and if ri−di<0, then fi is the receive factor.

Case presentation and methodology analysis

Case presentation

There are many influence factors for combat capability, and the relationships between these factors are complex and intertwined. The hierarchical characteristics of these factors are obvious, so combat capability is a typical complex system.

Identifying and analyzing the key factors that influence combat capability is crucial for improving it. Sixteen factors (f1~f16) that influence combat capability can be identified and classified into five dimensions: communication , intelligence , command , logistics , and fire support , as shown in Table 2. Based on their attributes, these five dimensions can be considered as belonging to the command and control and communication systems , as well as the fire and logistics support systems . The detailed hierarchical relationships are shown in Fig 6.

thumbnail
Download:
Fig 6. Hierarchy of combat capability.

https://doi.org/10.1371/journal.pone.0288326.g006

thumbnail
Download:
Table 2. Influence factors of combat capability.

https://doi.org/10.1371/journal.pone.0288326.t002

Communication, intelligence, command, logistics, and fire support factors are all critical in combat, and there are complex interrelationships among the factors within each subsystem, as well as between different subsystems.

The proposed method used to identifying the key factors influencing combat capability. In this case, combat capability includes 16 factors, and using the traditional DEMATEL method would require experts to make 240 judgments, which is obviously a significant workload. However, using the hierarchical DEMATEL method only requires experts to make 55 judgments, which clearly demonstrates the advantages of this method.

Step 1 Hierarchical decomposition

The above elements are divided by hierarchy to form the structure diagram shown below

The combat capability system is decomposed into a two-level structure according to the hierarchy, with the first level containing two subsystems, command and control and communications F1, and firepower and logistical support F2. The second level contains 5 subsystems of communication F1⊃1, intelligence F1⊃2, command F1⊃3, logistics F2⊃1, and fire support F2⊃2. These five subsystems specifically contain these 16 specific factors.

Step 2 Constructing the IDR matrix for each subsystem

The IDR matrix needs to invite experts to judge the influence relationship between the factors contained in the system, using the 0~4 scale method, 20 military theory researchers, weapon equipment professionals, combat commanders were invited to judge the system F,F1,F2,F1⊃1,F1⊃2,F1⊃3,F2⊃1,F2⊃2, each expert needs to make 55 decisions, and the matrices X(n), , , , , , , respectively represent the IDR matrix obtained from the judgment of the nth expert on the corresponding system.

The decision situation of expert 1 is shown in Fig 7. Due to the large volume of data, please see Appendix A at https://osf.io/gxtj5 for additional expert decision information.

thumbnail
Download:
Fig 7. The decision situation for expert 1.

https://doi.org/10.1371/journal.pone.0288326.g007

Step 3 Building expert consistency networks

Taking the construction of expert consistency network G11 as an example, the naming rule of the expert consistency network indicates that the superscript of G11 represents the system number, and the subscript represents the factor number. Therefore, the consistency network formed by all experts when judging the degree of influence of the first factor of system F (i.e., system F1) on itself.

The IDR matrix X(n) of 20 experts is extracted, and the element n = 1,2…,20 in the first row and first column is calculated according to Formula (9) to obtain the consistency matrix Y11 formed by the 20 experts’ scoring on this factor. The specific values corresponding to each row and column are shown in Table 3, and the expert consistency matrix for other elements can be found in Appendix B at https://osf.io/gxtj5.

thumbnail
Download:
Table 3. Consistency matrix .

https://doi.org/10.1371/journal.pone.0288326.t003

In order to see the sparsity between experts more intuitively, according to the consistency matrix Y11 of the network G11, we use Gephi to draw the structure of the consistency network G11 as shown in Fig 8, even the thickness of the edge represents the weight of the edge, from the figure can be roughly seen that the experts are not equally close to each other.

thumbnail
Download:
Fig 8. Expert consistency network.

https://doi.org/10.1371/journal.pone.0288326.g008

Step 4 Calculation of expert weight matrix by weighted network clustering coefficients

Continue to use network G11 as an example to illustrate. The clustering coefficients of the 20 experts for the network G11 are calculated according to Eqs (13) to (16) in Table 4.

thumbnail
Download:
Table 4. Clustering coefficients of experts in network G11.

https://doi.org/10.1371/journal.pone.0288326.t004

The weights of the experts are obtained by normalizing the clustering coefficients according to Eq (17), the normalization index m = 20, to serve the purpose of reducing the weights of the experts who are far from the group consensus and giving more weights to the experts with high consensus.

As can be seen from Table 5, the 8th, 11th, 15th, and 20th experts have significantly lower weight values than the other experts, which means that when judging the degree of influence on the system F1 itself, the opinions of these experts are clearly inconsistent with other experts, and this situation can also be found from the matrix in Table 3, where the expert 20 with the lowest weight value, for example, he has a consensus degree of 0.75 with only 3 experts and consensus degree with other experts degree are all only 0.5, resulting in his low weight when scoring this element.

thumbnail
Download:
Table 5. Weights of experts in network G11.

https://doi.org/10.1371/journal.pone.0288326.t005

Iterating through all factors in turn, the weight matrices W(n), , , , , , and , where n = 1,2…,20, can be obtained for the 20 experts in making decisions on systems F,F1,F2,F1⊃1,F1⊃2,F1⊃3,F2⊃1 and F2⊃2.The weight matrices of expert 1 for each subsystem is given here representatively, as shown in Fig 9.

thumbnail
Download:
Fig 9. Weight matrix of expert 1.

https://doi.org/10.1371/journal.pone.0288326.g009

Due to the limitation of space, detailed data for the remaining expert weighting matrices can be found in Appendix C at https://osf.io/gxtj5.

Step 5 Calculate the weighted IDR matrix

According to Eq (20), the weighted IDR matrix of each system is obtained as shown in Fig 10.

thumbnail
Download:
Fig 10. Weighted IDR matrix.

https://doi.org/10.1371/journal.pone.0288326.g010

Step 6 Solving the super IDR matrix using hierarchical DEMATEL method

Combined with the weighted IDR matrix of each subsystem, the super IDR matrix can be calculated according to the hierarchical DEMATEL method and Eqs (5) to (6) in Section Hierarchical DEMATEL method. The super IDR matrix integrates the mutual influence relationships of all factors, and the calculation results are retained to three decimal places, as shown in Table 6.

thumbnail
Download:
Table 6. Super IDR matrix for combat capability system.

https://doi.org/10.1371/journal.pone.0288326.t006

Step 7 Bringing the super IDR matrix into the traditional DEMATEL method, using Eqs (1) to (4) can be calculated to obtain the reasonability and centrality of each factor, and non-Archimedean infinitesimal is ε = 0.00001.

As can be seen from Table 7, factors f4, f6, f9, f13, f14, f15, and f16 are receive factors, and the remaining factors are cause factors, which is consistent with our common sense.

thumbnail
Download:
Table 7. Reasonability.

https://doi.org/10.1371/journal.pone.0288326.t007

Table 8 shows the centrality of each element, based on which we can further obtain the weights of each element. According to the weights in Table 9, it can be concluded that the importance ranking of factors a influencing combat capability is

thumbnail
Download:
Table 8. Centrality.

https://doi.org/10.1371/journal.pone.0288326.t008

thumbnail
Download:
Table 9. Weight.

https://doi.org/10.1371/journal.pone.0288326.t009

It can be seen that the commander’s command ability, communication interference and anti-jamming ability, the degree of intelligence of the accusation system, the adequacy of the war reserve, and the speed of attack are the most critical five factors.

Comparative analysis

Algorithm stability and deviation introduction.

In most studies related to decision-making methods, scholars often analyze the superiority of the proposed method from a subjective perspective by explaining the reasons for differences in decision-making results. However, as decision-making problems themselves are subjective judgment problems, analyzing differences in decision-making results from a subjective perspective cannot provide sufficient evidence for judgment, as the analyst tends to analyze in their favor. This paper provides a new approach by comparing the robustness of decision-making methods in interference environments to analyze the advantages and disadvantages of the methods, i.e., comparing the stability and deviation of decision-making results in the interference of original data. The interference of original data is used to simulate the problem of random deviation in expert scoring caused by interference factors. Although this random deviation is a low-probability event, it can still cause changes in decision-making results when it occurs. Therefore, it is necessary to consider the ability of decision-making algorithms to maintain stability and the degree of deviation towards the true value.

The detailed definitions of the stability and deviation of decision-making algorithms are:

The decision result of the complex system is obtained in descending order of weights , and the subscript number of each factor corresponds to the decision vector . The decision result obtained from the original data in the interference-free condition is denoted as ψ0 and the decision vector is η0.

Suppose that L decision experiments are conducted in the interference scenario, and the L decision experiments are independent of each other, and the decision results are ψ1,ψ2,…ψL, and the corresponding decision vectors are η1,η2,…,ηL.

Definition 1 Decision algorithm stability

The cosine of the angle of the decision vector η1,η2,…,ηL in the Euclidean space is considered as the degree of agreement between the two decision outcomes in the L decision experiments, and for the decision vectors ηi and ηj, the degree of agreement between the two vectors is (21)

When cosθij = 1, it means that the decision vector ηi is completely consistent with ηj and the degree of consistency between the decision vectors has symmetry, i.e., cosθij = cosθji. From this, we can obtain the consistency matrix of L experiments (22)

The stability of the algorithm for L experiments is (23)

The matrix I is a matrix with all elements of 1, I∈RL×L, denotes the most ideal case. When P = I, it means that all the decision results are equal when L experiments are conducted, and the stability of the algorithm is infinite at this time, which means that the decision algorithm is highly stable and the decision results do not change in any random interference scenario.

Definition 2 Deviation of decision-making algorithms

The decision vector for the i th experiment in L decision experiments is ηi, then we have (24)

β is the deviation of decision-making algorithms in L decision experiments, which indicates the cumulative deviation of the experimental results from the true value.

As can be seen from the mathematical expressions of stability and deviation, stability mainly describes the degree to which the algorithm keeps the decision results consistent and stable with each other in multiple experiments. When the stability of the algorithm is high, it means that the decision results do not vary much from each other, indicating that the algorithm does not change the decision results easily due to random disturbances. The greater the deviation, the greater the deviation of the decision result from the true value after interference.

For the stability and deviation of a certain algorithm, there are several possibilities:

  1. Large stability and large deviation
    Implying that although the results of multiple experiments are stable, they deviate far from the true value.
  2. Large stability and small deviation
    Meaning that the results of multiple experiments are stable and each experiment is close to the true value, which is the most desirable situation.
  3. Small stability and large deviation
    Meaning that the results of multiple experiments are not stable, and the results deviate from the true value, which is the worst case.
  4. Small stability and small deviation
    Meaning that although the combined results are close to the true value, the algorithm is not stable enough and is susceptible to random interference.

Comparative analysis of different methods.

To analyze the scientific validity of the method proposed in this paper and its superiority compared to other methods, we compare the fixed-point iteration method for calculating expert weights in reference [36] with the method proposed in this paper. This part continues to use the data in Section Case presentation as the original data, adding data perturbation scenarios for multiple experiments, each experiment is independent of each other, and the data used in both methods in the experiments are guaranteed to be exactly the same after the perturbation.

When the perturbation is zero, the decision results calculated according to the fixed-method of immobile points are

Some differences can be seen with the decision results calculated by the method in this paper, however, we cannot compare the advantages and disadvantages of these two methods by subjective analysis of the decision results only. Therefore, in this section, the analysis is based on the stability and deviation.

We set the perturbation environment as follows:

The extent to which the data are perturbed is controlled by two parameters p1 and p2, p1∈[0,1] denotes the proportion of experts that are perturbed and p2∈[0,1] denotes the proportion of elements that are perturbed in perturbed experts. As an example, we set p1 = 0.3, p2 = 0.2, p1 means that 30% of the experts will be disturbed, p2 means 20% of these experts’ decision data will be randomly perturbed. The perturbed data will be randomly updated to an integer in the range [0–4], and the probability distribution follows a uniform distribution.

In order to consider all cases as much as possible, a range of p1∈0.04~0.36, p2∈0.04~0.36 with 0.02 steps was set and a total of 281 sets of experiments were performed considering different combinations of the two parameters. 40 randomized experiments were conducted in each group, and the averaging method, the fixed-point iteration method and the method of this paper were compared and analyzed with the same data and conditions. The averaging method is the control group, and this method treats all experts’ weights as equal, and in this paper, each expert’s weight is set to 0.05.

From Table 10, it can be seen that inf appears in the second row, indicating that the stability of the algorithm reaches infinity at this point. According to Formula (22), when the results of L independent experiments are exactly the same, matrix P = I, which means that the stability of the algorithm is infinite, as indicated by Formula (23), and the corresponding offset of the algorithm is 0. This means that throughout the L experimental process, all experiments are exactly the same as the original value.

thumbnail
Download:
Table 10. Experimental data of the first 10 groups.

https://doi.org/10.1371/journal.pone.0288326.t010

The first 10 sets of experimental data are given in Table 10, where method 1 refers to the averaging method, method 2 refers to the method in this paper, and method 3 refers to the method in [36]. Due to the large amount of data, the detailed data of 281 sets of experiments regarding stability and deviation are recorded in Appendix D at https://osf.io/gxtj5. According to the results of 281 sets of experiments, the methods with the largest stability and the smallest deviation in each group of experiments were recorded separately, and the number and percentage of occurrences of the three methods were counted, as shown in Table 11.

thumbnail
Download:
Table 11. Data and percentage of optimal performance of the 3 methods.

https://doi.org/10.1371/journal.pone.0288326.t011

As can be seen from Table 11, in the comparison of 281 sets of interference experiments, the method proposed in this paper has the largest stability value 199 times, occupying 68.86% of the number of experiments, and the smallest deviation value 236 times, accounting for 81.66% of the total number of experiments. It means that among these three methods, the method of this paper has the highest stability and accuracy, the method in [36] is the second, and the method of averaging directly on the expert decision matrix has the worst stability and accuracy.

Conclusion

In this study, we proposed a group hierarchical DEMATLE method for the identification of key factors of complex systems. The method inherits the advantages of the hierarchical DEMATLEL method, which can effectively reduce the workload of experts and, at the same time, large-scale group decision making enables more scientific and comprehensive decision results, which involves the expert weight matrix solving method to bring new ideas for the weight calculation when group experts make decisions. The main contributions and innovations of this paper are as follows:

  1. Taking into account the expertise and limited knowledge of experts, the experts are assigned weights by factors to measure the overall performance of experts more finely with the weight matrix.
  2. The consensus of group experts is described by constructing an expert consistency network, and the degree of consensus of experts is expressed by the assigned clustering coefficients as an important basis for calculating the weights.
  3. Stability and deviation indexes are proposed to test the effectiveness of the algorithm, which makes the decision algorithm test more convincing.

In this paper, the proposed method is applied to the identification of key factors of combat capability complex systems, and the proposed method is compared with other methods, and the experimental results have achieved good results. However, this article does not take into account the adjustment of expert opinions, and lacks the decision adjustment and opinion correction process of experts in reaching group consensus. This will be the focus of future research.

References

  1. 1. Wang L, Xue H. Integrated decision-making method for heterogeneous attributes based on probabilistic linguistic cross-entropy and priority relations. Entropy 2020;22:1009. pmid:33286778
  2. 2. Chen W, Chen Y. Two-step measurement of water–energy–food symbiotic coordination and identification of key influencing factors in the Yangtze River Basin. Entropy 2021;23:798. pmid:34201631
  3. 3. Nagarajan D, Lathamaheswari M, Broumi S, Kavikumar J. A new perspective on traffic control management using triangular interval type-2 fuzzy sets and interval neutrosophic sets. Operations Research Perspectives 2019; 6:100099.
  4. 4. Simic V, Gokasar I, Deveci M, Karakurt A. An integrated critic and MABAC based Type-2 neutrosophic model for public transportation pricing system selection. Socio-Economic Planning Sciences. 2022;80: 101157.
  5. 5. Puška A, Nedeljković M, Prodanović R, Vladisavljević R, Suzić R. Market assessment of pear varieties in Serbia using Fuzzy Cradis and critic methods. Agriculture. 2022;12: 139.
  6. 6. Su Y, Zhao M, Wei G, Wei C, Chen X. Probabilistic uncertain linguistic EDAS method based on prospect theory for multiple attribute group decision-making and its application to Green Finance. International Journal of Fuzzy Systems. 2022;24: 1318–1331.
  7. 7. Kumar A, A A, Gupta H. Evaluating green performance of the airports using hybrid BWM and Vikor methodology. Tourism Management. 2020;76: 103941.
  8. 8. Agarwal N, Tayal DK. A new model based on the extended COPRAS method for improving performance during the accreditation process of Indian Higher Educational Institutions. Computer Applications in Engineering Education. 2023;31: 728–754.
  9. 9. Azbari KE, Ashofteh P-S, Golfam P, Singh VP. Optimal wastewater allocation with the development of an SECA multi-criteria decision-making method. Journal of Cleaner Production. 2021;321: 129041.
  10. 10. Wei C, Wu J, Guo Y, Wei G. Green supplier selection based on codas method in probabilistic uncertain linguistic environment. Technological and Economic Development of Economy. 2021;27: 530–549.
  11. 11. Hashemkhani Zolfani S, Yazdani M, Zavadskas EK. An extended stepwise weight assessment ratio analysis (Swara) method for improving criteria prioritization process. Soft Computing. 2018;22: 7399–7405.
  12. 12. Keshavarz-Ghorabaee M, Amiri M, Zavadskas EK, Turskis Z, Antucheviciene J. Determination of objective weights using a new method based on the removal effects of criteria (MEREC). Symmetry. 2021;13: 525.
  13. 13. Irvanizam I, Syahrini I, Zi NN, Azzahra N, Iqbal M, Marzuki M, et al. An improved EDAS method based on bipolar neutrosophic set and its application in group decision-making. Applied Computational Intelligence and Soft Computing. 2021;2021: 1–16.
  14. 14. Fontela E, Gabus A. DEMATEL: Progress achieved. Futures 1974; 6:361–3.
  15. 15. Costa F, Lispi L, Staudacher AP, Rossini M, Kundu K, Cifone FD. How to Foster Sustainable Continuous Improvement: A cause-effect relations map of Lean Soft Practices. Operations Research Perspectives 2019; 6:100091.
  16. 16. Huang L, Zhen L, Wang J, Zhang X. Blockchain implementation for Circular Supply Chain Management: Evaluating critical success factors. Industrial Marketing Management 2022; 102:451–64. 2022.02.009.
  17. 17. Alam ST, Ahmed S, Ali SM, Sarker S, Kabir G, ul-Islam A. Challenges to covid-19 vaccine supply chain: Implications for sustainable development goals. International Journal of Production Economics 2021;239:108193. pmid:34121813
  18. 18. Yazdi M, Khan F, Abbassi R, Rusli R. Improved DEMATEL methodology for effective safety management decision-making. Safety Science 2020;127:104705.
  19. 19. Amirghodsi S, Naeini AB, Makui A. An integrated Delphi-DEMATEL-Electre method on Gray numbers to rank technology providers. IEEE Transactions on Engineering Management 2022;69:1348–64.
  20. 20. Lee W-S, Huang AYH, Chang Y-Y, Cheng C-M. Analysis of Decision Making Factors for equity investment by DEMATEL and Analytic Network process. Expert Systems with Applications 2011;38:8375–83.
  21. 21. Wu H-H, Chang S-Y. A case study of using DEMATEL method to identify critical factors in green supply chain management. Applied Mathematics and Computation 2015;256:394–403.
  22. 22. Gigović L, Pamučar D, Lukić D, Marković S. GIS-fuzzy DEMATEL MCDA model for the evaluation of the sites for Ecotourism Development: A case study of “dunavski ključ” region, Serbia. Land Use Policy 2016;58:348–65. 2016.07.030.
  23. 23. Zhang Z-X, Wang L, Wang Y-M, Martínez L. A novel alpha-level sets based fuzzy DEMATEL method considering experts’ hesitant information. Expert Systems with Applications 2023;213:118925. 2022.118925.
  24. 24. Giri BC, Molla MU, Biswas P. Pythagorean fuzzy DEMATEL method for supplier selection in Sustainable Supply Chain Management. Expert Systems with Applications 2022; 193:116396.
  25. 25. Sathyan R, Parthiban P, Dhanalakshmi R, Sachin MS. An Integrated Fuzzy MCDM approach for modelling and prioritising the enablers of responsiveness in automotive supply chain using Fuzzy DEMATEL, Fuzzy AHP and Fuzzy Topsis. Soft Computing 2022; 27:257–77.
  26. 26. Li Y, Zhao G, Wu P, Qiu J. An integrated gray DEMATEL and ANP method for evaluating the Green Mining Performance of Underground Gold Mines. Sustainability 2022;14:6812.
  27. 27. Liu X, Deng Q, Gong G, Zhao X, Li K. Evaluating the interactions of multi-dimensional value for sustainable product-service system with Grey DEMATEL-ANP approach. Journal of Manufacturing Systems 2021;60:449–58.
  28. 28. Xia X, Govindan K, Zhu Q. Analyzing internal barriers for automotive parts remanufacturers in China using grey-DEMATEL approach. Journal of Cleaner Production 2015;87:811–25.
  29. 29. Michnik J. Weighted influence non-linear gauge system (wings)–an analysis method for the systems of interrelated components. European Journal of Operational Research 2013;228:536–44.
  30. 30. Chen C-Y, Huang J-J. A novel DEMATEL approach by considering normalization and invertibility. Symmetry 2022;14:1109.
  31. 31. Du Y-W, Li X-X. Hierarchical DEMATEL method for Complex Systems. Expert Systems with Applications 2021;167:113871.
  32. 32. Du Y-W, Shen X-L. Group hierarchical DEMATEL method for reaching consensus. Computers & Industrial Engineering 2023;175:108842.
  33. 33. Xu X-H, Sun Q, Pan B, Liu B. Two-layer weight large group decision-making method based on multi-granularity attributes. Journal of Intelligent & Fuzzy Systems 2017;33:1797–807.
  34. 34. Xu X, Yin X, Chen X. A large-group emergency risk decision method based on data mining of public attribute preferences. Knowledge-Based Systems 2019;163:495–509.
  35. 35. Large-scale group DEMATEL decision making method from the perspective of complex network, Systems Engineering—Theory & Practice.2021
  36. 36. Chen Z, Zhong P, Liu M, Ma Q, Si G. An integrated expert weight determination method for design concept evaluation. Scientific Reports 2022;12. pmid:35428829
  37. 37. Xu Z, Cai X. Deriving weights from interval multiplicative preference relations in group decision making. Group Decision and Negotiation 2012; 23:695–713.
  38. 38. Ramanathan R. Group decision making using multiplicative AHP. Proceedings of the International Symposium on the Analytic Hierarchy Process 1996.
  39. 39. Barzilai J, Lootsma&nbsp FA. Power relations and group aggregation in multiplicative AHP and smart. Proceedings of the International Symposium on the Analytic Hierarchy Process 1994.
  40. 40. Azadfallah M. The extraction of expert weights from pair wise comparisons in Delphi Method. Journal of Applied Information Science 2015;3.
  41. 41. Koksalmis E, Kabak Ö. Deriving Decision Makers’ weights in Group Decision making: An overview of objective methods. Information Fusion 2019; 49:146–60.
  42. 42. Yue Z. Deriving Decision Maker’s weights based on distance measure for interval-valued intuitionistic fuzzy group decision making. Expert Systems with Applications 2011;38:11665–70.
  43. 43. Yue Z. Approach to group decision making based on determining the weights of experts by using projection method. Applied Mathematical Modelling 2012;36:2900–10.
  44. 44. Wang F, Wan S. Possibility degree and divergence degree based method for interval-valued intuitionistic fuzzy multi-attribute group decision making. Expert Systems with Applications 2020;141:112929.
  45. 45. Pang J, Liang J, Song P. An adaptive consensus method for multi-attribute group decision making under Uncertain Linguistic Environment. Applied Soft Computing 2017;58:339–53.
  46. 46. Yang Y, He J. A novel method based on fixed point iteration and improved topsis method for multi-attribute group decision making[J]. International journal of innovative computing, information and control 2021,
  47. 47. Liu Y, Li Y, Liang H, Dong Y. Strategic experts’ weight manipulation in 2-rank consensus reaching in Group Decision making. Expert Systems with Applications 2023; 216:119432.
  48. 48. He J, Yang Y. Deriving coordinated experts’ weights based on sequential quadratic programming algorithm for multi-attribute Group Decision making. Soft Computing 2022.
  49. 49. Lee H-S, Tzeng G-H, Yeih W, Wang Y-J, Yang S-C. Revised DEMATEL: Resolving the infeasibility of DEMATEL. Applied Mathematical Modelling 2013;37:6746–57.
  50. 50. Onnela J-P, Saramäki J, Kertész J, Kaski K. Intensity and coherence of motifs in weighted complex networks. Physical Review E 2005;71. pmid:16089800