An adaptive categorical effect size method based on intuitionistic meta fuzzy functions

Abstract

There are several categorical effect size methods in the literature. It is not clear which method performs better for a given dataset and it is a challenging task to select the correct method for a given dataset. In this sense, to overcome the questions like “Which method should we choose?” and “Which categorical effect size method is more reliable for a given dataset?”, an adaptive categorical effect size method based on intuitionistic meta fuzzy functions is introduced in the paper. Thus, the main motivation of the proposed method is to obtain more accurate outcomes by combining the results of better performing methods instead of relying on only one method. In the study, the intuitionistic fuzzy c-means clustering algorithm is adapted to meta fuzzy functions by incorporating not only membership degrees but also non-membership degrees to improve the clustering accuracy of meta fuzzy functions. Meta fuzzy functions are the linear combination of seven categorical effect size methods and the weights, which are calculated from membership grades from intuitionistic fuzzy c-means algorithm. Among the functions, the one with the lowest mean absolute percentage error is selected as the best. To evaluate the performance of the proposed method, 2 × 3, 2 × 4, and 3 × 4 contingency tables were simulated. Additionally, the performance of the proposed method is also assessed by applying it to a real-time dataset. Experimental results show that the proposed method outperforms compared to the evaluated seven categorical effect size methods in terms of mean absolute percentage error. Also, the calculated effect sizes are within the range of ±10% in terms of bias. Thus, the results verified that proposed method achieves greater reliability.

Introduction

Statistical significance (p-value) is the probability that the observed difference between two groups is due to chance. If the p-value is greater than the chosen alpha level, it is assumed that any observed difference can be explained by the variability of the sample size. When conducting statistical comparisons with exceptionally large sample sizes, it is highly likely that the p-value will consistently indicate a significant difference. However, statistically significant differences that arise due to the large number of data points do not always represent meaningful differences in reality¹. A statistically significant result may sometimes arise simply from using a large sample. Statistical significance depends on both the sample size and effect size (ES) but the effect size is generally independent of the sample size². Therefore, reporting only the p-value, especially in large samples, is not sufficient for readers to fully understand the implications^3,4. Effect size (ES) is substantial of quantitative research, and it indicates the real magnitude of the effect. In addition to statistical significance, it enables researchers to understand the practical significance of the findings. Statistical hypothesis tests can be misleading due to type 1 and type 2 errors made depending on the sample size. For this reason, it is necessary to report the effect size as well as the p-value in many disciplines. The seven categorical effect size methods, which is used for $r\times c$ contingency tables in statistics in the study, are explained in section “Categorical effect size methods”. The $Cramer{^\prime}s V$ effect size measure has some disadvantages. First, $Cramer{^\prime}s V$ is a symmetric measure of association^5,6,7. Second, it is zero under the assumption of independence. Third, interpretation of $Crame{r}{\prime}s V$ effect size measures is difficult⁸. $Tschuprow{^\prime}s T$ measure is closely related to $Cramer{^\prime}s V$ measure but less well-known⁹. Since it is a simple function of the Pearson chi-square statistic, it is among the commonly used effect sizes. Barely, the bias of the measure is large in data with small samples and it is difficult to interpret⁸. $Cohen{^\prime}s w$ is more appropriate for larger contingency tables¹⁰. $Uncertainty coefficient (U)$ is also commonly used effect size to measure the validity of a statistical classification algorithm¹¹.

Considering the disadvantages of the ES methods, it is important to select the correct ES method for a given dataset. To overcome the aforementioned disadvantages, selected 7 ES methods are aggregated in functions based on their performances for a given dataset. In this sense, the motivation of this paper is to combine different categorical effect sizes methods in functions with Meta Fuzzy Functions $(MFF)$ based on Intuitionistic Fuzzy C-Means Clustering $\left(IFCM\right)$ algorithm. Fuzzy c-means (FCM) clustering algorithm is used in MFF. FCM, proposed by Bezdek et al.¹², stands out as one of the frequently employed methods because of its simplicity and the benefits it offers compared to the k-means clustering algorithm. Nevertheless, it has certain drawbacks, including its susceptibility to initial settings and sensitivity to noise. In this sense, IFCM that accounts for hesitancy of an object belonging to a cluster is employed in MFF. Intuitionistic Fuzzy Sets $(IFSs)$ are introduced as a modification of Zadeh’s fuzzy set theory by Atanassov^13,14. The main difference between fuzzy sets and $IFSs$ is that fuzzy sets only consider membership degree while $IFSs$ consider both membership and non-membership degrees. That is, IFSs account also for the hesitancy of membership grades in clusters. Thus, the centers of the clusters are obtained more accurately. It has been determined by the studies that IFSs are more effective than traditional fuzzy set theory by overcoming uncertainty¹⁵. $IFSs$ have been commonly used for forecasting and engineering problems. In addition to time series and forecasting methods, $IFSs$ are widely used in the field of medicine for clustering images and diagnostics^16,17,18. Numerous studies employing IFSs have been proposed by Fan et al.¹⁹, Kumar and Gangwar²⁰, Lei, et al.²¹, Tak²², Gwak et al.²³.

Because aforementioned advantages of IFCM in the literature, it is employed in MFF. The $MFF$ was proposed by Tak²⁴. The purpose of the MFF is to combine methods or definitions used for the same purpose. Its logic is simply based on meta-analysis. Meta-analysis is a method that combines the outcomes of multiple studies to yield stronger results for a specific purpose. For example, Tak and Gök²⁵ and Gök and Tak²⁶ utilized the MFF to merge different definitions of currency crisis. By employing this approach, they aimed to enhance the accuracy and reliability of their analysis. Similarly, Tak et al.²⁷ employed the MFF to combine various time series methods. Their objective was to improve the forecasting performance by integrating multiple forecasting techniques within the framework. Cevik et al.²⁸ used the MFF approach to forecast the number of immigrants within the maritime line. Tak²⁹ used the MFF approach to forecast combination. These studies have shown that combining different methods with the $MFF$ has better estimation accuracy.

Yabacı Tak and Ercan³⁰ ensembled some ES definitions for two independent groups with MFF to obtain a more accurate effect size value. Yabacı Tak and Ercan³⁰ combined six effect size methods for numerical variables with the MFF approach by using classical fuzzy c-means algorithm $(FCM),$ which can be used with or without the assumption of normal distribution. The combined methods in the previous study were not used for categorical variables. Thus, numerous categorical ES methods are combined in this study. Besides, the $FCM$ clustering method only uses membership degrees while calculating the cluster centers. Thus, the $MFF$ approach with the $IFCM$, which provides a more accurate estimation of the cluster centers, has been developed in the study.

In the light of this information, we will introduce intuitionistic meta fuzzy categorical effect size functions $\left( {I - MFCESF} \right)$ approach. The aim of the study is to obtain better outcomes by combining seven categorical effect size measures in functions. The purpose of combining the ES is the assumption that each measure might have much or partial information for a given dataset. Therefore, while the methods that perform better will be gathered into one function, the methods that perform worse will be gathered into another. In the remainder of the paper, we will describe the IFCM and the meta fuzzy functions briefly in the section “Preliminaries”. The proposed method $\left(I-MFCESF\right)$ is discussed in section “Intuitionistic meta fuzzy categorical effect size functions (1-MFCESF)”. The performance of the proposed method is evaluated with some applications for simulated and real datasets in section “Evaluation”. Finally, the results of the proposed method are discussed in section “Conclusion”.

Preliminaries

The methods (effect sizes, intuitionistic fuzzy c-means and meta fuzzy function) that are used in the paper are detailed in this section.

Categorical effect size methods

Short descriptions of seven types of ES measures are provided for $r\times c$ contingency tables. $Cramer{\prime}s V$ is proposed in 1946 and it is an effect size measure that is generally used with nominal variables in $r\times c$ contingency tables^{7,31,32,33,34}. It is calculated in Eq. (1) based on Pearson’s chi-square statistic. It takes values between 0 and + 1.

$$V = \sqrt {\frac{{\varphi^{2} }}{{\min \left( {c - 1,r - 1} \right)}}} = \sqrt {\frac{{{\raise0.7ex\hbox{${\chi^{2} }$} \!\mathord{\left/ {\vphantom {{\chi^{2} } n}}\right.\kern-0pt} \!\lower0.7ex\hbox{$n$}}}}{{\min \left( {c - 1,r - 1} \right)}} }$$

(1)

where, ${\chi }^{2}$ is the Pearson’s chi-squared statistics, $n$ is the total observations number, $c$ is the number of coloumns and $r$ is the number of rows. In the Eq. (1), numerator of formula is based on the observed frequencies, denominator of formula is based on an unobserved frequencies. Therefore, when $Crame{r}{\prime}s V=1$, the marginal frequencies are not zero and r or c has not zero cell frequencies.

$Tschuprow{^\prime}s T$ is a ES which measures the association between two nominal variables in $rxc$ contingency tables³⁵. It takes values between 0 and +1, and calculated in Eq. (2).

$$T = \sqrt {\frac{{\varphi^{2} }}{{\left( {c - 1} \right) \times \left( {r - 1} \right)}}}$$

(2)

where, ${\chi }^{2}$ is the Pearson’s chi-squared statistics, $c$ is the number of coloumns and $r$ is the number of rows.

Another measure of categorical effect size is the $Pearson{^\prime}s contingency coefficient$ ($Pearson{^\prime} s c$). It takes values between 0 and + 1. $Pearson{^\prime} s c$ can be calculated in Eq. (3)³⁶.

$$Pearson{^\prime}s c = \sqrt {\frac{{\chi^{2} }}{{\chi^{2} + n}}}$$

(3)

where, $\chi^{2}$ is the Pearson’s chi-squared statistics, and n is the total number of observations.

$Cohen{^\prime} s w$ effect size is proposed by Cohen³⁷.$Cohen{^\prime} s w$ should be used for larger contingency tables. Cohen’s w effect size measure is obtained in Eq. (4).

$$w = \sqrt {\mathop \sum \limits_{i = 1}^{m} \frac{{\left( {p_{1i} - p_{0i} } \right)^{2} }}{{p_{0i} }}}$$

(4)

where, m is the number of cells, $p_{0i}$ is the value of the ith cell under the null hypothesis, $p_{1i}$ is the value of the ith cell under the alternative hypothesis.

$Goodman - Kruskal Tau \left( {G - K Tau} \right)$ is another ES measure of nominal variables. It measures the predictability of the column or row variable given the value of other variables, in percentage. The measure varies between 0 and 1^38,39. $G - K Tau$ is calculated in Eq.(5)⁴⁰.

$$GK - Tau = \frac{{n\mathop \sum \nolimits_{ij} \left( {\frac{{a_{ij}^{2} }}{{a_{.j} }}} \right) - \mathop \sum \nolimits_{i} a_{i.}^{2} }}{{n^{2} - \mathop \sum \nolimits_{i} a_{i.}^{2} }} , \;\; i = 1, \ldots r,\;\; j = 1, \ldots c$$

(5)

where, $n$ is the total number of observation, $a_{ij}$ is the value of number of observation in ith row and jth column, $a_{.j}$ is the total number of observation in jth column and $a_{i.}$ is the total number of observation in ith row.

$Uncertainty coefficient \left( U \right)$ is first introduced by Theil⁴¹. It is also called Proficiency, Entropy Coefficient or Theil’s U. It is often used as a measure of the ES of nominal variables in statistics and takes the value between 0 and + 1. This measure is defined in Eq. (6)

$$U\left( {XY} \right) = \frac{{H\left( X \right) - H\left( {XY} \right)}}{H\left( X \right)} = \frac{{I\left( {X;Y} \right)}}{H\left( X \right)}$$

(6)

$$H\left( X \right) = - \mathop \sum \limits_{x} P_{X} \left( x \right)logP_{X} \left( x \right) , H\left( {XY} \right) = - \mathop \sum \limits_{x,y} P_{X,Y} \left( {x,y} \right)logP_{{X \rm{\mid }Y}} \left( {x \rm{\mid }y} \right)$$

(7)

where, $H\left( X \right)$ is the entropy of a single distribution, $H\left( {XY} \right)$ is the conditional entropy and $U\left( {XY} \right)$ is the uncertainty coefficient. $P_{X,Y} \left( {x,y} \right)$ is the conditional distribution.

$Goodman - Kruskal Lambda \left( \lambda \right)$ statistic is an effect size proposed to measure the strength of the relationship between two nominal variables by evaluating the proportional reduction of error (PRE)³⁹. Also, $\lambda$ is the asymmetrical measure. The $\lambda$ statistic takes value between 0 and 1. How to calculate the $\lambda$ statistic is given in Eq. (8).

$$\lambda = \frac{{E_{1} - E_{2 } }}{{E_{1} }}$$

(8)

where, $E_{1}$ is the number of prediction errors made when the independent variable is ignored, $E_{2}$ equal to the number of prediction errors made when the prediction is based on the independent variable.

IFCM

Over the past decades, the fuzzy set theory proposed by Zadeh¹⁴ has been expanded with different approaches. Among these, intuitionistic fuzzy set theory, which has been commonly used in the literature and has many applications in different fields, was developed by Atanassov¹³. While only the membership degree is taken into account in the FCM, non-membership degree is also taken into account in IFCM. So that, the centers of the clusters are calculated more accurately. Algorithm are given below²²:

Step-1.:

Determine the number of clusters $\left( c \right)$, the fuzziness index (f), and initialize the cluster centers $\left( {v_{i} } \right)$ randomly.

Step-2.:

Calculate the degrees of membership ($\mu$) and non-membership ($u$). Formulas are given in Eqs. (9–11):

$$\mu_{ik} = \left[ {\mathop \sum \limits_{j = 1}^{c} \left( {\frac{{d\left( {x_{k} ,v_{i} } \right)}}{{d\left( {x_{k} ,v_{j} } \right)}}} \right)^{{\frac{2}{f - 1}}} } \right]^{ - 1} , \;\;i = 1,2, \ldots ,c ;\;\;k = 1,2, \ldots ,n$$

(9)

where $d\left( . \right)$ is the Euclidean distance between kth data in the ith cluster center:

$$u_{ik} = \left( {1 - \mu_{ik}^{\alpha } } \right)^{{{\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 \alpha }}\right.\kern-0pt} \!\lower0.7ex\hbox{$\alpha $}}}} , \alpha > 0$$

(10)

$$\mu_{ik}^{*} = 1 - u_{ik}$$

(11)

Step-3.:

Update the cluster centers by using Eq. (12):

$$v_{i} = \frac{{\mathop \sum \nolimits_{k = 1}^{n} \left( {\mu_{ik}^{*} } \right)^{f} x_{k} }}{{\mathop \sum \nolimits_{k = 1}^{n} \left( {\mu_{ik}^{*} } \right)^{f} }} , i = 1,2, \ldots ,c$$

(12)

Step-4.:

Algorithm is ended if the difference between two iterations are dropped under some given threshold ε; otherwise, repeated Step-2 and Step-3.

Meta fuzzy functions

Tak²⁴ proposes MFF to combine different methods or definitions, such as prediction and forecasting. The MFF consists of three components: functions, weights, and the best meta fuzzy function. Functions; the linear combination of weights and the findings of the selected methods. Weights: the membership grades that are obtained from FCM clustering algorithm are used to compute weights. The best meta fuzzy function: the function that has the best evaluation criteria. Meta fuzzy functions begin with obtaining the outcomes of the methods chosen for a purpose as the input matrix. After that, the input matrix is clustered using fuzzy c-means clustering algorithm to separate the categorical ES methods based on how well they predict outcomes. As a result, each method will be assigned to a cluster with a membership grade. Then, using membership grades for each cluster, the weights of the methods are calculated. In this case, there will be an equal number of functions as the cluster number. Finally, the best meta fuzzy function is selected based on its evaluation criteria.

Intuitionistic meta fuzzy categorical effect size functions $\left( {{\varvec{I}} - {\varvec{MFCESF}}} \right)$

$Cramer{^\prime} s v, \;\;Tschuprow{^\prime} s T, \;\;Pearson{^\prime} s c, \;\;Cohen{^\prime} s w,\;\; G - K Tau,\;\; U$ and $\lambda$ methods can be used to calculate effect size measures for a dataset. However, there is no definite information in the literature about which method is better or in which situations it should be used. Therefore, the performance of the methods may change according to the type of datasets. Because the performance of the ES measures in the proposed method is uncertain, we are looking for the optimum weights of the ES measures in the combination function. For this purpose, $I - MFCESF$ method is proposed in this paper. The ES measures are clustered based on their performances by using the IFCM. There will be as many functions as the number of clusters. Functions are obtained by multiplying each method by its weight in the clusters. The ES measures that perform better for the dataset will be in a function with a higher membership degree, while the ES measures that perform worse will be in another function with a higher degree of membership. Finally, the function with the minimum model evaluation criterion is selected as $I - MFCESF _{best}$ and new effect size value will be calculated for the dataset. So, $I - MFCESF$ method is an adaptive combination of categorical effect size measures. Step-by-step algorithm, pseudocode and flowchart are given below for $I - MFCESF$ approach.

Algorithm 1

Step 1.:

Determine $m$ categorical ES measures and simulated data randomly for t iterations. Obtain input matrix (Z) by applying $m$ measures to the simulated dataset for t repeats.

$$Z = [Z_{ij} ] , i = 1,2, \ldots ,t ;\;\; j = 1,2, \ldots ,m$$

(13)

where, $Z_{ij}$ is the ES value of ith repeat for jth measure.

$$Z = \left[ {\begin{array}{*{20}c} {Z_{1,1} } & {Z_{1,2} } & \ldots & {Z_{1,t} } \\ {Z_{2,1} } & {Z_{2,2} } & \ldots & {Z_{2,t} } \\ \vdots & \vdots & \ldots & \vdots \\ {Z_{m,1} } & {Z_{m,2} } & \ldots & {Z_{m,t} } \\ \end{array} } \right]$$

Step 2.:

The input matrix is clustered by using intuitionistic fuzzy c-means.

Step 2.1.:

The number of fuzzy clusters $\left( c \right)$ is determined and fuzzy index value $\left( f \right)$ and center of clusters $\left( v \right)$ are initialized.

Step 2.2.:

The degrees of membership ($\mu$) and non-membership value are calculated in each cluster with Eqs. (9–11).

Step 2.3.:

The new clusters center is calculated by using Eq. (12).

Step 2.4.:

If the difference between two iterations drops under some threshold, stop the algorithm; otherwise, repeat Step 1 and Step 2.

Step 3.:

Intuitionistic meta categorical effect size functions are obtained. $I - MFCESF$ is given in Eq. (14).

$$I - MFCESF_{i} \left( z \right) = \mathop \sum \limits_{j = 1}^{m} w_{ij} z_{j} , \;\; i = 1,2, \ldots ,c$$

(14)

$$w_{ij} = \frac{{\mu_{ij}^{*} }}{{\mathop \sum \nolimits_{j = 1}^{m} \mu_{ij}^{*} }} ,\; i = 1,2, \ldots , c$$

(15)

where, c is the number of clusters, $\mu_{ij}^{*}$ is the membership grades of jth method in $i$th cluster,$I - MFCESF_{i}$ is the ith intuitionistic meta categorical effect size functions, and $w_{ij}$ is weight of j.th method in $i$th cluster.

Step 4.:

Select the best intuitionistic meta categorical effect size functions that has the minimum Mean absolute percentage error (MAPE).

MAPE values are calculated for select $I - MFCESF_{best}$. Mape formula is given in Eq. (16).

$$MAPE = \frac{1}{n}\mathop \sum \limits_{i = 1}^{n} \left| {\frac{{y_{i} - \hat{y}_{i} }}{{y_{i} }}} \right|$$

(16)

where, $y_{i }$ is the mean of the ES value calculated from each method for the population and $\widehat{{y_{i} }}$ is the predicted ES value obtained from 1000 simulated samples. The pseudo code and the flow chart of $I - MFCESF$ based on MFF is given Algorithm 2 and Fig. 1, respectively.

Figure 1

Flowchart of $I-MFCESF$

Evaluation

The estimation performance of the proposed I-MFCESF method is evaluated through both simulation studies and the use of real-world datasets. In the simulation study, random generation of two categorical variables (x and y) is performed to create contingency tables of different sizes (2 × 3, 2 × 4, and 3 × 4). These tables are generated for a sample size of N = 1000 and repeated for t = 1000 iterations. Real-world datasets are obtained from the UCI Machine Learning Repository⁴², and 1000 different samples are taken with replacement from these datasets. By applying the selected categorical effect size methods to each dataset, an input matrix (Z) is obtained. The I-MFCESF method incorporates two crucial parameters: the number of clusters (c) and the fuzziness index parameter (m). To determine the optimal number of clusters (c), the minimum mean absolute percentage error (MAPE) for the I-MFCESF is calculated iteratively between 2 and 5. Due to the lack of consensus on the optimal value for the fuzziness index parameter of IFCM (intuitionistic fuzzy c-means algorithm), a value of 2 is selected for this study. The performance of the proposed method is evaluated using the MAPE, which measures the average percentage difference between the estimated values and the true values.

The simulation study and real-world dataset applications of the I-MFCESF method are conducted using R Studio. Various R package namely “ppclust,” “effectsize,” “DescTools,” “fclust,” “rcompanion, ” “remotes,” “githubinstall,” and “Metrics, ” are utilized^{43,44,45,46,47}. As an application, seven different selected categorical ES measures are combined by using the MFF based on the intuitionistic fuzzy c-means to obtain more accurate results for all datasets.

Simulated 2 $\times$ 3, 2 $\times 4$ and 3 $\times$ 4 contingency tables for the datasets of categorical variables

Two categorical variables x and y ($2 \times 3$, $2 \times 4$ and $3 \times 4$ contingency tables) are simulated randomly for N = 1000 sample size and t = 1000 iterations. Selected measures: $Cramer{^\prime} s v$ (Metasure 1), $Tschuprow{^\prime} s T$ (Measure 2), $Pearson{^\prime} s c$(Measure 3), $Cohen{^\prime} s w$ (Measure 4), $G - K Tau$(Measure 5), $U$ (Measure 6) and $\lambda$ (Measure 7) are applied to all datasets. The input matrix $\left( Z \right)$ consists of the outcomes of the ES measures for the simulated data set. The proposed method utilizes the IFCM clustering algorithm, where the fuzziness index parameter (m) is set to 2. After obtaining the input matrix, the IFCM algorithm is applied. In this method, the number of functions is equal to the optimal number of clusters. Functions are obtained by multiplying the weights of the methods with the actual value and sum them (Eq. 14) up. The weights of each method in each function are obtained as in Algorithm 1 (Step 3). Finally, the MAPE values are calculated for each from obtained $I - MFCESF$ functions. When calculating the MAPE values, the actual value is considered as the average of the values calculated from the dataset of the selected seven ES measures. The function with the lowest Mean Absolute Percentage Error is chosen as $I - MFCESF_{best }$ and the new ES value is computed based on this selection.

The first dataset is simulated for 2 $\times$ 3 contingency table and the input matrix ($Z$) is obtained by applying the selected categorical ES methods. The first five and last five prediction values of the input matrix are summarized in Table 1.

Table 1 Input Matrix for 2 $\times$ 3 contingency table

For the first simulated dataset, the optimal cluster number, which is set to 2, is determined by selecting the minimum MAPE value for $I-MFCESFs$. As a result, two functions are obtained by multiplying each method with their respective weights. The weights for the $I-MFCESF$ are computed using intuitionistic membership grades, as outlined in Table 2. The functions of the proposed method are obtained using the following equations (Eqs. 17, 18).

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.2117 + Pearson^{\prime}s c \times 0.2352 + Tschuprows^{\prime}T \times 0.2388 \\ & \;\;\; + Cohen^{\prime}s w \times 0.2349 + G - K Tau \times 0.0082 + U \times 0.0133 \\ & \;\;\; + \lambda \times 0.0578 \\ \end{aligned}$$

(17)

$$\begin{aligned} I - MFCESF_{2 } & = Cramer^{\prime}s v \times 0.0866 + Pearson^{\prime}s c \times 0.0143 + Tschuprows^{\prime}T \times 0.0057 \\ & \;\;\; + Cohen^{\prime}s w \times 0.0150 + G - K Tau \times 0.3022 + U \times 0.2985 \\ & \;\;\; + \lambda \times 0.2777 \\ \end{aligned}$$

(18)

Table 2 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for 2 $\times$ 3 contingency table

Table 2 provides a clear depiction that $I - MFCESF_{2 }$ exhibits the lowest MAPE. Therefore, $I - MFCESF_{2 }$ is identified as the best I-MFCESF. The MAPE values are computed and presented in Table 3, to assess the performance of the proposed method.

Table 3 MAPE and BİAS values of the proposed and selected effect size methods for 2 $\times$ 3 contingency table

Table 3 clear that the I-MFCESF outperforms the other categorical ES methods in terms of the MAPE values. According to the Li et al.⁴⁸ a parameter prediction is considered acceptable when the bias is within ± 10%. The bias value of the proposed method was determined as − 1% in Table 3. Thus, the accuracy of the method is also sufficient in terms of bias.

A subsequent dataset is simulated for a 2 × 4 contingency table, and the input matrix (Z) is obtained by applying the chosen categorical ES methods. Table 4 provides a summary of the first five and last five prediction values found in the input matrix.

Table 4 Input matrix for 2 $\times$ 4 contingency table

The weights for the $I-MFCESF$ are calculated by using intuitionistic membership grades as in Table 5, and the functions of the proposed method are obtained as in Eqs. (19, 20).

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.239 + Pearson^{\prime}s c \times 0.2386 + Tschuprows^{\prime}T \times 0.2335 \\ & \;\;\; + Cohen^{\prime}s w \times 0.2380 + G - K Tau \times 0.0039 + U \times 0.0116 \\ & \;\;\; + \lambda \times 0.0395 \\ \end{aligned}$$

(19)

$$\begin{aligned} I - MFCESF_{2 } & = I - MFCESF_{best } = Cramer^{\prime}s v \times 0.0386 + Pearson^{\prime}s c \times 0.0144 \\ & \;\;\; + Tschuprows^{\prime}T \times 0.0252 + Cohen^{\prime}s w \times 0.0156 + G \\ & \;\;\; - K Tau \times 0.3088 + U \times 0.3036 + \lambda \times 0.2938 \\ \end{aligned}$$

(20)

Table 5 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for 2 $\times$ 4 contingency table

Table 5 clearly shows ${I-MFCESF}_{2 }$ has the lowest MAPE. Thus, the best I-MFCESF is ${I-MFCESF}_{2}$. The MAPE values of the methods are computed, and the results are presented in Table 6 to assess the performance of the proposed method.

Table 6 MAPE and BİAS values of the proposed and selected effect size methods for $2\times 4$ contingency tables

Based on the information provided in Table 6, it is evident that the I-MFCESF method demonstrates superior performance compared to the individual categorical effect size methods in terms of MAPE. The bias of the proposed method is determined as respectively − 1.9%. Because bias is between ± 10%, the accuracy of the proposed method is also sufficient in terms of bias.

Lastly, a dataset is simulated for a 3 × 4 contingency table, and the input matrix (Z) is generated by applying the selected categorical ES methods. Table 7 provides a summary of the first five and last five prediction values found in the input matrix.

Table 7 Input Matrix for 3 $\times$ 4 contingency table

The weights for the $I-MFCESF$ are calculated by using intuitionistic membership grades as in Table 8.

Table 8 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for 3 $\times$ 4 contingency table

Table 8 demonstrates that two functions are computed by multiplying each method with their respective weights. In the case of I-MFCESF, the weights are determined using intuitionistic membership grades. The functions of the proposed method are derived using the equations provided in Eqs. (21, 22).

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.1541 + Pearson^{\prime}s c \times 0.2722 + Tschuprows^{\prime}T \times 0.2496 \\ & \;\;\; + Cohen^{\prime}s w \times 0.2708 + G - K Tau \times 0.0155 + U \times 0.0182 \\ & \;\;\; + \lambda \times 0.0197 \\ \end{aligned}$$

(21)

$$\begin{aligned} I - MFCESF_{2 } & = I - MFCESF_{best } = Cramer^{\prime}s v \times 0.1699 + Pearson^{\prime}s c \times 0.0161 \\ & \;\;\; + Tschuprows^{\prime}T \times 0.0677 + Cohen^{\prime}s w \times 0.0181 + G - K Tau \times 0.2439 \\ & \;\;\; + U \times 0.2425 + \lambda \times 0.2417 \\ \end{aligned}$$

(22)

According to Table 8, it is evident that ${I-MFCESF}_{2 }$ exhibits the lowest MAPE. Therefore, ${I-MFCESF}_{2}$ is identified as the best I-MFCESF. The MAPE of the methods are computed and presented in Table 9 to assess the performance of the proposed method.

Table 9 MAPE and BİAS values of the proposed and selected effect size methods for $3\times 4$ contingency tables

Based on the information provided in Table 9, it is evident that the I-MFCESF method outperforms the individual categorical ES methods in terms of MAPE. The I-MFCESF bias value was determined as respectively. − 3.2 %. Because bias is between ±10%, the accuracy of the proposed method is also sufficient in terms of bias. Figures 2, 3 and 4 illustrate the MAPE and Bias values of the proposed methods and selected methods for various contingency tables.

Figure 2

MAPE and Bias values of the I-MFCESF and effect size methods for $2\times 3$ simulated data.

Figure 3

MAPE and Bias values of the I-MFCESF and effect size methods for $2\times 4$ simulated data.

Figure 4

MAPE and Bias values of the I-MFCESF and effect size methods for $3\times 4$ simulated data.

Real-world categorical dataset for 2 $\times$ 3, 2 $\times$ 4 and 3 $\times$ 4 contingency tables

The first dataset contains 34 variables; 33 of which are categorical and one of them is numerical. There are 366 observations in the dataset. The dataset is a related to the differential diagnosis of erythematous-squamous diseases. The data is taken from the UCI Machine Learning Repository database. It can be open accessed via (https://archive.ics.uci.edu/ml/datasets/Dermatology). The “family history”, “eosinophi”, and “erythema” variables in the “Dermatology” dataset are used. In the dataset, the family history feature has the value “1” if any of these diseases has been observed in the family, and “0” otherwise. Eosinophi has the value “0”” if feature was not present, “1” indicate the relative intermediate values, “2” indicate the largest amount possible. Erythema has the value “0” if feature was not present, “3” indicates the largest amount possible, and “1”, “2” indicate the relative intermediate values. A totally of 1000 different samples with replacements are drawn from the Dermatology dataset. In the proposed method, the input matrix $(Z)$ is obtained from the outputs of the calculated categorical ES measures for these samples. Then, the membership grades are obtained by clustering the input matrix with the IFCM algorithm. The fuzziness index parameter ($m)$ is taken as “2”. Using the membership grades, the weights of each categorical ES method in each cluster are calculated. The next step is to obtain the fuzzy functions by using the weights. There will be as many fuzzy functions as the optimum number of clusters. The optimum cluster number is searched between “2” and “5”, iteratively. Finally, the fuzzy function with the smallest MAPE is chosen and the new effect size value is calculated.

Family history and Eosinophi variables ($2\times 3$ contingency tables)

“Family history” and “Eosinophi” variables are selected in the Dermatology dataset for $2\times 3$ contingency table. The input matrix $(Z)$ is obtained from outcomes of seven ES measures for these variables. The first five and last five prediction values of the input matrix are summarized in Table 10.

Table 10 Input Matrix for family history and eosinophi variables contingency table

The weights for the $I-MFCESF$ are calculated as in Table 11 and ${I-MFCESF}_{1}$ and ${I-MFCESF}_{2}$ are obtained as in Eqs. (23, 24) for Family history and Eosinophi variables.

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.2384 + Pearson^{\prime}s c \times 0.2474 + Tschuprows^{\prime}T \times 0.2441 \\ & \;\;\; + Cohen^{\prime}s w \times 0.2451 + G - K Tau \times 0.0144 + U \times 0.0016 \\ & \;\;\; + \lambda \times 0.0090 \\ \end{aligned}$$

(23)

$$\begin{aligned} I - MFCESF_{2 } & = Cramer^{\prime}s v \times 0.0490 + Pearson^{\prime}s c \times 0.0117 + Tschuprows^{\prime}T \times 0.0186 \\ & \;\;\; + Cohen^{\prime}s w \times 0.0165 + G - K Tau \times 0.2975 + U \times 0.3056 \\ & \;\;\; + \lambda \times 0.3012 \\ \end{aligned}$$

(24)

Table 11 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for family history and eosinophi variables

In consideration of Table 11, it is obviously seen that the ${I-MFCESF}_{2 }$ has the lowest MAPE. Thus, the best I-MFCESF is ${I-MFCESF}_{2}$. Seven methods contribute the performance of the second function. Besides, the sixth method makes the most contribution, but the seventh, fifth, third, fourth, second, and first methods also have an impact on the effectiveness of I-MFCESF. The MAPE of the methods are computed, and the results are presented in Table 12 to evaluate the performance of the proposed method. Additionally, Fig. 5 provides a visual representation of the MAPE and Bias values for the proposed and selected methods specifically for the family history and eosinophi variables.

Table 12 MAPE and BİAS values of the proposed and selected effect size methods for family history and eosinophi variables

Figure 5

MAPE and Bias values of the I-MFCESF and effect size methods for family history and eosinophi variables.

According to Table 12, it is obviously seen that proposed I-MFCESF outperforms other categorical effect size methods in terms of the MAPE criterion. Moreover, the bias value of the proposed method is in the range of ± 10%, and it was found to be sufficient in terms of bias. As a result, the new ES value is calculated as 0.020 from Eq. (25).

$$\begin{aligned} I - MFCESF_{best } & = 0.0833 \times 0.0490 + 0.1106 \times 0.0117 + 0.0936 \times 0.0186 \\ & \;\;\; + 0.114 \times 0.0165 + 0.0124 \times 0.2975 + 0.0230 \times 0.3056 \\ & \;\;\; + 0.0012 \times 0.3012 = 0.020 \\ \end{aligned}$$

(25)

Family history and Eryhthema variables ($2\times 4$ contingency tables)

For $2\times 4$ contingency table, “Family history” and “Erythema” variables are selected in the Dermatology dataset. The input matrix of $I-MFCESF$ are obtained from outcomes of seven effect size measures for these variables. The input matrix is summarized in Table 13.

Table 13 Input Matrix for family history and eosinophi variables contingency table

When the number of clusters was iteratively tried between 2 and 5 to obtain the smallest MAPE, it was determined as 3 for this data set. The weights for the $$I-MFCESF$$ are calculated as in Table 14 and $${I-MFCESF}_{1}$$, $${I-MFCESF}_{2}$$ and $${I-MFCESF}_{3}$$ are obtained as in Eqs. (26–28).

Table 14 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for family history and eryhthema variables

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.0115 + Pearson^{\prime}s c \times 0.0003 + Tschuprows^{\prime}T \times 0.0159 \\ & \;\;\; + Cohen^{\prime}s w \times 0.0018 + G - K Tau \times 0.3194 + U \times 0.3301 \\ & \;\;\; + \lambda \times 0.3209 \\ \end{aligned}$$

(26)

$$\begin{aligned} I - MFCESF_{2 } & = Cramer^{\prime}s v \times 0.4854 + Pearson^{\prime}s c \times 0.0036 + Tschuprows^{\prime}T \times 0.4241 \\ & \;\;\; + Cohen^{\prime}s w \times 0.0186 + G - K Tau \times 0.0437 + U \times 0.0035 \\ & \;\;\; + \lambda \times 0.0211 \\ \end{aligned}$$

(27)

$$\begin{aligned} I - MFCESF_{3 } & = Cramer^{\prime}s v \times 0.0458 + Pearson^{\prime}s c \times 0.4004 + Tschuprows^{\prime}T \times 0.1290 \\ & \;\;\; + Cohen^{\prime}s w \times 0.3939 + G - K Tau \times 0.0186 + U \times 0.0017 \\ & \;\;\; + \lambda \times 0.0106 \\ \end{aligned}$$

(28)

According to Table 14, it is seen that the ${I-MFCESF}_{2 }$ has the lowest MAPE and the best I-MFCESF is ${I-MFCESF}_{2}$. Seven methods contribute to the performance of the proposed method. Besides, the first method makes the most contribution, but the third, fifth, seventh, fourth, second, and sixth methods also have an impact on the effectiveness of I-MFCESF respectively. The MAPE values of the methods are given in Table 15 to evaluate the performance of the proposed method. Also, Fig. 6 represents the MAPE and the Bias values of the proposed and selected methods for family history and eryhthema variables.

Table 15 MAPE and BİAS values of the proposed and selected effect size methods for family history and eryhthema variables

Figure 6

MAPE and Bias values of the I-MFCESF and selected methods for family history and eryhthema variables.

It is clear from the Table 15 that proposed I-MFCESF give very accuracy prediction results for both evaluation criteria MAPE and bias. The MAPE value of the proposed method is better than other categorical effect size methods and the bias value is in the range of ± 10%. Therefore, I-MFCESF was found to be sufficient in terms of MAPE and bias. As a result, the new effect size value is calculated as 0.1328 from Eq. (29).

$$\begin{aligned} I - MFCESF_{best } & = 0.1478 \times 0.4854 + 0.1706 \times 0.0036 \\ & \;\;\; + 0.1316 \times 0.4241 + 0.1732 \times 0.0186 + 0.0300 \times 0.0437 \\ & \;\;\; + 0.0240 \times 0.0035 + 0.0010 \times 0.0211 = 0.1328 \\ \end{aligned}$$

(29)

Eosinophi and Eryhthema variables $(3\times 4)$ contingency tables

For $3\times 4$ contingency table, “Eosinophi” and “Eryhthema” variables are selected in the Dermatology dataset. The input matrix of $I-MFCESF$ are obtained from outcomes of seven effect size measures for these variables. The input matrix is summarized in Table 16.

Table 16 Input matrix for family history and eosinophi variables contingency table

Table 17 is show that the weights are calculated on eosinophi and eryhthema variables. The functions ${I-MFCESF}_{1}$ and ${I-MFCESF}_{2}$, which were created over the weights are given in Eqs. (30) and (31).

$$\begin{aligned} I - MFCESF_{1 } & = Cramer^{\prime}s v \times 0.2132 + Pearson^{\prime}s c \times 0.0032 + Tschuprows^{\prime}T \times 0.1232 \\ & \;\;\; + Cohen^{\prime}s w \times 0.0096 + G - K Tau \times 0.2186 + U \times 0.2152 \\ & \;\;\; + \lambda \times 0.2169 \\ \end{aligned}$$

(30)

$$\begin{aligned} I - MFCESF_{2 } & = Cramer^{\prime}s v \times 0.0446 + Pearson^{\prime}s c \times 0.3405 + Tschuprows^{\prime}T \times 0.2297 \\ & \;\;\; + Cohen^{\prime}s w \times 0.3354 + G - K Tau \times 0.0121 + U \times 0.0211 \\ & \;\;\; + \lambda \times 0.0166 \\ \end{aligned}$$

(31)

Table 17 Weights of the $\mathrm{I}-\mathrm{MFCESF}$ for eosinophi and eryhthema variables

Considering Table 17, it is clear that regarding the MAPE criterion, ${I-MFCESF}_{2 }$ function the best prediction performance for this contingency table. The most contributed performance of the proposed method is $Pearson{^\prime}s c$. Also, other selected methods have smaller impact on the performance of the best function. Figure 7 represents the MAPE and the Bias values of the selected and proposed methods for eosinophi and eryhthema variables.

Figure 7

MAPE and Bias values of the I-MFCESF and selected methods for eosinophi and eryhthema variables

Table 18 lists the performances of selected and proposed method. It is obvious by looking at the MAPE and the Bias values of the methods that the best performance is produced by the proposed method. The bias value of the proposed methods is in the range of ± 10%, and the MAPE value of the proposed method is the lowest according to other effect size methods. Finally, new effect size value is calculated by using Eq. (32).

$$\begin{aligned} I - MFCESF_{best } & = 0.0190 \times 0.0446 + 0.1522 \times 0.3405 + 0.0910 \times 0.2297 \\ & \;\;\; + 0.1541 \times 0.3354 + 0.0177 \times 0.0121 + 0.0128 \times 0.0211 \\ & \;\;\; + 0.0162 \times 0.0166 = 0.1260 \\ \end{aligned}$$

(32)

Table 18 MAPE and BİAS values of the proposed and selected effect size methods for eosinophi and eryhthema variables

Conclusion

The significant two key points of the study can be highlighted as follows. The first, a new approach categorical effect size method based on the IFCM and MFF is used to ensemble seven different categorical effect size measures. Thus, instead of depending on a single categorical effect size method, seven categorical effect size methods are aggregated for more reliable and accurate outcomes. The second, I-MFCESF is an adaptive method that adjust itself based on the given dataset. Some advantages of I-MFCESF are below:

The proposed method incorporates seven different categorical effect size measures that are proposed under various conditions. In the literature, $Cramer{^\prime}s v$, $Pearson{^\prime}s c$, $Tschuprows{^\prime}T$, $Cohen{^\prime}s w$, $G-K Tau$, $U$ and $\lambda$ effect size measures are most used to $r\times c$ contingency tables. The interpretation ranges of these methods are in the same scale. Thus, these techniques are selected for the proposed method.

$IFCM,$ in which the hesitancy of an object belonging to a cluster with a degree of membership valueis taken into consideration, is used to improve the performance of the proposed method to obtain more accurate results.

$I-MFCESF$ is gathered the information of selected effect size measures in functions by considering their accuracy performances for a dataset. For example, for a given dataset, the X method may perform better than the Y method, while in another dataset, the Y method may perform better than the X method. In this case, the weight of the X method will be higher in the best in the first dataset, while the weight of Y method in the best function will be higher in the second dataset. For this reason, the proposed method has adaptive properties.

$I-MFCESF$ is usually select the best effect size measures with a higher weight in terms of MAPE among seven measures.

To demonstrate the performances of the proposed method, we generate two randomly independent categorical variables for N = 1000 sample and t = 1000 repeat. Besides, we have investigated Dermatology real-world dataset which are taken from the UCI Machine Learning Repository database. According to the simulation results, MAPE was obtained as 0.4168 with a bias of − 0.0106 for the 2 × 3 contingency table, 0.3581 with a bias of − 0.0019 for the 2 × 4 contingency table, and 0.2753 with a bias of − 0.0032 for the 3 × 4 contingency table. The results obtained from the real data, on the other hand, were 0.3196 MAPE with a bias of − 0.0083 for the 2 × 3 contingency table, 0.4767 MAPE with a bias of − 0.0595 for the 2 × 4 contingency table, and 0.3335 MAPE with a bias of − 0.0370 for the 3 × 4 contingency table. Both the simulation study and the applications on the real data set showed us that; the proposed method can predict the results better than the other effect size measures in terms of MAPE and bias values. The MAPE value of the proposed method was found to be lower in all the application results compared to the other methods, and the bias value was in the range of ± 10%. From the results we can claim that I-MFCESFs improve prediction accuracy by combining different effect sizes results. The limitation of the study can be identified as the fact that the performance of the proposed method is affected by the performance of a clustering algorithm. Although, IFCM accounts for the hesitancy of an object to be belong to a cluster, it does not consider the outliers in the dataset. In this sense, possibilistic fuzzy clustering algorithm, that accounts for the outliers, can be adapted in MFF. This scenario is left for the future study. Therefore, as a future research direction, we plan to combine the effect size measures used for different types of variables and utilize possibilistic fuzzy c-means. Also, to improve the performance of the proposed method, different categorical effect size measures can be included in MFF.