Abstract
In the era of the Internet and big data, data permeate the entire process of multiple criteria decision making (MCDM). Therefore, generation of rational solutions from current observations and historical data has become an important and interesting issue. To address this issue, this paper proposes a data-driven MCDM framework in the context of the evidential reasoning approach. Three challenges in the framework are met, including the transformation of observations into assessments, the learning of parameters and their constraints from historical data, and the generation of a data-driven solution. The proposed framework is then used to model the diagnosis of thyroid cancer and generate data-driven diagnostic results. The three challenges in the application are met to aid radiologists in improving the diagnostic accuracy of thyroid cancers. To examine whether the application of the proposed data-driven MCDM framework to the diagnosis of thyroid cancer can help improve diagnostic accuracy, we conduct a case study by using the examination reports of three radiologists from July 2015 to October 2017 in the ultrasonic department of a tertiary hospital located in Hefei, Anhui Province, China.
Similar content being viewed by others
References
Athey, S. (2017). Beyond prediction: Using big data for policy problems. Science, 355, 483–485.
Balla, V., Gaganis, C., Pasiouras, F., & Zopounidis, C. (2014). Multicriteria decision aid models for the prediction of securities class actions: Evidence from the banking sector. OR Spectrum, 36, 57–72.
Baykasoğlu, A., & Gölcük, İ. (2015). Development of a novel multiple-attribute decision making model via fuzzy cognitive maps and hierarchical fuzzy TOPSIS. Information Sciences, 301, 75–98.
Baykasoğlu, A., Gölcük, İ., & Akyol, D. E. (2017). A fuzzy multiple-attribute decision making model to evaluate new product pricing strategies. Annals of Operations Research, 251, 205–242.
Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30, 1145–1159.
Butler, J. C., Jia, J. M., & Dyer, J. (1997). Simulation techniques for the sensitivity analysis of multi-criteria decision models. European Journal of Operational Research, 103(3), 531–546.
Butler, J. C., Morrice, D. J., & Mullarkey, P. W. (2001). A multiple attribute utility theory approach to ranking and selection. Management Science, 47(6), 800–816.
Cao, C., Leong, T. Y., Leong, A. P. K., & Seow, F. C. (1998). Dynamic decision analysis in medicine: A data-driven approach. International Journal of Medical Informatics, 51, 13–28.
Cappelli, C., Castellano, M., Pirola, I., Cumetti, D., Agosti, B., Gandossi, E., et al. (2007). The predictive value of ultrasound findings in the management of thyroid nodules. QJM Monthly Journal of the Association of Physicians, 100(1), 29–35.
Chan, B. K., Desser, T. S., McDougall, I. R., Weigel, R. J., & Jeffrey, R. B. (2003). Common and uncommon sonographic features of papillary thyroid carcinoma. Journal of Ultrasound in Medicine, 22(10), 1083–1090.
Chen, T. Y. (2014). An ELECTRE-based outranking method for multiple criteria group decision making using interval type-2 fuzzy sets. Information Sciences, 263(3), 1–21.
Chen, J. H., Alagappan, M., Goldstein, M. K., Asch, S. M., & Altman, R. B. (2017). Decaying relevance of clinical data towards future decisions in data-driven inpatient clinical order sets. International Journal of Medical Informatics, 102, 71–79.
Corrente, S., Greco, S., & Słowiński, R. (2016). Multiple criteria hierarchy process for ELECTRE Tri methods. European Journal of Operational Research, 252(1), 191–203.
Corrente, S., Greco, S., & Słowiński, R. (2017). Handling imprecise evaluations in multiple criteria decision aiding and robust ordinal regression by n-point intervals. Fuzzy Optimization and Decision Making, 16, 127–157.
Datnow, A., & Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17, 7–28.
Dunn, K. E., Airola, D. T., Lo, W. J., & Garrison, M. (2013). What teachers think about what they can do with data: Development and validation of the data driven decision-making efficacy and anxiety inventory. Contemporary Educational Psychology, 38, 87–98.
Fischer, G. W. (1995). Range sensitivity of attribute weights in multiattribute value models. Organizational Behavior and Human Decision Processes, 62(3), 252–266.
Frates, M. C., Benson, C. B., Charboneau, J. W., Cibas, E. S., Clark, O. H., Coleman, B. G., et al. (2005). Management of thyroid nodules detected at us: Society of radiologists in ultrasound consensus conference statement. Radiology, 237(3), 794–800.
Fu, C., & Wang, Y. M. (2015). An interval difference based evidential reasoning approach with unknown attribute weights and utilities of assessment grades. Computers & Industrial Engineering, 81, 109–117.
Fu, C., & Xu, D. L. (2016). Determining attribute weights to improve solution reliability and its application to selecting leading industries. Annals of Operations Research, 245, 401–426.
Fu, C., Xu, D. L., & Yang, S. L. (2016). Distributed preference relations for multiple attribute decision analysis. Journal of the Operational Research Society, 67, 457–473.
Fu, C., & Yang, S. L. (2011). Analyzing the applicability of Dempster’s rule to the combination of interval-valued belief structures. Expert Systems with Applications, 38(4), 4291–4301.
Fu, C., & Yang, S. L. (2012). The conjunctive combination of interval-valued belief structures from dependent sources. International Journal of Approximate Reasoning, 53(5), 769–785.
Fu, C., Yang, J. B., & Yang, S. L. (2015). A group evidential reasoning approach based on expert reliability. European Journal of Operational Research, 246(3), 886–893.
García-Cascales, M. S., Lamata, M. T., & Sánchez-Lozano, J. M. (2012). Evaluation of photovoltaic cells in a multi-criteria decision making process. Annals of Operations Research, 199, 373–391.
Genders, T. S. S., Spronk, S., Stijnen, T., Steyerberg, E. W., Lesaffre, E., & Hunink, M. G. M. (2012). Methods for calculating sensitivity and specificity of clustered data: A tutorial. Radiology, 265(3), 910–916.
Govindan, K., & Sivakumar, R. (2016). Green supplier selection and order allocation in a low-carbon paper industry: Integrated multi-criteria heterogeneous decision-making and multi-objective linear programming approaches. Annals of Operations Research, 238, 243–276.
Gullo, D. F. (2013). Improving instructional practices, policies, and student outcomes for early childhood language and literacy through data-driven decision making. Early Childhood Education, 41(6), 413–421.
Halu, A., Scala, A., Khiyami, A., & González, M. C. (2016). Data-driven modeling of solar-powered urban microgrids. Science Advances, 2(1), e1500700.
Hand, D. J., & Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45(2), 171–186.
Hedgebeth, D. (2007). Data-driven decision making for the enterprise: An overview of business intelligence applications. The Journal of Information and Knowledge Management Systems, 37(4), 414–420.
Horvath, E., Majlis, S., Rossi, R., Franco, C., Niedmann, J. P., Castro, A., et al. (2009). An ultrasonogram reporting system for thyroid nodules stratifying cancer risk. Journal of Clinical Endocrinology and Metabolism, 90(5), 1748–1751.
Horvath, E., Silva, C. F., Majlis, S., Rodriguez, I., Skoknic, V., Castro, A., et al. (2017). Prospective validation of the ultrasound based TIRADS (Thyroid Imaging Reporting And Data System) classification: results in surgically resected thyroid nodules. European Radiology, 27(6), 2619–2628.
Hu, M. Q. (2015). A data-driven feed-forward decision framework for building clusters operation under uncertainty. Applied Energy, 141, 229–237.
Hudson, D. L., & Estrin, T. (1984). EMERGE—A data-driven medical decision making aid. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(1), 87–91.
Huh, W. T., Levi, R., Rusmevichientong, P., & Orlin, J. B. (2011). Adaptive data-driven inventory control with censored demand based on Kaplan–Meier estimator. Operations Research, 59(4), 929–941.
Jain, R. K., Qin, J. J., & Rajagopal, R. (2017). Data-driven planning of distributed energy resources amidst socio-technical complexities. Nature Energy, 2, 17112.
Jiang, Z. Z., Zhang, R. Y., Fan, Z. P., & Chen, X. H. (2015). A fuzzy matching model with Hurwicz criteria for one-shot multi-attribute exchanges in E-brokerage. Fuzzy Optimization and Decision Making, 14, 77–96.
Kadziński, M., Corrente, S., Greco, S., & Słowiński, R. (2014). Preferential reducts and constructs in robust multiple criteria ranking and sorting. OR Spectrum, 36, 1021–1053.
Keeney, R. L. (2002). Common mistakes in making value trade-offs. Operations Research, 50(6), 935–945.
Kong, G. L., Jiang, L. L., Yin, X. F., Wang, T. B., Xu, D. L., Yang, J. B., et al. (2018). Combining principal component analysis and the evidential reasoning approach for healthcare quality assessment. Annals of Operations Research. https://doi.org/10.1007/s10479-018-2789-z.
Kovalchuk, S. V., Krotov, E., Smirnov, P. A., Nasonov, D. A., & Yakovlev, A. N. (2017). Distributed data-driven platform for urgent decision making in cardiological ambulance control. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2016.09.017.
Kundu, S., Kers, J. G., & Janssens, A. J. W. (2016). Constructing hypothetical risk data from the area under the ROC curve: Modelling distributions of polygenic risk. PLoS ONE, 11(3), e0152359.
Kwak, J. Y., Han, K. H., Yoon, J. H., Moon, H. J., Son, E. J., Park, S. H., et al. (2011). Thyroid imaging reporting and data system for us features of nodules: A step in establishing better stratification of cancer risk. Radiology, 260(3), 892–899.
Lan, J. B., Chen, Y. W., Ning, M. Y., & Wang, Z. X. (2015). A new linguistic aggregation operator and its application to multiple attribute decision making. Operations Research Perspectives, 2, 156–164.
Levi, R., Perakis, G., & Uichanco, J. (2015). The data-driven newsvendor problem: New bounds and insights. Operations Research, 63(6), 1294–1306.
Liu, H. C., Han, K., Gayah, V. V., Friesz, T. L., & Yao, T. (2015). Data-driven linear decision rule approach for distributionally robust optimization of on-line signal control. Transportation Research Part C, 59, 260–277.
Moon, W. J., Jung, S. L., Lee, J. H., Na, D. G., Baek, J. H., Lee, Y. H., et al. (2008). Benign and malignant thyroid nodules: US differentiation-multicenter retrospective study. Radiology, 247(3), 762–770.
Morrison, J. J., Hostetter, J., Wang, K., & Siegel, E. L. (2015). Data-driven decision support for radiologists: Re-using the national lung screening trial dataset for pulmonary nodule management. Journal of Digital Imaging, 28(1), 18–23.
Otegbeye, M., Scriber, R., Ducoin, D., & Glasofer, A. (2015). Designing a data-driven decision support tool for nurse scheduling in the emergency department: A case study of a southern new jersey emergency department. Journal of Emergency Nursing, 41(1), 30–35.
Papathanasiou, J., & Kenward, R. (2014). Design of a data-driven environmental decision support system and testing of stakeholder data-collection. Environmental Modelling and Software, 55, 92–106.
Park, J. Y., Lee, H. J., Jang, H. W., Kim, H. K., Yi, J. H., Lee, W., et al. (2009). A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid, 19(11), 1257–1264.
Shi, C., Chen, W. D., & Duenyas, I. (2016). Nonparametric data-driven algorithms for multiproduct inventory systems with censored demand. Operations Research, 62(2), 362–370.
Wakker, P. P., Jansen, S. J. T., & Stiggelbout, A. M. (2004). Anchor levels at a new tool for the theory and measurement of multiattribute utility. Decision Analysis, 1(4), 217–234.
Wang, J. M. (2012). Robust optimization analysis for multiple attribute decision making problems with imprecise information. Annals of Operations Research, 197(1), 109–122.
Wang, T. R., Liu, J., Li, J. Z., & Niu, C. H. (2016). An integrating OWA-TOPSIS framework in intuitionistic fuzzy settings for multiple attribute decision making. Computers & Industrial Engineering, 98, 185–194.
Wang, Y. M., & Luo, Y. (2010). Integration of correlations with standard deviations for determining attribute weights in multiple attribute decision making. Mathematical and Computer Modelling, 51(1–2), 1–12.
Winston, W. (2011). Operations research: Applications and algorithms. Beijing: Tsinghua University Press.
Wong, A. K. C., & Wang, Y. (2003). Pattern discovery: A data driven approach to decision support. IEEE Transactions on Systems, Man, and Cybernetics: Part C—Applications and Reviews, 33(1), 114–124.
Xu, D. L. (2012). An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis. Annals of Operations Research, 195(1), 163–187.
Yan, R., Ma, Z. J., Zhao, Y., & Kokogiannakis, G. (2016). A decision tree based data-driven diagnostic strategy for air handling units. Energy and Buildings, 133, 37–45.
Yang, J. B., Wang, Y. M., Xu, D. L., & Chin, K. S. (2006). The evidential reasoning approach for MADA under both probabilistic and fuzzy uncertainties. European Journal of Operational Research, 171(1), 309–343.
Yang, J. B., & Xu, D. L. (2002). On the evidential reasoning algorithm for multiple attribute decision analysis under uncertainty. IEEE Transactions on Systems Man and Cybernetics Part A—Systems and Humans, 32(3), 289–304.
Yang, J. B., & Xu, D. L. (2013). Evidential reasoning rule for evidence combination. Artificial Intelligence, 205, 1–29.
Zhang, M. J., Wang, Y. M., Li, L. H., & Chen, S. Q. (2017). A general evidential reasoning algorithm for multi-attribute decision analysis under interval uncertainty. European Journal of Operational Research, 257(3), 1005–1015.
Acknowledgements
This research is supported by the National Natural Science Foundation of China (Grant Nos. 71622003, 71571060, 71690235, 71690230, 71521001, and 71531008).
Author information
Authors and Affiliations
Corresponding author
Appendix: Proof of Theorem 1
Appendix: Proof of Theorem 1
Theorem 1
Given the individual assessment B(ei(al)) = {(P(Ω), 1)} and other individual assessments B(ej(al)) = {(Hn, βn,j(al)), n = 1, …, N; (Ω, βΩ,j(al))} (j ≠ i), the aggregated assessment B(y(al)), by using the ER rule presented in Definition 1, satisfies Property 1.
Proof
To verify the conclusion in this theorem, the combination of the assessment B(ei(al)) = {(P(Ω), 1)} with the aggregated result of the first (i − 1) assessments is presented as follows.
Suppose that {(Hn, βn,b(i-1)(al)), n = 1, …, N; (Ω, βΩ,b(i-1)(al))} is the aggregated result of the first (i − 1) assessments generated by using Definition 1. The combination result of this and B(ei(al)) = {(P(Ω), 1)} is defined as
where
and
From Eqs. (A.7)–(A.9), we have
and
Through \( \hat{\beta }_{n,b(i)} (a_{l} ) \), \( \hat{\beta }_{\varOmega ,b(i)} (a_{l} ) \), and \( \hat{\beta }_{P(\varOmega ),b(i)} (a_{l} ) \), it can be deduced from Eqs. (A.4)–(A.6) that
Finally, because \( \vec{\beta }_{n,b(i)} (a_{l} ) \) = \( \vec{\beta }_{n,b(i - 1)} (a_{l} ) \), \( \vec{\beta }_{\varOmega ,b(i)} (a_{l} ) \) = \( \vec{\beta }_{\varOmega ,b(i - 1)} (a_{l} ) \), and \( \vec{\beta }_{P(\varOmega ),b(i)} (a_{l} ) \) = \( \vec{\beta }_{P(\varOmega ),b(i - 1)} (a_{l} ) \), one has from Eqs. (A.2)–(A.3) that
The above analysis indicates that the aggregated assessment B(y(al)), by using Definition 1, satisfies Property 1. □
Rights and permissions
About this article
Cite this article
Fu, C., Liu, W. & Chang, W. Data-driven multiple criteria decision making for diagnosis of thyroid cancer. Ann Oper Res 293, 833–862 (2020). https://doi.org/10.1007/s10479-018-3093-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-018-3093-7