Skip to main content
SpringerLink
Log in

Multilevel Uncertainty Integration

  • Living reference work entry
  • First Online:
Handbook of Uncertainty Quantification

  • 486 Accesses

Abstract

This chapter discusses a Bayesian methodology to integrate model verification, validation, and calibration activities for the purpose of overall uncertainty quantification in model-based prediction. The methodology is first developed for single-level models and then extended to systems that are studied using multilevel models that interact with each other. Two types of interactions among multilevel models are considered: (1) Type-I, where the output of a lower-level model (component and/or subsystem) becomes an input to a higher-level system model, and (2) Type-II, where parameters of the system model are inferred using lower-level models and tests (that describe simplified components and/or isolated physics). The various models; their inputs, parameters, and outputs; experimental data; and various sources of model error are connected through a Bayesian network. The results of calibration, verification, and validation with respect to each individual model are integrated using the principles of conditional probability and total probability and propagated through the Bayesian network in order to quantify the overall system-level prediction uncertainty. For Type-II model, the relevance of each lower-level output to the system-level quantity of interest is quantified by comparing Sobol indices, thus measuring the extent to which a lower-level test represents the characteristics of the system so that the calibration results can be reliably used in the system level. The proposed methodology is illustrated with numerical examples that deal with heat conduction and structural dynamics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Urbina, A., Mahadevan, S., Paez, T.L.: Quantification of margins and uncertainties of complex systems in the presence of aleatoric and epistemic uncertainty. Reliab. Eng. Syst. Saf. 96(9), 1114–1125 (2011)

    Article  Google Scholar 

  2. Sankararaman, S., Ling, Y., Mahadevan, S.: Uncertainty quantification and model validation of fatigue crack growth prediction. Eng. Fract. Mech. 78(7), 1487–1504 (2011)

    Article  Google Scholar 

  3. Sankararaman, S., Mahadevan, S.: Model parameter estimation with imprecise and unpaired data. Inverse Probl. Sci. Eng. 20(7), 1017–1041 (2012)

    Article  MathSciNet  Google Scholar 

  4. Sankararaman, S., Mahadevan, S.: Model validation under epistemic uncertainty. Reliab. Eng. Syst. Saf. 96(9), 1232–1241 (2011)

    Article  Google Scholar 

  5. Ling, Y., Mahadevan, S.: Quantitative model validation techniques: new insights. Reliab. Eng. Syst. Saf. 111, 217–231 (2013)

    Article  Google Scholar 

  6. Sankararaman, S., Mahadevan, S.: Likelihood-based representation of epistemic uncertainty due to sparse point data and/or interval data. Reliab. Eng. Syst. Saf. 96(7), 814–824 (2011)

    Article  Google Scholar 

  7. Jeffrey, H.: Theory of Probability. Oxford University Press, Oxford (1998)

    Google Scholar 

  8. Sankararaman, S., Ling, Y., Shantz, C., Mahadevan, S.: Uncertainty quantification in fatigue crack growth prognosis. Int. J. Progn. Heal. Manag. 2(1), 15 (2011)

    Google Scholar 

  9. Sankararaman, S., Ling, Y., Shantz, C., Mahadevan, S.: Inference of equivalent initial flaw size under multiple sources of uncertainty. Int. J. Fatigue 33(2), 75–89 (2011)

    Article  Google Scholar 

  10. Sankararaman, S., Ling, Y., Mahadevan, S.: Statistical inference of equivalent initial flaw size with complicated structural geometry and multi-axial variable amplitude loading. Int. J. Fatigue 32(10), 1689–1700 (2010)

    Article  Google Scholar 

  11. Sankararaman, S., McLemore, K., Mahadevan, S., Bradford, S.C., Peterson, L.D.: Test resource allocation in hierarchical systems using bayesian networks. AIAA J. 51(3), 537–550 (2013)

    Article  Google Scholar 

  12. Mullins, J., Li, C., Mahadevan, S., Urbina, A.: Optimal Selection of Calibration and Validation Test Samples under Uncertainty. In: IMAC XXXII, Orlando, pp. 391–401 (2014)

    Google Scholar 

  13. Li, C., Mahadevan, S.: Sensitivity Analysis for Test Resource Allocation. In: IMAC XXXIII, Orlando (2015)

    Book  Google Scholar 

  14. Mullins, J., Li, C., Sankararaman, S., Mahadevan, S.: Probabilistic integration of validation and calibration results for prediction level uncertainty quantification: application to structural dynamics. In: 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Boston (2013)

    Google Scholar 

  15. Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. 63(3), 425–464 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  16. Sankararaman, S., Mahadevan, S.: Comprehensive framework for integration of calibration, verification and validation. In: 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, Honolulu, pp. 1–12 (2012)

    Google Scholar 

  17. Li, C., Mahadevan, S.: Uncertainty quantification and output prediction in multi-level problems. In: 16th AIAA Non-Deterministic Approaches Conference, National Harbor (2014)

    Google Scholar 

  18. Li, C., Mahadevan, S.: Role of calibration, validation, and relevance in multi-level uncertainty integration. Reliab. Eng. Syst. Saf. 148, 32–43 (2016)

    Article  Google Scholar 

  19. Sankararaman, S., Mahadevan, S.: Likelihood-based approach to multidisciplinary analysis under uncertainty. J. Mech. Des. 134(3), 031008 (2012)

    Article  Google Scholar 

  20. Babuska, I., Oden, J.T.T.: Verification and validation in computational engineering and science: basic concepts. Comput. Methods Appl. Mech. Eng. 193(36–38), 4057–4066 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  21. Roy, C.J.: Review of code and solution verification procedures for computational simulation. J. Comput. Phys. 205(1), 131–156 (2005)

    Article  MATH  Google Scholar 

  22. AIAA: Guide for the verification and validation of computational fluid dynamics simulations. American Institute of Aeronautics and Astronautics (AIAA), no. AIAA G-077-1998 (1998)

    Google Scholar 

  23. Defense Modelling and Simulation Office, Verification, Validation, and accreditation (VV & A) recommend practices guide, Alexandia (1998)

    Google Scholar 

  24. Oberkampf, W.L., Blottner, F.G.: Issues in computational fluid dynamics code verification and validation. AIAA J 36(5), 687–695 (1998)

    Article  Google Scholar 

  25. Oberkampf, W.L., Trucano, T.G.G.: Verification and validation in computational fluid dynamics. Prog. Aerosp. Sci. 38(3), 209–272 (2002)

    Article  Google Scholar 

  26. Benay, R., Chanetz, B., Delery, J.: Code verification/validation with respect to experimental data banks. Aerosp. Sci. Technol. 7(4), 239–262 (2003)

    Article  Google Scholar 

  27. Roy, C.J., Oberkampf, W.L.: A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Comput. Methods Appl. Mech. Eng. 200(25), 2131–2144 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Roache, P.J.: Verification of codes and calculations. Aiaa J. 36(5), 696–702 (1998)

    Article  Google Scholar 

  29. Roache, P.J.: Verification and Validation in Computational Science and Engineering. Hermosa Publishers, Albuquerque (1998)

    Google Scholar 

  30. Oberkampf, W.L., Trucano, T.G., Hirsch, C.: Verification, validation, and predictive capability in computational engineering and physics. Appl. Mech. Rev. 57(5), 345–384 (2004)

    Article  Google Scholar 

  31. Roy, C.J., McWherter-Payne, M.A., Oberkampf, W.L.: Verification and validation for laminar hypersonic flowfields, part 1: verification. Aiaa J. 41(10), 1934–1943 (2003)

    Article  Google Scholar 

  32. Rebba, R., Mahadevan, S., Huang, S.: Validation and error estimation of computational models. Reliab. Eng. Syst. Saf. 91(10–11), 1390–1397 (2006)

    Article  Google Scholar 

  33. Liang, B., Mahadevan, S.: Error and uncertainty quantification and sensitivity analysis in mechanics computational models. Int. J. Uncertain. Quantif. 1(2), 147–161 (2011)

    Article  MathSciNet  Google Scholar 

  34. Rangavajhala, S., Sura, V.S., Hombal, V.K., Mahadevan, S.: Discretization error estimation in multidisciplinary simulations. AIAA J. 49(12), 2673–2683 (2011)

    Article  Google Scholar 

  35. Ferziger, J., Peric, M.: Computational Methods for Fluid Dynamics. Springer, Berlin (1996)

    Book  MATH  Google Scholar 

  36. Ainsworth, M., Oden, J.T.T.: A posteriori error estimation in finite element analysis. Comput. Methods Appl. Mech. Eng. 142(1–2), 1–88 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  37. Oberkampf, W.L., DeLand, S.M., Rutherford, B.M., Diegert, K.V., Alvin, K.F.: Error and uncertainty in modeling and simulation. Reliab. Eng. Syst. Saf. 75(3), 333–357 (2002)

    Article  Google Scholar 

  38. Haldar, A., Mahadevan, S.: Probability, Reliability, and Statistical Methods in Engineering Design. John Wiley, New York (2000)

    Google Scholar 

  39. Ghanem, R., Spanos, P.D.: Polynomial chaos in stochastic finite elements. J. Appl. Mech. 57(1)(89), 197–202 (1990)

    Google Scholar 

  40. Buhmann, M.D.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge university press, Cambridge/New York (2003)

    Book  MATH  Google Scholar 

  41. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT, Cambridge (2006)

    MATH  Google Scholar 

  42. Richards, S.A.: Completed Richardson extrapolation in space and time. Commun. Numer. Methods Eng. 13(7), 573–582 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  43. Xu, P., Su, X., Mahadevan, S., Li, C., Deng, Y.: A non-parametric method to determine basic probability assignment for classification problems. Appl. Intell. 41(3), 681–693 (2014)

    Article  Google Scholar 

  44. Babuska, I., Rheinboldt, W.C.: A posteriori error estimates for the finite element method. Int. J. Numer. Methods Eng. 12(10), 1597–1615 (1978)

    Article  MATH  Google Scholar 

  45. Demkowicz, L., Oden, J.T., Strouboulis, T.: Adaptive finite elements for flow problems with moving boundaries. part I: variational principles and a posteriori estimates. Comput. Methods Appl. Mech. Eng. 46(2), 217–251 (1984)

    Google Scholar 

  46. Rasmussen, C.E.: Evaluation of Gaussian processes and other methods for non-linear regression. PhD dissertation, University of Toronto, 1996

    Google Scholar 

  47. Rasmussen, C.E.: The infinite Gaussian mixture model. In: NIPS, Denver, vol. 12, pp. 554–560 (1999)

    Google Scholar 

  48. Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., VonLuxburg, U., Ratsch, G. (eds.) Advanced Lectures on Machine Learning, vol. 3176, pp. 63–71 (2004)

    Google Scholar 

  49. Santner, T.J., Williams, B.J., Notz, W.I.: The design and analysis of computer experiments. Springer, Dordrecht/New York (2013)

    MATH  Google Scholar 

  50. Bichon, B.J., Eldred, M.S., Swiler, L.P., Mahadevan, S., McFarland, J.M.: Efficient global reliability analysis for nonlinear implicit performance functions. Aiaa J. 46(10), 2459–2468 (2008)

    Article  Google Scholar 

  51. McFarland, J.M.: Uncertainty Analysis for Computer Simulations throuth Validation and Calibraion. Vanderbilt University, Nashville (2008)

    Google Scholar 

  52. Cressie, N.: Spatial Statistics. John Wiley, New York (1991)

    MATH  Google Scholar 

  53. Chiles, J.-P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty, vol. 344. Wiley-Interscience, New York (1999)

    Book  MATH  Google Scholar 

  54. Wackernagel, H.: Multivariate Geostatistics: An Introduction with Applications. Springer, Berlin/New York (2003)

    Book  MATH  Google Scholar 

  55. Trucano, T.G., Swiler, L.P., Igusa, T., Oberkampf, W.L., Pilch, M.: Calibration, validation, and sensitivity analysis: what’s what. Reliab. Eng. Syst. Saf. 91(10–11), 1331–1357 (2006)

    Article  Google Scholar 

  56. Seber, G.A.F., Wild, C.J.: Nonlinear Regression. Wiley, New York (1989)

    Book  MATH  Google Scholar 

  57. Edwards, A.W.F.: Likelihood. Cambridge University Press, Cambridge, UK (1972)

    MATH  Google Scholar 

  58. Pawitan, Y.: In all likelihood: statistical modelling and inference using likelihood. Oxford University Press, Oxford/New York (2001)

    MATH  Google Scholar 

  59. Leonard, T., Hsu, J.: Bayesian Methods. Cambridge University Press, Cambridge (2001)

    MATH  Google Scholar 

  60. Lee, P.: Bayesian Statistics, 3rd edn. Arnold, London (2004)

    MATH  Google Scholar 

  61. Malinverno, A., Briggs, V.A.: Expanded uncertainty quantification in inverse problems: hierarchical Bayes and empirical Bayes. Geophysics 69(4), 1005–1016 (2004)

    Article  Google Scholar 

  62. Park, I., Amarchinta, H.K., Grandhi, R.V.: A Bayesian approach for quantification of model uncertainty. Reliab. Eng. Syst. Saf. 95(7), 777–785 (2010)

    Article  Google Scholar 

  63. Oliver, T.A., Moser, R.D.: Accounting for uncertainty in the analysis of overlap layer mean velocity models. Phys. Fluids 24(7), 075108 (2012)

    Article  Google Scholar 

  64. Arendt, P.D., Apley, D.W., Chen, W.: Quantification of model uncertainty: calibration, model discrepancy, and identifiability. J. Mech. Des. 134(10), 100908 (2012)

    Article  Google Scholar 

  65. Ling, Y., Mullins, J.G., Mahadevan, S.: Options for the inclusion of model discrepancy in Bayesian calibration. In: 16th AIAA Non-Deterministic Approaches Conference, National Harbor. American Institute of Aeronautics and Astronautics (2014)

    Google Scholar 

  66. Liu, F., Bayarri, M.J., Berger, J.O.: Modularization in Bayesian analysis, with emphasis on analysis of computer models. Bayesian Anal. 4(1), 119–150 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  67. Sankararaman, S.: Uncertainty Quantification and Integration in Engineering Systems. Vanderbilt University, Nashville (2012)

    Google Scholar 

  68. Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Markov chain Monte Carlo in practice. Chapman and Hall, London (1996)

    MATH  Google Scholar 

  69. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    Article  Google Scholar 

  70. Gilks, W.R., Wild, P.: Adaptive rejection sampling for Gibbs sampling. Appl. Stat. 41(2), 337–348 (1992)

    Article  MATH  Google Scholar 

  71. Neal, R.M.: Slice sampling. Ann. Stat. 31(3), 705–767 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  72. American Society of Mechanical Engineers: Guide for Verification and Validation in Computational Solid Mechanics, p. 53. American Society of Mechanical Engineers, New York (2006)

    Google Scholar 

  73. Coleman, H.W., Stern, F.: Uncertainties and CFD code validation. J. Fluids Eng. Asme 119(4), 795–803 (1997)

    Article  Google Scholar 

  74. Oberkampf, W.L., Barone, M.F.: Measures of agreement between computation and experiment: validation metrics. J. Comput. Phys. 217(1), 5–36 (2006)

    Article  MATH  Google Scholar 

  75. Ferson, S., Oberkampf, W.L., Ginzburg, L.: Model validation and predictive capability for the thermal challenge problem. Comput. Methods Appl. Mech. Eng. 197(29–32), 2408–2430 (2008)

    Article  MATH  Google Scholar 

  76. Hills, R.G., Leslie, I.H.: Statistical validation of engineering and scientific models: validation experiments to application. Sandia National Labs., Albuquerque/Livermore (2003).

    Book  Google Scholar 

  77. Urbina, A., Paez, T.L., Hasselman, T.K., Wathugala, G.W., Yap, K.: Assessment of model accuracy relative to stochastic system behavior. In: Proceedings of 44th AIAA Structures, Structural Dynamics, Materials Conference, Norfolk, pp. 7–10 (2003)

    Google Scholar 

  78. Gelfand, A.E., Dey, D.K.: Bayesian model choice: asymptotics and exact calculations. J. R. Stat. Soc. Ser. B-Methodol. 56(3), 501–514 (1994)

    MathSciNet  MATH  Google Scholar 

  79. Geweke, J.: Bayesian model comparison and validation. Am. Econ. Rev. 97(2), 60–64 (2007)

    Article  MathSciNet  Google Scholar 

  80. Zhang, R.X., Mahadevan, S.: Bayesian methodology for reliability model acceptance. Reliab. Eng. Syst. Saf. 80(1), 95–103 (2003)

    Article  Google Scholar 

  81. Mahadevan, S., Rebba, R.: Validation of reliability computational models using Bayes networks. Reliab. Eng. Syst. Saf. 87(2), 223–232 (2005)

    Article  Google Scholar 

  82. Rebba, R., Mahadevan, S.: Computational methods for model reliability assessment. Reliab. Eng. Syst. Saf. 93(8), 1197–1207 (2008)

    Article  Google Scholar 

  83. Sankararaman, S., Mahadevan, S.: Assessing the reliability of computational models under uncertainty. In: 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Boston, pp. 1–8 (2013)

    Google Scholar 

  84. Thacker, B.H., Paez, T.L.: A simple probabilistic validation metric for the comparison of uncertain model and test results. In: ASME Verification and Validation Symposium, Las Vegas (2013)

    Google Scholar 

  85. Liu, Y., Chen, W., Arendt, P., Huang, H.-Z.: Toward a better understanding of model validation metrics. J. Mech. Des. 133(7), 071005 (2011)

    Article  Google Scholar 

  86. Roache, P.J.: Fundamentals of Verification and Validation. Hermosa Press, Socorro (2009)

    Google Scholar 

  87. Oberkampf, W.L., Roy, C.C.J.: Verification and Validation in Scientific Computing. Cambridge University Press, New York (2010)

    Book  MATH  Google Scholar 

  88. O’Hagan, A.: Fractional Bayes Factors for Model Comparison. J. R. Stat. Soc. 57(1), 99–138 (1995)

    MathSciNet  MATH  Google Scholar 

  89. Jiang, X., Mahadevan, S.: Bayesian risk-based decision method for model validation under uncertainty. Reliab. Eng. Syst. Saf. 92(6), 707–718 (2007)

    Article  Google Scholar 

  90. Cha, S.: Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Model. METHODS Appl. Sci. 1(4) (2007)

    Google Scholar 

  91. De Maesschalck, R., Jouan-Rimbaud, D., Massart, D.L.: The Mahalanobis distance. Chemom. Intell. Lab. Syst. 50(1), 1–18 (2000)

    Article  Google Scholar 

  92. Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis: The Primer. John Wiley, Chichester (2008)

    MATH  Google Scholar 

  93. Sobol’, I.M.: Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 55(1–3), 271–280 (2001)

    Google Scholar 

  94. Li, C., Mahadevan, S.: Global sensitivity analysis for system response prediction using auxiliary variable method. In: 17th AIAA Non-Deterministic Approaches Conference, Kissimmee (2015)

    Google Scholar 

  95. Li, C., Mahadevan, S.: Relative contributions of aleatory and epistemic uncertainty sources in time series prediction. Int. J. Fatigue 82, 474–486 (2016)

    Article  Google Scholar 

  96. Singhal, A.: Modern information retrieval: a brief overview. IEEE Data Eng. Bull. 24(4), 35–43 (2001)

    Google Scholar 

  97. Van Horn, K.S.: Constructing a logic of plausible inference: a guide to Cox’s theorem. Int. J. Approx. Reason. 34(1), 3–24 (2003)

    Article  MATH  Google Scholar 

  98. Sankararaman, S., Mahadevan, S.: Integration of model verification, validation, and calibration for uncertainty quantification in engineering systems. Reliab. Eng. Syst. Saf. 138, 194–209 (2015)

    Article  MATH  Google Scholar 

  99. Li, C., Mahadevan, S.: Uncertainty quantification and integration in multi-level problems. In: IMAC XXXII, Orlando, vol. 3 (2014)

    Google Scholar 

  100. Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat. 27(3), 832–837 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  101. Red-Horse, J.R.R., Paez, T.L.L.: Sandia National Laboratories validation workshop: Structural dynamics application. Comput. Methods Appl. Mech. Eng. 197(29–32), 2578–2584 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  102. Chopra, A.K.: Dynamics of Structures: Theory and Applications to Earthquake Engineering, 4th edn. Prentice Hall, Englewood Cliffs (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Civil and Environmental Engineering, Vanderbilt University, Nashville, TN, USA

    Sankaran Mahadevan & Chenzhao Li

  2. NASA Ames Research Center, SGT Inc., Moffett Field, CA, USA

    Shankar Sankararaman

Authors
  1. Sankaran Mahadevan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shankar Sankararaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chenzhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sankaran Mahadevan .

Editor information

Editors and Affiliations

  1. Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA

    Roger Ghanem

  2. Los Alamos National Laboratory, Los Alamos, New Mexico, USA

    David Higdon

  3. California Institute of Technology , Pasadena, California, USA

    Houman Owhadi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this entry

Cite this entry

Mahadevan, S., Sankararaman, S., Li, C. (2016). Multilevel Uncertainty Integration. In: Ghanem, R., Higdon, D., Owhadi, H. (eds) Handbook of Uncertainty Quantification. Springer, Cham. https://doi.org/10.1007/978-3-319-11259-6_8-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-11259-6_8-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Online ISBN: 978-3-319-11259-6

  • eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation