Skip to main content
SpringerLink
Log in

Accurate predictions from noisy data: replication versus exploration with applications to structural failure

  • RESEARCH PAPER
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

This paper investigates an effective test strategy for structural failure criterion characterization. The goals include identification of potential failure modes and a better approximation of failure boundary, e.g., failure load mapping with respect to geometry and load conditions. We typically replicate and test the same structural configuration several times in order to deal with noisy observation. However, our study shows that replication is not necessarily needed, because of the smoothing effect of surrogate models, and we show that exploring with as many different configurations as possible is more important. We illustrate the failure criterion characterization with two structural examples with various surrogate models, including polynomial response surface (PRS), support vector regression (SVR) and Gaussian process regression (GPR). We also examine the treatment of replicated test data for surrogate fitting. While fitting to all replicated test data works well for GPR, fitting only to the mean values of the replicated data helps a not-well-tuned surrogate (SVR in this paper) by compensating for the proneness to overfitting. When the noise level is significant as compared to the error due to surrogate modeling, a denser matrix might be prone to overfitting for GPR and SVR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  • Basudhar A (2012) Selection of anisotropic kernel parameters using multiple surrogate information. Paper presented at the 14th AIAA/ISSMO multidisciplinary analysis and optimization conference, Indianapolis, Indiana, 17–19 September

  • Chaudhuri A, Haftka R T, Ifju P, Villanueva D, Chang K, Rue J, Tyler C, Schmitz T (2013) Experimental optimization and uncertainty quantification of flapping wing of a micro air vehicle. Paper presented at the 10th World congress on structural and multidisciplinary optimization, Orlando, FL, May 20–24

  • Cherkassky V, Ma YQ (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17(1):113–126. doi:10.1016/S0893-6080(03)00169-2

    Article  MATH  Google Scholar 

  • Cherkassky V, Mulier F (2006) Learning from data: concepts, theory, and methods. Wiley, Hoboken

    Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087. doi:10.1115/1.1897403

    Article  Google Scholar 

  • Department of Defense (2002a) Department of Defense Handbook, Composite materials handbook, vol 1, Chapter 8 MIL-HDBK-17-3F

  • Department of Defense (2002b) Department of Defense Handbook, Composite materials handbook, vol 3, Chapter 4 MIL-HDBK-17-3F

  • Eldred MS, Giunta AA, Wojtkiewicz SF, Trucano TG (2002) Formulations for surrogate-based optimization under uncertainty. Paper presented at the 9th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Atlanta, GA, 4–6 September

  • Elfving G (1952) Optimum allocation in linear regression theory. Ann Math Stat 23(2):255–262. doi:10.1214/aoms/1177729442

    Article  MATH  MathSciNet  Google Scholar 

  • Forrester AIJ, Keane A J (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79

    Article  Google Scholar 

  • Forrester A I J, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley

  • Giunta A A, Dudley J M, Narducci R, Grossman B, Haftka R T, Mason W H, Watson L T (1994) Noisy aerodynamic response and smooth approximations in HSCT design. In: Paper presented at the 5th symposium on multidisciplinary analysis and optimization, Panama City Beach, FL, September 7–9

  • Giunta AA, Mcfarland JM, Swiler LP, Eldred MS (2006) The promise and peril of uncertainty quantification using response surface approximations. Struct Infrastruct E 2(3–4):175–189. doi:10.1080/15732470600590507

    Article  Google Scholar 

  • Giunta A A, Watson L T (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. Paper presented at the 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, St. Louis, MO, September 2–4

  • Goel T, Haftka RT, Shyy W, Watson LT (2008) Pitfalls of using a single criterion for selecting experimental designs. Int J Numer Methods Eng 75(2):127–155. doi:10.1002/Nme.2242

    Article  MATH  Google Scholar 

  • Gunn SR (1998) Support vector machines for classification and regression. University of Southampton

  • Hawkins D M (2004) The problem of overfitting. J Chem Inf Comput Sci 44(1):1–12. doi:10.1021/ci0342472

    Article  Google Scholar 

  • Ilcewicz L, Murphy B (2005) Safety & certification initiatives for composite airframe structure. Paper presented at the 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference, Austin, Texas, April 18–21

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13

    Article  Google Scholar 

  • Jin R, Du X, Chen W (2003) The use of metamodeling techniques for optimization under uncertainty. Struct Multidiscip Optim 25(2):99–116. doi:10.1007/s00158-002-0277-0

    Article  Google Scholar 

  • Jordaan E M, Smits G F (2002) Estimation of the regularization parameter for support vector regression. Paper presented at the World conference on computational intelligence 2002, Honolulu, Hawaii, May 12–17

  • Kiefer J (1961) Optimum designs in regression problems .2. Ann Math Stat 32(1):298–325. doi:10.1214/aoms/1177705160

    Article  MATH  MathSciNet  Google Scholar 

  • Kim C, Choi KK (2008) Reliability-based design optimization using response surface method with prediction interval estimation. J Mech Des 130(12):121401–121412. Artn 121401. doi:10.1115/1.2988476

    Article  Google Scholar 

  • Marshall Rouse DCJ, McGowan DM, Bush HG, Allen Waters W (2005) Utilization of the building-block approach in structural mechanics research. Paper presented at the 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics & materials conference, Austin, Texas, 18–21 April

  • MATLAB and Statistics Toolbox Release (2012) The MathWorks, Inc., Natick, Massachusetts

  • Myers R H, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York

    MATH  Google Scholar 

  • Papila M, Haftka RT (2000) Response surface approximations: noise, error repair, and modeling errors. AIAA J 38(12):2336–2343. doi:10.2514/2.903

    Article  Google Scholar 

  • Picheny V (2009) Improving accuracy and compensating foruncertainty in surrogate modeling. University of Florida, Gainesville

    Google Scholar 

  • Picheny V, Ginsbourger D, Richet Y, Caplin G (2013a) Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics 55(1):2–13. doi:10.1080/00401706.2012.707580

    Article  MathSciNet  Google Scholar 

  • Picheny V, Wanger T, Ginsbourger D (2013b) A benchmark of kriging-based infill criteria for noisy. Struct Multidiscip Optim (in press). doi:10.1007/s00158-013-0919-4

  • Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28. doi:10.1016/j.paerosci.2005.02.001

    Article  Google Scholar 

  • Rasmussen C E, Williams C K I (2005) Gaussian processes in machine learning. MIT Press, Cambridge

    Google Scholar 

  • Sacks J, Welch W J, Mitchell T J, Wynn H P (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

    Article  MATH  MathSciNet  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313. doi:10.1007/s00158-004-0389-9

    Article  Google Scholar 

  • Smola AJ, Scholkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222. doi:10.1023/B:Stco.0000035301.49549.88

    Article  MathSciNet  Google Scholar 

  • St. John RC, Draper NR (1975) D-optimality for regression design—review. Technometrics 17(1):15–23

    Article  MATH  MathSciNet  Google Scholar 

  • Vapnik V N (1998) Statistical learning theory. Wiley-Interscience, New York

    MATH  Google Scholar 

  • Viana FAC (2010) Surrogates toolbox user’s guide (Ver. 2.1). Gainesville

  • Viana FAC, Haftka RT, Hamman R, Venter G (2011) Efficient global optimization with experimental data: revisiting the paper helicopter design. Paper presented at the 52nd AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference, Denver, Colorado, 4–7 April

Download references

Acknowledgments

The authors greatly appreciate the support by National Science Foundation grant CMMI-1131103.

Author information

Authors and Affiliations

  1. Department of Mechanical & Aerospace Engineering, University of Florida, Gainesville, FL, USA

    Taiki Matsumura, Raphael T. Haftka & Nam H. Kim

Authors
  1. Taiki Matsumura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raphael T. Haftka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nam H. Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nam H. Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Matsumura, T., Haftka, R.T. & Kim, N.H. Accurate predictions from noisy data: replication versus exploration with applications to structural failure. Struct Multidisc Optim 51, 23–40 (2015). https://doi.org/10.1007/s00158-014-1115-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-014-1115-x

Keywords

Search

Navigation