Robert G. Sargent
Abstract
This paper is a tutorial paper on how to obtain point estimates and confidence intervals of steady state means of simulation output data. The methods of using replications, batch means, and regenerative cycles for obtaining these point and interval estimates are discussed in detail and are applied to a simple time-shared computer model to illustrate their use. A brief discussion is included on using time series methods to obtain these estimates. The advantages and disadvantages of the various methods are given, including specific recommendations as to when certain methods might be used.
- 1 Adiri, I.and Avi-Itzhak, B., "A Time-Sharing Queue With a Finite Number of Customers", Journal of ACM, Vol. 16, No. 2, April 1969. Google Scholar
Digital Library
- 2 Anderson, H.A. and Sargent, R.G., "A Statistical Evaluation of the Scheduler of an Experimental Interactive Computing System", Statistical Computer Performance Evaluation, Academic Press, 1972.Google Scholar
- 3 Anderson, H.A. and Sargent, R.G., "Investigation into Scheduling for an Interactive Computing System," IBM Journal of Res. & Dev., Vol. 18, No. 2, March 1974.Google Scholar
- 4 Baskett, F., "Confidence Intervals for Simulation Results: A Case Study of Buffer Pool Performance", Proceedings of Computer Science & Statistics: 7th Annual Symposium on the Interface.Google Scholar
- 5 Box, G.E.P. and Jenkins, G.M., Time Series Analysis Forecasting and Control, Holden-Day, 1970. Google ScholarDigital Library
- 6 Conway, R.W., "Some Tactical Problems in Digital Simulation", Management Science, Vol. 10, No. 1, October 1973.Google Scholar
- 7 Crane, M.A. and Iglehart, D.L., "Simulating Stable Stochastic Systems, I: General Multiserver Queues", Journal of the ACM, Vol. 21, No. 1, January 1974. Google ScholarDigital Library
- 8 Crane, M.A. and Iglehart, D.L., "Simulating Stable Stochastic Systems, II: Markov Chains", Journal of the ACM, Vol. 21, No. 1, January 1974. Google ScholarDigital Library
- 9 Crane, M.A. and Iglehart, D.L., "Simulating Stable Stochastic Systems, III: Regenerative Processes and Discrete-Events Simulations", Operations Research, Vol. 23, No. 1, Jan.-Feb. 1975.Google Scholar
- 10 Crane, M.A. and Iglehart, D.L., "Simulating Stable Stochastic Systems, IV: Approximation Techniques", Management Science, Vol. 21, No. 11, July 1975.Google Scholar
- 11 Devor, R.E., Fisher, R.R. and Dessouky, M.F., "Analysis of Simulation Output Characteristics by Autoregressive-Integrated Moving Average Models", Technical Report, Department of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign, 1972.Google Scholar
- 12 Duket, S.D. and Pritsker, A.A.B., "Spectral Methods for Simulation Output", Technical Paper, Pritsker and Associates.Google Scholar
- 13 Emshoff, J.R. and Sisson, R.L., Design and Use of Computer Simulation Models, The Macmillen Co., 1970.Google Scholar
- 14 Fishman, G.S., "The Analysis of Simulation-Generated Time Series", Management Science, Vol. 13, No. 7, March 1967.Google Scholar
- 15 Fishman, G.S., Concepts and Methods in Discrete Event Digital Simulation, John Wiley & Sons, 1973.Google Scholar
- 16 Fishman, G.S., "Batch Means in Digital Simulation", Technical Report No. 75-7, Curriculum in Operations Research and System Analysis, University of North Carolina at Chapel Hill, 1975.Google Scholar
- 17 Hunt, A.W., "Statistical Evaluation and Verification of Digital Simulation Models Through Spectral Analysis", Ph.D. Dissertation, The University of Texas at Austin, 1970.Google Scholar
- 18 Iglehart, D.L., "Simulating Stable Stochastic Systems, V: Comparison of Ratio Estimators", Naval Res. Logist. Quart., Vol. 22, No. 3, Sept. 1975.Google Scholar
- 19 Iglehart, D.L., "Simulating Stable Stochastic Systems, VI: Quantile Estimation", Journal of ACM, Vol. 23, No. 2, April 1976. Google ScholarDigital Library
- 20 Jenkin, G.M. and Watts, D.G., Spectral Analysis and Its Application, Holden-Day, 1968.Google Scholar
- 21 KleiJnen, J.P.C., "Statistical Techniques in Simulation: Part I and Part II," Marcel Dekker, 1974 and 1975.Google Scholar
- 22 Lavenberg, S.S. and Slutz, D.R., "Introduction to Regenerative Simulation", IBM Journal of Res. & Dev., Vol. 19, No. 5, Sept. 1975.Google Scholar
- 23 Law, A.M., "A Comparison of Two Techniques for Determining the Accuracy of Simulation Output", Technical Report No. 75-11, Department of Industrial Engineering at Madison, 1975.Google Scholar
Index Terms
- Statistical analysis of simulation output data
Recommendations
Statistical analysis of simulation output data
ANSS '76: Proceedings of the 4th symposium on Simulation of computer systemsThis paper is a tutorial paper on how to obtain point estimates and confidence intervals of steady state means of simulation output data. The methods of using replications, batch means, and regenerative cycles for obtaining these point and interval ...
Jackknifed variance estimators for simulation output analysis
WSC '15: Proceedings of the 2015 Winter Simulation ConferenceWe develop new point estimators for the variance parameter of a steady-state simulation process. The estimators are based on jackknifed versions of nonoverlapping batch means, overlapping batch means, and standardized time series variance estimators. ...
Asymptotic results for batch-variance methods in simulation output analysis
Special issue: Selected papers from the 31st international conference on computers & industrial engineeringWe consider the batching methods for estimating the variance of the sample variance based on steady-state correlated data. Intensive research has been devoted to the problem of estimating the variance of the sample mean, but little to the sample ...
Comments