Abstract

Computer system models provide detailed answers to system performance. For a given system configuration, a system model estimates the cycles per instruction that the system would incur while running a given workload. In addition, it estimates the proportion of time that is spent in different parts of the system and other related metrics such as bus utilizations.

Consider those inputs to a system model that are estimated with uncertainty. Examples include cache miss rates that are obtained via trace-driven cache simulation and also sometimes by extrapolating beyond the simulation domain. Errors incurred during the measurement and fitting processes are propagated to the system model outputs. On the other hand, other inputs such as hardware latencies are known precisely.

In this paper we propose several measures of uncertainty of system model outputs when it stems from uncertainty in the inputs. Some of these measures are based on sensitivity of an output to the inputs. We propose ways of defining and determining these sensitivities and turning them into uncertainty measures. Other measures are based on sampling schemes.

Additionally, we determine uncertainty measures for the system model outputs over a wide range of inputs covering large system design spaces. This is done by first selecting a set of input configurations based on an experimental design methodology where the uncertainty measures are determined. Then these data are used to interpolate the uncertainty measure function over the rest of the input space. We quantitatively characterize each input’s contribution to the output uncertainty over the input’s entire range. We also propose ways that call attention to high output uncertainty regions in the input space.

The methodology is illustrated on system models developed at Sun Microsystems Laboratories. The particular goal of the performance analysis is a design of level two caches.

Keywords: Error analysis; Measurement; Evaluation; Modeling; Simulation of multiple-processor systems; Spline and piecewise polynomial approximation; Nonparametric statistics; Experimental design; Queuing theory

Article Outline

1. Introduction

2. Output uncertainty estimation for a general system model

2.1. Uncertainty under a stochastic parameter model

2.2. Sensitivity based uncertainty measure

2.3. Uncertainty under a deterministic parameter model

3. Output uncertainty for a broad range of system configurations

3.1. Broad range sensitivity based measure

4. System model examples

5. Conclusion

Acknowledgements

Appendix A. Partial derivative estimation via a local polynomial model

References

Vitae