Steven M. Blackburn
Amer Diwan
Matthias Hauswirth
Peter F. Sweeney
ABSTRACT
We call ourselves 'computer scientists', but are we scientists? If we are scientists, then we must practice the scientific method. This includes a solid experimental evaluation. In our experience, our experimental methodology is ad hoc at best, and nonexistent at worst.
This workshop brings together experts from different areas of computer science to discuss, explore, and attempt to identify the principles of sound experimental evaluation
- ]]Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, B. Moss, Aashish Phansalkar, Darko Stefanovic, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. The dacapo benchmarks: java benchmarking development and analysis. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 169--190, New York, NY, USA, 2006. ACM. Google Scholar
Digital Library
- ]]Andy Georges, Dries Buytaert, and Lieven Eeckhout. Statistically rigorous java performance evaluation. In OOPSLA '07: Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications}, pages 57--76, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- ]]Todd Mytkowicz, Amer Diwan, Matthias Hauswirth, and Peter F. Sweeney. Producing wrong data without doing anything obviously wrong! In ASPLOS '09: Proceeding of the 14th international conference on Architectural support for programming languages and operating systems, pages 265--276, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- ]]Todd Mytkowicz, Amer Diwan, Matthias Hauswirth, and Peter F. Sweeney. Evaluating the accuracy of Java profilers. In PLDI '10: Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation, pages 187--197, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
Index Terms
- Workshop on experimental evaluation of software and systems in computer science (Evaluate 2010)
Recommendations
IWGS 2010 workshop report: The First ACM SIGSPATIAL International Workshop on Geostreaming (San Jose, California - November 2, 2010)
The ACM SIGSPATIAL International Workshop on Geostreaming (IWGS) was held for the first time in conjunction with the 18th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL GIS 2010). The workshop has ...
Comments