Abstract
There is a good deal of miscommunication among experimenters and theorists about how to evaluate a theory that can be rejected by sufficient data, but may nevertheless be a useful approximation. A standard experimental design reports whether a general theory can be rejected on an informative test case. This paper, in contrast, reports an experiment designed to meaningfully pose the question: “how good an approximation does a theory provide on average.” It focuses on a class of randomly selected games, and estimates how many pairs of experimental subjects would have to be observed playing a previously unexamined game before the mean of the experimental observations would provide a better prediction than the theory about the behavior of a new pair of subjects playing this game. We call this quantity the model’s equivalent number of observations, and explore its properties.
References
Allais M. (1953) Le comportement de l’homme rationel devant le risque, critique des postulates et axiomes de l’ecole americaine. Econometrica 21, 503–546
Bereby-Meyer Y., Alvin E. Roth: Learning in noisy games: partial reinforcement and the sustainability of cooperation. Am. Econ. Rev. 96(4): 1029–1042 (2006)
Bolton G.E., Ockenfels A. (2000) ERC: a theory of equity, reciprocity and competition. Am Econ Rev 90, 166–193
Bornstein G., Winter E., Goren H. (1996) Experimental study of repeated team games. Eur J Polit Econ 12, 629–639
Brown J.N., Rosenthal R.W. (1990) Testing the minimax hypothesis: a re-examination of O’Neill’s game experiment. Econometrica 58, 1065–1081
Busemeyer J.R., Wang Y.M. (2000) Model comparisons and model selections based on generalization criterion methodology. J Math Psychol 44(1): 171–189
Bush R.R., Mosteller F. (1955) Stochastic Models for Learning. New York, Wiley
Camerer C.F., Ho T. (1999) Experience-weighted attraction in games. Econometrica 67, 827–874
Cheung Y.W., Friedman D. (1997) Individual learning in normal form games: some laboratory results. Games Econ Behavior 19, 46–76
Cohen J. (1994) The earth is round (p < 0.05). Am Psychol 49(12): 997–1003
Cooper D., Garvin S., Kagel J. (1997) Signalling and adaptive learning in an entry limit pricing game. Rand J Econ 28(4): 662–683
Edwards W. (1961) Probability learning in 1000 trials. J Exp Psychol 62, 385–394
Engle-Warnick J., Slonim R.L. (2006) Inferring repeated-game strategies from actions: evidence from trust game experiments. Econ Theory 28(3): 603–632
Erev I. (1998) Signal detection by human observers: a cutoff reinforcement learning model of categorization decisions under uncertainty. Psychol Rev 105(2): 280–311
Erev, I.: SAS programs for the parameter estimation and ENO computation of the NFP model. Retrieved September 4, 2006, from the following web address: http://iew3. technion.ac.il/Home/Users/erev.html#part2
Erev I., Barron G. (2005) On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychol Rev 112, 912–931
Erev I., Haruvy E. (2005) On the potential uses and current limitations of data driven learning models. J Math Psychol 49(5): 357–371
Erev I., Rapoport A. (1998) Magic, reinforcement learning and coordination in a market entry game. Games Econ Behav 23, 280–298
Erev I., Roth A.E. (1998) Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88(4): 848–881
Erev I., Bereby-Meyer Y., Roth A.E. (1999) The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models. J Econ Organ Behav 39, 111–128
Erev I., Roth A.E., Slonim S.L., Barron G. (2002) Predictive value and the usefulness of game theoretic models. Int J Forecast. 18(3): 359–368
Erev, I., Roth, A.E., Slonim, S.L., Barron, G.: Descriptive models as generators of forecasts with known equivalent number of observations (ENO). Working paper (2005)
Ert, E., Erev, I.: Replicated alternatives and the role of confusion, chasing, and regret in decisions from experience. J Behav Decis Making (in press) (2007)
Estes W.K. (1950) Toward a statistical theory of learning. Psychol Rev 57, 94–107
Feltovich N. (2000) Learning and equilibrium in an asymmetric information game: an experimental study. Econometrica 68(3): 605–642
Friedman, M.: Essays in Positive Economics. pp. 3–43 Chicago: University of Chicago Press (1953)
Fudenberg D., Levine D. (1998) Theory of Learning in Games. Cambridge, MIT Press
Goeree J.K., Holt C.A. (1999) Stochastic game theory: for playing games not just for doing theory. Proc Nat Acad Sci 96, 10564–10567
Granger C.W.J., Ramanathan R. (1984) Improved methods of combing forecasts. J Forecast 3, 197–204
Gupta S., Wilton P.C. (1987) Combination of forecasts: an extension. Manage Sci 33(3): 356–372
Hopkins E. (2002) Two competing models of learning in games. Econometrica 70, 2141–2166
Luce R.D. (1959) Individual Choice Behavior. New York, Wiley
Kagel J.H., Roth A.E. (1995) Handbook of Experimental Economics. Princeton, Princeton University Press
Nagel R., Tang F-F. (1998) An experimental study on the centipede game in normal form—an investigation on learning. J. Math. Psychol. 42(2): 256–284
Nash J.F. (1950) Equilibrium points in n-person games. Proc Nat Acad Sci 36, 48–49
Ochs J., Roth A.E. (1989) An experimental study of sequential bargaining. Am Econ Rev 79, 355–384
O’Neill B. (1987) Nonmetric test of the minimax theory of two-person zerosum games. Proc Nat Acad Sci USA 8, 2106–2109
O’Neill B. (1991) Comments on Brown and Rosenthal’s reexamination. Econometrica 59, 503–507
Roth, A.E.: Individual rationality as a useful approximation: comments on Tversky’s rational theory and constructive choice. In: Arrow, K., Colombatto, E., Perlman, M., Schmidt, C., (eds.) The Rational Foundations of Economic Behavior, pp. 198–202 New York: Macmillan, http://kuznets.fas.harvard.edu/ aroth/rational.html (1996)
Roth A.E. (2002) The economist as engineer: game theory, experimental economics and computation as tools of design economics. Econometrica 70(4): 1341–1378
Roth A.E., Erev I. (1995) Learning in extensive-form games: experimental data and simple dynamic models in intermediate term. Games Econ Behav, Spec Issue Nobel Symp 8, 164–212
Roth A.E., Malouf M.K. (1979) Game theoretical models and the role of information in bargaining. Psychol Rev 86, 574–594
Salmon T.C. (2001) An evaluation of econometric models of adaptive learning. Econometrica 69, 1597–1628
Sarin R., Vahid F. (2001) Predicting how people play games: a simple dynamic model of choice. Games Econ Behav 34, 104–122
Selten, R., Chmura, T.: Stationary concepts for experimental 2 × 2 games. Bonn Econ Discussion Paper (2005)
Siegel S., Siegel S., Andrews J.M. (1964) Choice, Strategy, and Utility. New York, McGraw-Hill
Slonim, R., Roth, A., Erev, I.: Regularities in play of zero sum games with unique mixed strategy equilibria. Working Paper (2006)
Suppes, P., Atkinson, R.C.: Markov Learning Models for Multiperson Interactions. Stanford: Stanford University Press (1960)
Tversky A. (1996) Rational theory and constructive choice. In: Arrow K., Colombatto E., Perlman M., Schmidt C. (eds) The Rational Foundations of Economic Behavior. New York, Macmillan, pp. 185–197
von Neumann J. (1928) Zur theorie der Gesellschaftespiele. Math Ann 100, 295–320
Weibull, J.W.: Testing game theory, Stockholm School of Economics. Working Paper 382 (2000)
Weibull, J.W.: Testing game theory. In: Huck, S. (ed.) Advances in Understanding Strategic Behaviour: Game Theory, Experiments and Bounded Rationality. Essay in Honour of Werner Güth. pp. 85–104 Palgrave: MacMillan, (2004)
Wooders J., Shachat J.M. (2001) On the irrelevance of risk attitudes in repeated two-outcome Games. Games Econ Behav 34(2): 342–363
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by a grant from the U.S. National Science Foundation and the USA–Israel Binational Science Foundation. We are very grateful for helpful conversations with David Budescu, Jerry Busemeyer, Gary Chamberlain, Paul Feigin, Dave Krantz, Jack Porter, Tom Wallsten, Wolfgang Viechtbauer, and Richard Zeckhauser.
Rights and permissions
About this article
Cite this article
Erev, I., Roth, A.E., Slonim, R.L. et al. Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games. Economic Theory 33, 29–51 (2007). https://doi.org/10.1007/s00199-007-0214-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00199-007-0214-y