Skip to main content
Log in

Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games

Economic Theory Aims and scope Submit manuscript

Abstract

There is a good deal of miscommunication among experimenters and theorists about how to evaluate a theory that can be rejected by sufficient data, but may nevertheless be a useful approximation. A standard experimental design reports whether a general theory can be rejected on an informative test case. This paper, in contrast, reports an experiment designed to meaningfully pose the question: “how good an approximation does a theory provide on average.” It focuses on a class of randomly selected games, and estimates how many pairs of experimental subjects would have to be observed playing a previously unexamined game before the mean of the experimental observations would provide a better prediction than the theory about the behavior of a new pair of subjects playing this game. We call this quantity the model’s equivalent number of observations, and explore its properties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Allais M. (1953) Le comportement de l’homme rationel devant le risque, critique des postulates et axiomes de l’ecole americaine. Econometrica 21, 503–546

    Article  Google Scholar 

  • Bereby-Meyer Y., Alvin E. Roth: Learning in noisy games: partial reinforcement and the sustainability of cooperation. Am. Econ. Rev. 96(4): 1029–1042 (2006)

  • Bolton G.E., Ockenfels A. (2000) ERC: a theory of equity, reciprocity and competition. Am Econ Rev 90, 166–193

    Article  Google Scholar 

  • Bornstein G., Winter E., Goren H. (1996) Experimental study of repeated team games. Eur J Polit Econ 12, 629–639

    Article  Google Scholar 

  • Brown J.N., Rosenthal R.W. (1990) Testing the minimax hypothesis: a re-examination of O’Neill’s game experiment. Econometrica 58, 1065–1081

    Article  Google Scholar 

  • Busemeyer J.R., Wang Y.M. (2000) Model comparisons and model selections based on generalization criterion methodology. J Math Psychol 44(1): 171–189

    Article  Google Scholar 

  • Bush R.R., Mosteller F. (1955) Stochastic Models for Learning. New York, Wiley

    Google Scholar 

  • Camerer C.F., Ho T. (1999) Experience-weighted attraction in games. Econometrica 67, 827–874

    Article  Google Scholar 

  • Cheung Y.W., Friedman D. (1997) Individual learning in normal form games: some laboratory results. Games Econ Behavior 19, 46–76

    Article  Google Scholar 

  • Cohen J. (1994) The earth is round (p < 0.05). Am Psychol 49(12): 997–1003

    Article  Google Scholar 

  • Cooper D., Garvin S., Kagel J. (1997) Signalling and adaptive learning in an entry limit pricing game. Rand J Econ 28(4): 662–683

    Article  Google Scholar 

  • Edwards W. (1961) Probability learning in 1000 trials. J Exp Psychol 62, 385–394

    Article  Google Scholar 

  • Engle-Warnick J., Slonim R.L. (2006) Inferring repeated-game strategies from actions: evidence from trust game experiments. Econ Theory 28(3): 603–632

    Article  Google Scholar 

  • Erev I. (1998) Signal detection by human observers: a cutoff reinforcement learning model of categorization decisions under uncertainty. Psychol Rev 105(2): 280–311

    Article  Google Scholar 

  • Erev, I.: SAS programs for the parameter estimation and ENO computation of the NFP model. Retrieved September 4, 2006, from the following web address: http://iew3. technion.ac.il/Home/Users/erev.html#part2

  • Erev I., Barron G. (2005) On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychol Rev 112, 912–931

    Article  Google Scholar 

  • Erev I., Haruvy E. (2005) On the potential uses and current limitations of data driven learning models. J Math Psychol 49(5): 357–371

    Article  Google Scholar 

  • Erev I., Rapoport A. (1998) Magic, reinforcement learning and coordination in a market entry game. Games Econ Behav 23, 280–298

    Article  Google Scholar 

  • Erev I., Roth A.E. (1998) Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88(4): 848–881

    Google Scholar 

  • Erev I., Bereby-Meyer Y., Roth A.E. (1999) The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models. J Econ Organ Behav 39, 111–128

    Article  Google Scholar 

  • Erev I., Roth A.E., Slonim S.L., Barron G. (2002) Predictive value and the usefulness of game theoretic models. Int J Forecast. 18(3): 359–368

    Article  Google Scholar 

  • Erev, I., Roth, A.E., Slonim, S.L., Barron, G.: Descriptive models as generators of forecasts with known equivalent number of observations (ENO). Working paper (2005)

  • Ert, E., Erev, I.: Replicated alternatives and the role of confusion, chasing, and regret in decisions from experience. J Behav Decis Making (in press) (2007)

  • Estes W.K. (1950) Toward a statistical theory of learning. Psychol Rev 57, 94–107

    Article  Google Scholar 

  • Feltovich N. (2000) Learning and equilibrium in an asymmetric information game: an experimental study. Econometrica 68(3): 605–642

    Article  Google Scholar 

  • Friedman, M.: Essays in Positive Economics. pp. 3–43 Chicago: University of Chicago Press (1953)

  • Fudenberg D., Levine D. (1998) Theory of Learning in Games. Cambridge, MIT Press

    Google Scholar 

  • Goeree J.K., Holt C.A. (1999) Stochastic game theory: for playing games not just for doing theory. Proc Nat Acad Sci 96, 10564–10567

    Article  Google Scholar 

  • Granger C.W.J., Ramanathan R. (1984) Improved methods of combing forecasts. J Forecast 3, 197–204

    Article  Google Scholar 

  • Gupta S., Wilton P.C. (1987) Combination of forecasts: an extension. Manage Sci 33(3): 356–372

    Article  Google Scholar 

  • Hopkins E. (2002) Two competing models of learning in games. Econometrica 70, 2141–2166

    Article  Google Scholar 

  • Luce R.D. (1959) Individual Choice Behavior. New York, Wiley

    Google Scholar 

  • Kagel J.H., Roth A.E. (1995) Handbook of Experimental Economics. Princeton, Princeton University Press

    Google Scholar 

  • Nagel R., Tang F-F. (1998) An experimental study on the centipede game in normal form—an investigation on learning. J. Math. Psychol. 42(2): 256–284

    Article  Google Scholar 

  • Nash J.F. (1950) Equilibrium points in n-person games. Proc Nat Acad Sci 36, 48–49

    Article  Google Scholar 

  • Ochs J., Roth A.E. (1989) An experimental study of sequential bargaining. Am Econ Rev 79, 355–384

    Google Scholar 

  • O’Neill B. (1987) Nonmetric test of the minimax theory of two-person zerosum games. Proc Nat Acad Sci USA 8, 2106–2109

    Article  Google Scholar 

  • O’Neill B. (1991) Comments on Brown and Rosenthal’s reexamination. Econometrica 59, 503–507

    Article  Google Scholar 

  • Roth, A.E.: Individual rationality as a useful approximation: comments on Tversky’s rational theory and constructive choice. In: Arrow, K., Colombatto, E., Perlman, M., Schmidt, C., (eds.) The Rational Foundations of Economic Behavior, pp. 198–202 New York: Macmillan, http://kuznets.fas.harvard.edu/ aroth/rational.html (1996)

  • Roth A.E. (2002) The economist as engineer: game theory, experimental economics and computation as tools of design economics. Econometrica 70(4): 1341–1378

    Article  Google Scholar 

  • Roth A.E., Erev I. (1995) Learning in extensive-form games: experimental data and simple dynamic models in intermediate term. Games Econ Behav, Spec Issue Nobel Symp 8, 164–212

    Article  Google Scholar 

  • Roth A.E., Malouf M.K. (1979) Game theoretical models and the role of information in bargaining. Psychol Rev 86, 574–594

    Article  Google Scholar 

  • Salmon T.C. (2001) An evaluation of econometric models of adaptive learning. Econometrica 69, 1597–1628

    Article  Google Scholar 

  • Sarin R., Vahid F. (2001) Predicting how people play games: a simple dynamic model of choice. Games Econ Behav 34, 104–122

    Article  Google Scholar 

  • Selten, R., Chmura, T.: Stationary concepts for experimental 2 × 2 games. Bonn Econ Discussion Paper (2005)

  • Siegel S., Siegel S., Andrews J.M. (1964) Choice, Strategy, and Utility. New York, McGraw-Hill

    Google Scholar 

  • Slonim, R., Roth, A., Erev, I.: Regularities in play of zero sum games with unique mixed strategy equilibria. Working Paper (2006)

  • Suppes, P., Atkinson, R.C.: Markov Learning Models for Multiperson Interactions. Stanford: Stanford University Press (1960)

  • Tversky A. (1996) Rational theory and constructive choice. In: Arrow K., Colombatto E., Perlman M., Schmidt C. (eds) The Rational Foundations of Economic Behavior. New York, Macmillan, pp. 185–197

    Google Scholar 

  • von Neumann J. (1928) Zur theorie der Gesellschaftespiele. Math Ann 100, 295–320

    Article  Google Scholar 

  • Weibull, J.W.: Testing game theory, Stockholm School of Economics. Working Paper 382 (2000)

  • Weibull, J.W.: Testing game theory. In: Huck, S. (ed.) Advances in Understanding Strategic Behaviour: Game Theory, Experiments and Bounded Rationality. Essay in Honour of Werner Güth. pp. 85–104 Palgrave: MacMillan, (2004)

  • Wooders J., Shachat J.M. (2001) On the irrelevance of risk attitudes in repeated two-outcome Games. Games Econ Behav 34(2): 342–363

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alvin E. Roth.

Additional information

This research was supported by a grant from the U.S. National Science Foundation and the USA–Israel Binational Science Foundation. We are very grateful for helpful conversations with David Budescu, Jerry Busemeyer, Gary Chamberlain, Paul Feigin, Dave Krantz, Jack Porter, Tom Wallsten, Wolfgang Viechtbauer, and Richard Zeckhauser.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Erev, I., Roth, A.E., Slonim, R.L. et al. Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games. Economic Theory 33, 29–51 (2007). https://doi.org/10.1007/s00199-007-0214-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00199-007-0214-y

JEL Classification Numbers

Navigation