Skip to main content

Advertisement

SpringerLink
Log in

Ambivalent economizations: the case of value added modeling in teacher evaluation

  • Published:
Theory and Society Aims and scope Submit manuscript

Abstract

Research on economization processes is increasingly taking seriously the social and material processes through which various policy domains are transformed into economic problems and solutions. This article engages “Value Added Modeling” (VAM) in teacher evaluation systems as a case study in economization. VAM is a statistical technology for evaluating the effectiveness of schoolteachers using student test scores, which wrests authority for the determination of quality teaching away from education professionals and toward quantitative economic modelers. Mobilizing field theory, we trace a half century of changing relationships among economists, other academics, and various policy audiences (from media to philanthropists to state and federal government) in struggles to define education policy concerning teacher quality. We show that economization is a set of overlapping, sometimes contradictory processes that can take different forms: in this case, the spread of an “economic style of reasoning” or the establishment of “economic policy devices.” The case of VAM shows stages of economization in which processes first proceeded independently of one another, then interacted in contradictory ways, and finally, mutually reinforced one another. What primarily drove these interactions was the struggle for scientific capital within the economics discipline and the changing place of education policy and VAM within it. Ultimately, VAM’s original role as a policy device for evaluating and selecting individual teachers has foundered even as it has become an important tool for economists to accrue scientific capital and expand their style of reasoning to broader audiences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Of course, Çalışkan and Callon come from the Actor Network Theory (Latour 2005) tradition, and they are making a similar move with economic or proto-economic things that ANT has made with science. Thus, for them, economization is not only about economic conceptualization as mental representations but also the extension of networks and the embedding of practical and conceptual things within them.

  2. For an exception, see Livne (2019), who conceives of economization as a variety of processes aimed at establishing “regimes of valuation.”

  3. In Abbott’s discussion of hinges he prefers the notion of “ecology” to that of field (Abbott 2005a); we agree with others who have noted that while these are not identical concepts, they are broadly compatible for analytical purposes (Liu and Emirbayer 2016; de Souza Leão and Eyal 2019).

  4. https://web.archive.org/web/20130601000759/http:/obs.rc.fas.harvard.edu/chetty/value_added.html (accessed 18 February 2020).

References

  • Abbott, A. (2005a). Ecologies and fields. http://home.uchicago.edu/~aabbott/Papers/BOURD.pdf. Accessed 16 Sept 2020.

  • Abbott, A. (2005b). Linked ecologies: States and universities as environments for professions. Sociological Theory, 23(3), 245–274.

  • Abdulkadiroglu, A., Pathak, P. A., Schellenberg, J., & Walters, C. R. (2017). Do parents value school effectiveness? Working paper. 23912. Cambridge: National Bureau of Economic Research.

  • Adler, M. (2013). Findings vs. interpretation in ‘the long-term impacts of teachers’ by Chetty et al. Education Policy Analysis Archives, 21(10).

  • Allais, S. (2012). ‘Economics imperialism,’ education policy and educational theory. Journal of Education Policy, 27(2), 253–274.

  • Allen, M., & Palaich, R. (2000). In pursuit of quality teaching: Five key strategies for policymaking. Denver: Education Commission of the States.

  • American Statistical Association. (2014). ASA Statement on using value-added models for educational assessment. https://www.amstat.org/asa/files/pdfs/POLASAVAM-Statement.pdf. Accessed 16 Sept 2020.

  • Amrein-Beardsley, A. (2014). Rethinking value-added models in education: Critical perspectives on tests and assessment-based accountability. New York: Routledge.

  • Amrein-Beardsley, A., & Collins, C. (2012). The SAS Education Value-Added Assessment System (SAS[R] EVAAS[R]) in the Houston Independent School District (HISD): Intended and unintended consequences. Education Policy Analysis Archives, 20(12).

  • Amrein-Beardsley, A., & Holloway, J. (2017). Value-added models for teacher evaluation and accountability: Commonsense assumptions. Educational Policy, 33(3), 516–542.

  • Angrist, J. D., & Pischke, J.-S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24(2), 3–30.

  • Angrist, J., Hull, P. D., Pathak, P. A., & Walters, C. R. (2017). Leveraging lotteries for school value-added: Testing and estimation. The Quarterly Journal of Economics, 132(2), 871–919.

  • Appelbaum, B. (2019). The economists’ hour: False prophets, free markets, and the fracture of society (First ed.). New York: Little, Brown and Company.

  • Baker, E.L., Barton, P.E., Darling-Hammond, L., Haertel, E., Ladd, H.F., Linn, R.L., Ravitch, D., Rothstein, R., Shavelson, R.J., & Shepard, L.R. (2010). Problems with the use of student test scores to evaluate teachers. Briefing paper 278. Washington, D.C: Economic Policy Insitute.

  • Becker, G. S. (1964). Human capital: A theoretical and empirical analysis, with special reference to education. New York: National Bureau of Economic Research, distributed by Columbia University Press.

  • Bell, T. H. (1986). Education policy development in the Reagan administration. The Phi Delta Kappan, 67(7), 487–493.

  • Bergman, P., & Hill, M. (2015). The effects of making performance information public: Evidence from Los Angeles teachers and a regression discontinuity design. SSRN scholarly paper ID 2615769. Rochester: Social Science Research Network.

  • Berman, E. P. (2014). Not just neoliberalism: Economization in US science and technology policy. Science, Technology & Human Values, 39(3), 397–431.

  • Berman, E.P. (n.d.) Thinking like an economist: How economics became the language of U.S. public policy. Unpublished manuscript.

  • Biddle, J. E., & Hamermesh, D. S. (2017). Theory and measurement: Emergence, consolidation, and Erosion of a consensus. History of Political Economy, 49(supplement), 34–57.

  • Bourdieu, P. (1975). The specificity of the scientific field and the social conditions of the progress of reason. Social Science Information, 14(6), 19–47.

  • Bourdieu, P. (1993). The field of cultural production: Essays on art and literature. New York: Columbia University Press.

  • Bourdieu, P. (1996). The rules of art: Genesis and structure of the literary field. Stanford: Stanford University Press.

  • Bourdieu, P. (2004). Science of science and reflexivity. Cambridge: Polity.

  • Breslau, D. (2003). Economics invents the economy: Mathematics, statistics, and models in the work of Irving fisher and Wesley Mitchell. Theory and Society, 32(3), 379–411.

  • Brown, W. (2015). Undoing the demos: Neoliberalism’s stealth revolution. Cambridge: MIT Press.

  • Bruno, I., Didier, E., & Vitale, T. (2014). Statactivism: Forms of action between disclosure and affirmation. Partecipazione & Conflitto, 7(2), 199–220.

  • Burtless, G. T. (1996). Does money matter?: The effect of school resources on student achievement and adult success. Washington, DC: Brookings Institution Press.

  • Çalışkan, K., & Callon, M. (2009). Economization, part 1: Shifting attention from the economy towards processes of economization. Economy and Society, 38(3), 369–398.

  • Callon, M. (1998). Laws of the markets. Oxford; Malden: Blackwell Publishers.

  • Chetty, R., Friedman, J., & Rockoff, J. (2014a). Discussion of the American Statistical Association’s statement (2014) on using value-added models for educational assessment. http://www.equality-of-opportunity.org/assets/documents/teachers_discussion_asa.pdf. Accessed 16 Sept 2020.

  • Chetty, R., Friedman, J., & Rockoff, J. (2014b). Measuring the impacts of teachers I: Evaluating Bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632.

  • Chetty, R., Friedman, J., & Rockoff, J. (2014c). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679.

  • Cohodes, S., Setren, E., & Walters, C.R. (2019). Can successful schools replicate? Scaling up Boston’s charter school sector. Working paper 25796. Cambridge: National Bureau of Economic Research.

  • Cook, G. (2019). Raj Chetty’s American dream. The Atlantic. https://www.theatlantic.com/magazine/archive/2019/08/raj-chettys-american-dream/592804/. Accessed 16 Sept 2020.

  • Coyle, D. (2014). GDP : A brief but affectionate history. Princeton: Princeton University Press.

  • Currie, J., Kleven, H., & Zwiers, E. (2019). Technology and big data are changing economics: Mining text to track methods. Working paper 26715. Cambridge: National Bureau of Economic Research.

  • Dix, G. (2019). Microeconomic forecasting: Constructing commensurable futures of educational reforms. Social Studies of Science, 49(2), 180–207.

  • Ellison, S., (2014). Attack of the cyborgs: Economic imperialism and the human deficit in educational policy-making & research. Journal of Educational Controversy 8(1).

  • Epstein, S. (1995). The construction of lay expertise: AIDS activism and the forging of credibility in the reform of clinical trials. Science, Technology, & Human Values, 20(4), 408–437.

  • Espeland, W. N., & Stevens, M. L. (1998). Commensuration as a social process. Annual Review of Sociology, 24(1), 313–343.

  • Eyal, G. (2006). The disenchantment of the orient: Expertise in Arab affairs and the Israeli state. Stanford: Stanford University Press.

  • Eyal, G. (2013). Spaces between fields. In P. S. Gorski (Ed.), Bourdieu and hstorical analysis (pp. 158–182). Durham: Duke University Press.

  • Eyal, G., & Levy, M. (2013). Economic indicators as public interventions. History of Political Economy, 45(suppl_1), 220–253.

  • Felch, J. (2010). Who’s teaching L.A.’s kids? Los Angeles Times. https://www.latimes.com/archives/la-xpm-2010-aug-14-la-me-teachers-value-20100815-story.html. Accessed 16 Sept 2020.

  • Felch, J., & Song, J. (2010). U.S. schools chief endorses release of teacher data. Los Angeles Times. https://www.latimes.com/archives/la-xpm-2010-aug-16-lame-0817-teachers-react-20100817-story.html. Accessed 16 Sept 2020.

  • Fletcher, J.M., Horwitz, L.I., and Bradley, E., (2014). Estimating the value added of attending physicians on patient outcomes. Working paper. 20534. National Bureau of economic research.

  • Foucault, M. (2008). The birth of biopolitics : Lectures at the Collège de France, 1978-79. Basingstoke [England]; New York: Palgrave Macmillan.

  • Fourcade, M. (2016). Ordinalization: Lewis A. Coser memorial award for theoretical agenda setting 2014. Sociological Theory, 34(3), 175–195.

  • Fourcade, M., Ollion, E., & Algan, Y. (2015). The superiority of economists. Journal of Economic Perspectives, 29(1), 89–114.

  • García, E., & Weiss, E. (2019). U.S. schools struggle to hire and retain teachers. Economic Policy Institute. https://files.epi.org/pdf/164773.pdf. Accessed 16 Sept 2020.

  • Garcia-Parpet, M.-F. (2007). The social construction of a perfect market: The strawberry auction at Fontaines-en-Sologne. In D. MacKenzie, F. Muniesa, & L. Siu (Eds.), Do economists make markets? On the performativity of economics (pp. 20–53). Princeton: Princeton University Press.

  • Gates, B. (2012). For teachers, shame is no solution. The New York Times. https://www.nytimes.com/2012/02/23/opinion/for-teachers-shame-is-no-solution.html. Accessed 16 Sept 2020.

  • Grant, G. (1973). Shaping social policy: The politics of the Coleman report. Teachers College Record, 75(1), 17–54.

  • Greene, J.P. (2016). Evidence for the disconnect between changing test scores and changing later life outcomes. Education Next. https://www.educationnext.org/evidence-for-the-disconnect-between-changing-test-scores-and-changing-later-life-outcomes/. Accessed 27 April 2019.

  • Hanushek, E.A. (1971). Teacher characteristics and gains in student achievement: Estimation using micro data. The American Economic Review, 61(2), 280–288.

  • Hanushek, E.A. (1968). The education of negroes and whites. PhD dissertation, MIT.

  • Hanushek, E. A. (1981). Education policy research--an industry perspective. Economics of Education Review, 1(2), 193–223.

  • Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature, 24(3), 1141–1177.

  • Hanushek, E. A., & Rivkin, S. (2012). The distribution of teacher quality and implications for policy. Annual Review of Economics, 4, 131–157.

  • Hertel-Fernandez, A. (2019). State capture: How conservative activists, big businesses, and wealthy donors reshaped the American states--and the nation. New York: Oxford University Press.

  • Hirschman, D. (2016). Inventing the economy: Or, how we learned to stop worrying and love the GDP. PhD dissertation: University of Michigan.

  • Hirschman, D., & Berman, E. P. (2014). Do economists make policies? On the political effects of economics. Socio-Economic Review, 12(4), 779–811.

  • Holden, L., & Biddle, J. (2017). The introduction of human capital theory into education policy in the United States. History of Political Economy, 49(4), 537–574.

  • Horn, J., & Wilburn, D. (2013). The Mismeasure of education. Charlotte: Information Age Publishing.

  • Imberman, S. A., & Lovenheim, M. F. (2016). Does the market value value-added? Evidence from housing prices after a public release of school and teacher value-added. Journal of Urban Economics, 91, 104–121.

  • Kane, T.J., McCaffrey, D.F., Miller, T. & Staiger., D.O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment. Research Paper. MET Project. Seattle: Bill & Melinda Gates Foundation.

  • Kiker, B. F. (1966). The historical roots of the concept of human capital. Journal of Political Economy, 74(5), 481–499.

  • Kinsley, M. (1990). Bart for president. The New Republic. https://newrepublic.com/article/76699/michael-kinsley-bart-simpson-president. Accessed 16 Sept 2020.

  • Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Cambridge: Harvard University Press.

  • Latour, B. (2005). Reassembling the social: An introduction to actor-network-theory. In Oxford. New York: Oxford University Press.

  • Layton, L. (2015). Senate overwhelmingly passes new National Education Legislation. The Washington Post. https://www.washingtonpost.com/local/education/senate-overwhelmingly-passes-new-national-education-legislation/2015/12/09/be1b1f94-9d2a-11e5-a3c5-c77f2cc5a43c_story.html/. Accessed 16 Sept 2020.

  • Lazear, E. P. (2000). Economic imperialism. The Quarterly Journal of Economics, 115(1), 99–146.

  • Liu, S., & Emirbayer, M. (2016). Field and ecology. Sociological Theory, 34(1), 62–79.

  • Livne, R. (2019). Values at the end of life: The logic of palliative care. Cambridge: Harvard University Press.

  • Lovett, I. (2010). Los Angeles teacher’s death exposes tensions. The New York Times. https://www.nytimes.com/2010/11/10/education/10teacher.html. Accessed 16 Sept 2020.

  • MacKenzie, D. A. (2006). An engine, not a camera: How financial models shape markets. Cambridge: MIT Press.

  • Marx, K. (1978). Capital: A critique of political economy (Vol. 1). London: Penguin Books.

  • McCarthy, M. M. (1994). The courts and school finance reform. Theory Into Practice, 33(2), 89–97.

  • McLean, R. A., & Sanders, W. L. (1983). Objective component of teacher evaluation: A feasibility study. Knoxville: University of Tennessee.

  • Medvetz, T. (2012). Think tanks in America. Chicago: University of Chicago Press.

  • Mehta, J. (2013). The allure of order: High hopes, dashed expectations, and the troubled quest to remake American schooling. New York: Oxford University Press.

  • Mennicken, A., & Espeland, W. N. (2019). What’s new with numbers? Sociological approaches to the study of quantification. Annual Review of Sociology, 45, 223–245.

  • Mihaly, K., McCaffrey, D. F., Staiger, D. O., & Lockwood, J. R. (2013). A composite estimator of effective teaching. MET project. Seattle: Gates Foundation.

  • Millman, J., & National Council on Measurement in Education. (1981). Handbook of teacher evaluation. Beverly Hills: Sage Publications.

  • Mitchell, T. (2002). Rule of experts: Egypt, techno-politics, modernity. Berkeley: University of California Press.

  • Murphy, M. (2017). The economization of life. Durham: Duke University Press.

  • Mushkin, S. J. (1962). Health as an investment. Journal of Political Economy, 70(5), 129–157.

  • Olson, L. (2004). ‘Value added’ models gain in popularity-education week. Education Week. https://www.edweek.org/ew/articles/2004/11/17/12value.h24.html. Accessed 16 Sept 2020.

  • Oreskes, N., & Conway, E. M. (2010). Merchants of doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. New York: Bloomsbury Press.

  • Paige, M., Amrein-Beardsley, A., & Close, K. (2019). Tennessee’s National Impact on teacher Evaluation law & Policy: An assessment of value-added model litigation. Tennessee Journal of Law and Policy, 13(2).

  • Panhans, M.T., & Singleton, J.D. (2017). The empirical Economist’s toolkit: From models to methods. History of Political Economy, 49(supplement), 127–157.

  • Panofsky, A. (2014). Misbehaving science: Controversy and the development of behavior genetics. Chicago: University of Chicago Press.

  • Pivovarova, M., Broatch, J., & Amrein-Beardsley, A. (2014). Chatty, et al. on the American Statistical Association’s recent position statement on value-added models (VAMs): Five points of contention. Teachers College Record.

  • Polanyi, K. (2001). The great transformation: the political and economic origins of our time (2nd ed.). Boston: Beacon Press.

  • Porter, T. M. (1995). Trust in Numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.

  • Ravitch, D. (2012). Interview with Ravitch. https://education.asu.edu/inside-the-academy-of-educationwp-content/uploads/2012/08/Transcript-Ravitch.pdf. Accessed 16 Sept 2020.

  • Reckhow, S. (2013). Follow the money: How Foundation dollars change public school politics. New York: Oxford University Press.

  • Riehl, E., Saavedra, J.E., & Urquiola, M. (2016). Learning and earning: An approximation to college value added in two dimensions. In C. M. Hoxby & K. Strange (Eds.), Productivity in higher education (pp. 105–132). Chicago: University of Chicago Press.

  • Rothstein, J. (2009). Student sorting and Bias in value added estimation selection on observables and unobservables. Education Finance and Policy, 4(4), 537–571.

  • Rothstein, J. (2017). Measuring the impacts of teachers: Comment. American Economic Review, 107(6), 1656–1684.

  • Sanders, W.L. (1999). Helping teachers raise student achievement: An interview with William L. Sanders. https://news.heartland.org/newspaperarticle/1999/11/01/helping-teachers-raise-student-achievement-interview-william-l-sanders. Accessed 16 Sept 2020.

  • Sanders, W.L. (2000). Value-added assessment from student achievement data: Opportunities and hurdles. Journal of Personnel Evaluation in Education, 14(4), 329–339.

  • Sanders, W.L., & Rivers, J.C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Heartland Institute Research Progress Report. Arlington Heights: Heartland Institute. https://www.heartland.org/_template-assets/documents/publications/3048.pdf.

  • Schmidt-Wellenburg, C., & Lebaron, F. (2018). There is no such thing as ‘the economy’: Economic phenomena analysed from a field-theoretical perspective. Historical Social Research / Historische Sozialforschung 43(3 (165)), 7–38.

  • Schochet, P.Z., & Chiang, H.S. (2010). Error rates in measuring teacher and school performance based on student test score gains. NCEE 2010–4004. Washington, D. C.: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences. US Department of Education.

  • de Souza Leão, L., & Eyal, G. (2019). The rise of randomized controlled trials (RCTs) in international development in historical perspective. Theory and Society, 48(3), 383–418.

  • Stampnitzky, L. (2013). Disciplining terror: How experts invented “Terrorism”. Cambridge: Cambridge University Press.

  • Teixeira, P. N. (2000). A portrait of the economics of education, 1960-1997. History of Political Economy, 32(Suppl 1), 257–288.

  • Teixeira, P.N. (2014). Gary Becker’s early work on human capital – collaborations and distinctiveness. IZA Journal of Labor Economics, 3(1), 1–20.

  • Treu, Rolf M. (2014). Vergara v California tentative decision. http://studentsmatter.org/wp-content/uploads/2014/08/SM_Final-Judgment_08.28.14.pdf. Accessed 16 Sept 2020.

  • US Department of Education. (2009). Race to the top program: executive summary. https://www2.ed.gov/programs/racetothetop/executive-summary.pdf. Accessed 16 Sept 2020.

  • US Department of Health, Education, and Welfare, Office of the Assistant Secretary for Planning and Evaluation. (1969). Toward a long-range plan for Federal Financial Support for higher education: A report to the president. Washington, D.C.: US Department of Health, Education, and Welfare.

  • Weber, M. (1978). Economy and society: An outline of interpretive sociology. Berkeley: University of California Press.

Download references

Acknowledgments

For their helpful comments, the authors would like to thank Marion Fourcade, Simone Polillo, several anonymous reviewers, and audiences at meetings of the Social Science History Association, American Sociological Association, Society for Social Studies of Science, and the UCLA Contentious Politics and Organizations Working Group. Research was supported by the Center for American Politics and Public Policy at UCLA.

Author information

Authors and Affiliations

  1. Department of Sociology, UCLA, 264 Haines Hall, 375 Portola Plaza, Los Angeles, CA, 90095-1551, USA

    Zachary Griffen

  2. Public Policy, Institute for Society and Genetics, and Sociology, UCLA, 621 Charles E. Young Dr., South, Box 957221, 3660 LSB, Los Angeles, CA, 90095-7221, USA

    Aaron Panofsky

Authors
  1. Zachary Griffen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron Panofsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zachary Griffen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Griffen, Z., Panofsky, A. Ambivalent economizations: the case of value added modeling in teacher evaluation. Theor Soc 50, 515–539 (2021). https://doi.org/10.1007/s11186-020-09417-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11186-020-09417-x

Keywords

Search

Navigation