Elsevier

Journal of Criminal Justice

Volume 38, Issue 4, July–August 2010, Pages 400-409
Journal of Criminal Justice

Methodological artifacts in tests of rational choice theory

https://doi.org/10.1016/j.jcrimjus.2010.04.008Get rights and content

Abstract

Criminal decision making is an inherently natural and highly individualized process; however, rather than allowing participants to self-identify the costs and benefits that impact their own decisions to offend, rational choice researchers have typically provided participants with a uniform list of consequences to consider. Indirect evidence suggests this technique may alter the participants’ perceptions of consequences, yet no study to date has examined this supposition directly. In the current study, participants were randomly assigned to experimental conditions in which they either received a list of traditional costs and benefits to assess or were asked to self-generate their own list to assess. As in past research, when participants were allowed to self-generated consequences they identified several “novel” costs/benefits that have certainty/severity rating comparable to many of the traditionally examined consequences. Results also showed that consequences are more likely to be perceived as possible outcomes (i.e., receive a non-zero probability) when they are presented by researchers than when they are self-generated. Finally, the average certainty and severity of negative consequences do not differ across condition, while ratings of the certainty and value of benefits from crime are relatively lower when they are presented by researchers. Implications for rational choice theory and survey research in criminology more broadly are discussed.

Introduction

Human behavior is driven by the desire to maximize the hedonic experience—a desire that is universally inherited (Balleine et al., 2007, Cabanac and Bonniot-Cabanac, 2007). Despite this uniformity in our motivations, there remains great inter-individual variability within the processes underlying our decisions to act (Mellers, Schwartz, & Cooke, 1998). Within criminology, the rational choice perspective implicitly recognizes that decision making is both inherently natural and highly individualized. That is, when faced with a criminal opportunity, potential offenders are presumed by their human nature to deduce a series of risks and rewards to contemplate (Bentham, 1789/1970). These costs and benefits are assumed to vary from person-to-person in their number, kind and intensity (Bentham, 1789/1970, Brezina, 2001).

However, in their attempts to model the hedonic calculus, rational choice scholars have generally allowed criminal decision making to be neither natural nor individualized. Specifically, when using the hypothetical scenario design rational choice scholars have traditionally asked participants to consider a common set of consequences predetermined by the researcher rather than permitting participants to deduce their own set of personally-relevant costs and benefits (e.g., Paternoster and Simpson, 1996, Pogarsky and Piquero, 2004). Although participants are typically given the freedom to differentially rate the certainty and severity of these consequences, they generally have little-to-no opportunity to amend the list of costs and benefits to make them more personally-relevant (but see Wolfe, Higgins, & Marcum, 2007). As a result, only the uniform set of consequences provided by researchers are included in statistical models of decision making, despite rational choice theory's recognition of individualized hedonic calculi.

What are the consequences of forcing such artificial structure on the otherwise natural and individualized decision to offend? Ultimately, this is an empirical question that to our knowledge has not been adequately addressed. The current study examined the premise that traditional tests of rational choice theory have uncovered findings that are, in part, methodological artifacts. This contention is based on the large body of research showing how slight changes in question presentation can produce substantial differences in participants’ responses (e.g., Clark and Schober, 1992, Krosnick, 1999, Schuman and Presser, 1981, Schwarz, 1999). Before discussing this research, a review the dominant methodology used by criminologists to study perceptual deterrence/rational choice theory is presented.

Criminologists commonly use the hypothetical scenario method (HSM) in order to test perceptual deterrence and rational choice theories. Although variations in the methodology exist, the traditional HSM consists of providing participants with a brief vignette describing an offending opportunity. Vignettes have been written to describe a variety of criminal/deviant acts such as academic cheating, tax cheating, doping in professional sports, police misconduct, digital piracy, software piracy, petty theft, drunk driving, sexual assault, physical assault, and multiple acts of corporate offending (Bachman et al., 1992, Bouffard, 2007, Carmichael and Piquero, 2004, Higgins et al., 2005, Klepper and Nagin, 1989, Nagin and Paternoster, 1993, Nagin and Paternoster, 1994, Paternoster and Simpson, 1996, Pogarsky and Piquero, 2004, Strelan and Boeckmann, 2006, Tibbetts, 1999, Wolfe et al., 2007). Participants are asked to assume the role of the protagonist in the scenario and self-report their likelihood of committing the offense.

The HSM also queries the participants’ perception of the consequences associated with the offense. Although there is variability in the number and types of consequences examined, the set of costs and benefits that have been traditionally examined include: formal legal problems (arrest/punishment), feelings of immorality, negative emotional states (guilt/shame), family problems (parental disapproval), social problems (peer disapproval), school problems (expulsion), and/or professional problems (loss of current/future employment). The traditionally examined benefits of crime are far fewer in number but include: the fun/thrill that comes from offending, and/or some indicator of criminal utility such as financial gain, professional advancement, or feeling good about oneself (see Appendix A for a list of studies that have included each of these various consequence types). Participants generally rate the perceived certainty and severity of these consequences, and these ratings are used to predict participants’ likelihood of offending (e.g., Bachman et al., 1992, Nagin and Paternoster, 1993, Tibbetts and Myers, 1999). Although these consequence ratings are often examined individually, some scholars aggregate them to produce a more global measure of costs and/or benefits (Carmichael and Piquero, 2004, Loewenstein et al., 1997, Nagin and Paternoster, 1993, Nagin and Paternoster, 1994). In either case, studies using the HSM have generally supported the deterrence/rational choice perspective in that traditionally examined costs of crime—especially legal sanctions, immorality, and negative emotional states—have been found to deter, while benefits such as excitement and utility have been shown to be significant enticements. Yet, there is at least some empirical evidence to suggest that the perceived consequence ratings used in these past studies may have been (at least, in part) an artifact of the HSM methodology.

The manner in which questions are framed, the values used to anchor Likert scales, and the order in which response options are presented are just a few of the factors that influence participants’ responses to survey questions (Biemer, Groves, Lyberg, Mathiowetz, & Sudman, 1991; Krosnick, 1999, Kühberger, 1998; Schuman and Presser, 1981, Schwarz, 1999). In a similar vein it is possible that providing participants with a set of predetermined consequences to contemplate (as in the traditional HSM) rather than allowing them to self-generate their own, alters participants’ perceptions of the costs and benefits of offending. In particular, research comparing participants’ responses to open- and closed-ended questions supports this contention.

Dating to as early as the 1920s, researchers have known that the same question written in either open- or closed-ended format can produce vastly different results (Converse, 1987; see also Boles and Burton, 1992, Hruschka et al., 2004). Perhaps the most methodologically rigorous examples of studies comparing fixed choice (closed-ended) and free elicitation (open-ended) responses are the Schuman and Presser (1979) split-ballot experiments. In these studies, participants were asked either open- or closed-ended questions about the job feature they valued most, the most important problem facing the country, and the quality that is most important for children to learn. Respectively, “importance of the work” (59.1 percent), “crime” (34.9 percent) and “thinking for themselves” (61.5 percent) were endorsed most often by participants completing closed-ended questions. In contrast, these same responses had endorsement rates of only 21.3 percent, 15.7 percent, and 4.6 percent (respectively) when the questions were left open-ended.

These experiments also reveal that the response sets for closed-ended questions commonly failed to include the opinions of a sizeable minority—or perhaps even the majority—of the sample. For example, among those completing the open-ended job quality question, 50.3 percent of the sample endorsed “novel” characteristics that did not correspond to any of the response options listed in the closed-ended question. Similarly, when answering open-ended questions about the country's most pressing issue and the quality most important to teach children, 32.7 percent and 78.2 percent of participants (respectively) endorsed novel characteristics (Schuman & Presser, 1979). Unfortunately, the inclusion of an “Other: ___” category does little to capture these novel responses as participants rarely invoke this option (Converse, 1987, Schuman and Presser, 1981). In other words, participants will generally endorse an item listed in the response set even if it is not reflective of their true opinion (see Sudman et al., 1996, Schwarz, 1999, Schwarz, 2007 for detailed discussions of the specific cognitive and linguistic processes that influence the manner in which individuals respond to open- versus closed-ended survey questions).

Thus, empirical research shows that survey results are influenced by the question format. This methodological artifact appears to stem from the distinct thought processes associated with each question type. From a cognitive standpoint, the difference between open- and closed-ended questions is a matter of recall versus recognition. Whereas recall techniques provide participants with an outlet for the responses that are most salient to them, recognition techniques tend to activate the participants’ memory in order to determine the “best” response (Schwarz and Hippler, 1991, Schwarz, 2007). In other words, closed-ended questions force participants to remember opinions they may had forgotten or thought too trivial to mention. What is potentially worse, in cases where the closed-ended question involves a response option that would otherwise not have been considered by the respondent, the individual may “create” (and report) a judgment that was not actually held prior to the question being asked (Schuman and Presser, 1981, Sudman et al., 1996). This then has the effect of increasing participants’ endorsements of opinions that are relatively inconsequential to them. In contrast, open-ended questions tend to capture only the more dominant opinions that are actively present in conscious thought (see also Schwarz & Oyserman, 2001).

The previous section presented a description of how presenting participants with a set of response options (i.e., using closed-end questions) can artificially inflate participants’ endorsement of inconsequential beliefs, while at the same time fail to capture participants’ more salient perceptions. In light of these existing studies, the current study set out to examine whether the traditional HSM is fostering similar “question form effects”. In particular, this study proposed that presenting participants with a predetermined set of costs and benefits to assess (rather than allowing participants to freely elicit their own) may artificially inflate the empirical support for these specific consequences, while at the same time overlook the influence of other more novel—and perhaps more salient—costs and benefits. Based on a review of the existing literature in criminology (and rational choice theory specifically), this issue has not been directly tested; however, Bouffard, 2002, Bouffard, 2007, Bouffard et al., 2008) has indirectly examined whether question form influences the perceived relevance of (but not certainty and severity estimates attached to) various costs and benefits presented with each method.

Specifically, in a series of rational choice studies, Bouffard, 2002, Bouffard, 2007, Bouffard et al., 2008) has examined the issues of question form effects and the individualized nature of decision making. In each of these studies, Bouffard, 2002, Bouffard, 2007, Bouffard et al., 2008) presented participants with various offending scenarios and then asked them to list all the costs and benefits they might experience by engaging in a given criminal act. Findings from this body of research indicate that many of the consequences commonly included in traditional HSM studies have low endorsement rates when participants are asked to freely elicit consequences. For instance, “immorality”—which is one of the most frequently identified deterrents in the HSM literature—was identified by less than five percent of participants (Bouffard, 2002). Furthermore, this free elicitation methodology uncovered many novel consequences that have been overlooked in HSM studies. For example, after reading a “date rape” scenario, 80 percent of participants identified the risk for sexually transmitted diseases as a potential cost of sexual aggression (Bouffard, 2002); yet in traditional HSM studies examining sexually aggressive decision making, such health consequences are not factored into the hedonic calculus (Bachman et al., 1992, Loewenstein et al., 1997, Nagin and Paternoster, 1993, Nagin and Paternoster, 1994). Underscoring the individualistic nature of decision making, Bouffard (2007) also found that various individual characteristics (e.g., age, gender, prior experience with offending without being apprehended) predicted whether the respondent found various consequences relevant to their decisions.

Although this body of research has begun to explore the possibility of question form effects (and individual differences in the content of hypothetical offending decision making) this research did not compare the two methodologies directly to determine the impact of question format on the certainty and severity ratings reported with each. In fact, no study to date has directly compared whether and to what extent the HSM and free elicitation methodologies produce differing results with respect to the endorsement of consequences. The current study sought to fill this void and thus represented a next step in the line of research which Bouffard, 2002, Bouffard, 2007, Bouffard et al., 2008) has begun.

Using a rational choice framework and drawing from the existing HSM and free elicitation research, an experimental design was employed to examine the ramifications of providing participants with a uniform set of consequences to consider. Based on the research comparing open- and closed-ended questions (e.g., Schuman & Presser, 1979), the following hypotheses were proposed:

Hypothesis 1

The traditional set of costs and benefits will be endorsed more frequently when they are presented by researchers than when they are self-generated by participants.

Hypothesis 2

A sizeable minority (or perhaps even the majority) of participants who self-generate consequences will endorse “novel” costs and benefits that have been overlooked by traditional rational choice studies.

Hypothesis 3

The traditional set of costs and benefits will be more impacful (e.g., have higher certainty and severity ratings) when they are self-generated by participants than when they are presented by researchers.

Section snippets

Sample

A sample of 208 undergraduate students served as participants in this study. The students were enrolled in an introductory level criminology course at a medium-sized, Midwestern university and received extra course credit for participating in an IRB approved study (one of several options for earning such credit). The sample is predominately White (93 percent), had slightly more men than women (56 percent male), and had an average age of 21 years (SD = 2.55). Comparisons with the university

Randomization check

Of the 208 participants in the sample, 105 (51 percent) were randomly assigned to the RGC condition and the remaining 103 (49 percent) to the SGC condition. Respectively across these two groups, no statistically significant differences were found regarding: average age (20.84 vs. 21.10), percent male (53.8 vs. 58.3), percent non-White (6.7 vs. 6.9), average social bond score (3.94 vs. 3.96), average self control score (3.05 vs. 3.15), prevalence of prior shoplifting (51.4 percent vs. 47.6

Discussion

From both a theoretical and policy-oriented perspective, it is important to understand how the perceived consequences of crime (both individually and collectively) influence decisions to offend. As rational choice scholars have undertaken this charge they have commonly employed the HSM, which presents participants with a predetermined set of consequences and, as a result, artificially structures the decision making process. The current study sought to determine if the perceived consequence

References (51)

  • A. Kühberger

    The influence of framing on risky decisions: A meta-analysis

    Organizational Behavior and Human Decision Processes

    (1998)
  • G. Pogarsky et al.

    Studying the research of deterrence: Can deterrence theory help explain police misconduct?

    Journal of Criminal Justice

    (2004)
  • N. Schwarz et al.

    Asking questions about behavior: cognition, communication, and questionnaire construction

    American Journal of Evaluation

    (2001)
  • R. Bachman et al.

    The rationality of sexual offending: testing a deterrence/rational choice conception of sexual assault

    Law & Society Review

    (1992)
  • B.W. Balleine et al.

    Introduction: current trends in decision making

    Annals of the New York Academy of Sciences

    (2007)
  • J. Bentham

    An Introduction to the principles of morals and legislation

    (1789/1970)
  • P.P. Biemer et al.

    Measurement error in surveys

    (1991)
  • J. Boles et al.

    An examination of free elicitation and response scale measures of feelings and judgments evoked by television advertisements

    Journal of the Academy of Marketing Science

    (1992)
  • J.A. Bouffard

    Methodological and theoretical implications of using subject-generated consequences in tests of rational choice theory

    Justice Quarterly

    (2002)
  • J.A. Bouffard

    Predicting differences in the perceived relevance of crime's costs and benefits in a test of rational choice theory

    International Journal of Offender Therapy and Comparative Criminology

    (2007)
  • J.A. Bouffard et al.

    Beyond the “science of sophomores”: does the rational choice explanation of crime generalize from university students to an actual offender sample?

    International Journal of Offender Therapy and Comparative Criminology

    (2008)
  • T. Brezina

    Assessing the rationality of criminal and delinquent behavior: a focus on actual utility

  • M. Cabanac et al.

    Decision making: rational or hedonic?

    Behavioral and Brain Functions

    (2007)
  • S. Carmichael et al.

    Sanctions, perceived anger, and criminal offending

    Journal of Quantitative Criminology

    (2004)
  • H.H. Clark et al.

    Asking questions and influencing answers

  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1988)
  • J.M. Converse

    Survey research in the United States: roots and emergence 1890-1960

    (1987)
  • M.L. Exum

    The application and robustness of the rational choice perspective in the study of intoxicated and angry intentions to aggress

    Criminology

    (2002)
  • A. Fink

    How to ask survey questions

    (1995)
  • M. Galesic et al.

    What is sexual harassment? It depends on who asks! Framing effects on survey responses

    Applied Cognitive Psychology

    (2007)
  • H.G. Grasmick et al.

    Testing the core empirical implications of Gottfredson and Hirschi's general theory of crime

    Journal of Research in Crime and Delinquency

    (1993)
  • G.E. Higgins et al.

    An application of deterrence theory to software piracy

    Journal of Criminal Justice and Popular Culture

    (2005)
  • D.J. Hruschka et al.

    Fixed-choice and open-ended response formats: a comparison from HIV prevention research in Zimbabwe

    Field methods

    (2004)
  • F.J. Ivis et al.

    The effect of question structure on self-reports of heavy drinking: Closed-ended versus open-ended questions

    Journal of Studies on Alcohol

    (1997)
  • S. Klepper et al.

    The deterrent effect of perceived certainty and severity of punishment revisited

    Criminology

    (1989)
  • Cited by (11)

    • False positive and false negative rates in self-reported intentions to offend: A replication and extension

      2014, Journal of Criminal Justice
      Citation Excerpt :

      Tests of deterrence and rational choice theory have commonly utilized the hypothetical scenario method (Bouffard, Exum, & Collins, 2010).

    • Rational choice beyond the classroom: Decision making in offenders versus college students

      2013, Journal of Criminal Justice
      Citation Excerpt :

      As noted previously, much of what is known about criminal decision making comes from college students who take part in hypothetical scenario research (see a recent review by Baker & Piquero, 2010). The types of criminal behaviors examined in these studies of include sexual assault/aggression (Bachman et al., 1992; Bouffard, 2002a; Loewenstein, Nagin, & Paternoster, 1997), physical assault (Bouffard, 2007; Carmichael & Piquero, 2004; Exum, 2002) and shoplifting/larceny (Bouffard, 2007; Bouffard, Exum, & Collins, 2010; Nagin & Paternoster, 1993; Rebellion, Piquero, Piquero, & Tibbetts, 2010). Arguably the most commonly studied criminal offense studied with the hypothetical scenario method is drunk driving (Bouffard, 2002b, 2007; Loewenstein et al., 1997; Nagin & Paternoster, 1993; Nagin & Pogarsky, 2001; Piquero & Pogarsky, 2002; Piquero & Tibbetts, 1996; Pogarsky, 2002, 2004; Thurman, 1986; Tibbetts & Herz, 1996).

    • Gender differences in rational decisions on shoplifting

      2022, Crime Prevention and Community Safety
    View all citing articles on Scopus

    This paper was accepted under the Editorship of Kent Joscelyn.

    View full text