Report
Multinomial process tree models of control and automaticity in weapon misidentification

https://doi.org/10.1016/j.jesp.2008.11.002Get rights and content

Abstract

When primed with a Black face, people are more likely to misidentify a non-weapon as a weapon. Weapon misidentification may hinge on the distinction between controlled and automatic processes. Various relationships between controlled and automatic processes are cast in the form of five multinomial process models, which are illustrated and compared. It is shown that variants of the traditional Process Dissociation model and the Stroop model are nested within the Quad-Model. Across four different studies, various complexity corrected model performance measures converged to support the Process Dissociation account. This account suggests that the automatic association between race and weapons is subordinate to controlled processing. More generally, these results suggest that the weapon-bias might be alleviated without interventions that directly target stereotypes.

Introduction

When determining if a weapon is present, people sometimes falsely claim to have seen a weapon when they have seen only a harmless object and a Black person (Payne, 2001; also see Correll et al., 2002, Greenwald et al., 2003, Lambert et al., 2005). Such split-second errors could be terribly important if the error means pulling a trigger rather than pressing a computer key. The goal of the current research is to compare different process accounts of this error, and specifically, process accounts that can be cast in the form of multinomial process tree models. To this end, first, a set of multinomial models will be illustrated, including Process Dissociation models and the quadruple process model. It will be shown that the models are connected not only conceptually, but also via a specific mathematical relationship (i.e., they are nested). Second, these models will be empirically compared in terms of their ability to account for weapon misidentification data.

Weapon misidentification may hinge on the distinction between controlled processing and automatic accessibility of the stereotype (Payne, 2001). We define controlled processing broadly, as the use of information most applicable to one’s current goals or task set. In the case of weapon identification, controlled processing can take the form of discriminating between the perceptual characteristics of gun and non-gun objects, and using this information to respond within a limited amount of time. In contrast, the activation of automatic processes may reflect primed associations that do not necessarily aid the accurate completion of one’s goals. In the case of weapon identification, the automatic process of most interest is the stereotypical association between African Americans primes and weapons.

Two decades of research on dual-process theories have established the importance of distinguishing between automatic and controlled influences in social cognition (Chaiken & Trope, 1999). These theories describe distinct processes, but they rarely specify how the processes relate to each other. A “second generation” of research has recently begun doing so (Bargh, 2006). A complete dual-process theory must explain not only when distinct processes are likely to drive behavior, but also how those processes interact. For example, when automatic and controlled responses conflict, how is that conflict resolved?

Some dual-process theories ascribe a relatively dominant role to controlled processing, with automatic processes influencing behavior only to the extent that controlled processing fails (Jacoby, 1991, Payne, 2001). Other theories treat controlled processing as relatively subordinate, as a means to adjust an initial impression or decision that was based on automatic processes (Devine, 1989, Gilbert et al., 1988, Kahneman, 2003; for discussion, see Conrey et al., 2005, Jacoby et al., 1999).

The relative dominance of controlled and automatic processes may depend on the task at hand. In the weapon identification task, consider three possible relationships between controlled and automatic processes:

  • 1.

    Control-Dominating relationship: A non-weapon is misidentified as a weapon only when both controlled processing fails and automatic racial bias occurs.

  • 2.

    Automaticity-Dominating relationship: A non-weapon is misidentified as a weapon when an automatic racial bias occurs, regardless of controlled processing.

  • 3.

    Probabilistic relationship: If both controlled processing succeeds and the automatic influence occurs on the same trial, the resulting conflict will be resolved probabilistically.

Note that these three possible relationships only refer to how conflict is resolved between control and automaticity. For example, even if the Control-Dominating relationship were true, it would still be possible for automaticity to play an important role in determining behavior. Automatic processes would still be relied upon to make decisions so long as controlled processes failed. Thus, the relationship between controlled and automatic processes may be very difficult to intuit a priori for any given task or situation. Whether the relationship takes one form or another is an empirical question that can be addressed via mathematical modeling.

The possible relationships between control and automaticity can be cast as multinomial process tree models. Multinomial models allow researchers to test theories of underlying processes in a way that traditional approaches like ANOVA cannot (Riefer & Batchelder, 1988). One reason for this is that multinomial models can have more than one process pathway or branch leading to the same response. To illustrate, consider a case where a person correctly identifies a gun after being primed with a Black face. According to the Process Dissociation model, there are two process pathways that can lead to this event. In the first pathway, control constrains processing to relevant perceptual characteristics of the gun, thereby leading to a correct gun response. In the second pathway controlled processing fails, but the gun response is still given because of the automatic influence of the prime. Multinomial models can help disentangle underlying processes in such situations (for a review, see Batchelder & Riefer, 1999).

In multinomial models, each parameter ranges from 0 to 1 and represents the probability with which a process occurs. The top panel of Fig. 1 shows the processes in the traditional Process Dissociation Model (Jacoby, 1991, Payne, 2001). In that model, when controlled processing succeeds with probability C, a correct response (+) is given in all conditions. When controlled processing fails with probability (1  C), the automatic influence of the prime determines the response. When the automatic influence is stereotype-consistent with probability A, the response is correct for the White-tool and Black-gun conditions, but incorrect (−) for the other conditions. With probability (1  A), the response is counter-stereotypical, with a tool response following Black primes and a gun response following White primes. Note that the A parameter is irrelevant whenever controlled processing succeeds. Thus, even though the automatic process operates faster, controlled processing dominates automaticity in the Process Dissociation model (see Payne, 2001, Payne, 2005, Payne et al., 2002, for evidence that C and A parameters represent controlled and automatic processes, respectively; for related evidence, see Klauer & Voss, 2008).

By contrast, in the bottom panel of Fig. 1, automaticity dominates controlled processing. We refer to this as the “Stroop model” because it is based on a model developed for the Stroop task (Lindsay & Jacoby, 1994), a task where the relatively automatic, unintended process of word-reading can dominate the intended process of color-naming. In the Stroop model, if the prime has an automatic influence, a stereotypic response is given, and this occurs regardless of the status of controlled processing. Only when an automatic influence does not occur (with probability 1  A) does controlled processing matter. Thus, Process Dissociation and Stroop models represent the Control-Dominating and Automaticity-Dominating relationships, respectively.

The third possible relationship can be represented by the Quad-Model (Conrey et al., 2005). The original depiction of the Quad-Model is shown in Fig. 2. Parameter AC is analogous to parameter A in the previous models in that it represents the stereotypic effect of the prime. Parameter D is analogous to parameter C in the previous models in that it represents correct discrimination of the object (gun or tool).

Importantly, the Quad-Model also has precise relationships to variants of the Process Dissociation and Stroop models. Relabeling D as C, and AC as A, the Quad-Model can be rewritten into an algebraically equivalent form, shown in Fig. 3 (for proof that Fig. 2, Fig. 3 are identical models, see Appendix A). Fig. 3 shows that the Process Dissociation model with a guessing parameter (Buchner, Erdfelder, & Vaterrodt-Plünnecke, 1995) and the Stroop model with a guessing parameter are both nested within the Quad-Model. These nested models will be referred to as Process Dissociation/G and Stroop/G. Although the OB parameter has sometimes been described as inhibitory (Conrey et al., 2005), Fig. 3 reveals other possible interpretations. Another way of describing the Quad-Model is that, with probability OB, the Process Dissociation/G model is followed; with probability (1  OB), the Stroop/G model is followed. Therefore, when OB = 1.0, the Quad-Model reduces to the Process Dissociation/G model; when OB = 0.0, the Quad-Model reduces to the Stroop/G model.

To our knowledge, this nested relationship has not been previously reported in the literature. The nested relationship shows that the OB parameter in the Quad-Model may provide information about which processes dominate others. For Control-Dominating situations, OB should approach 1; for Automaticity-Dominating situations, OB should approach 0. Additionally, the nested relationship reveals that the interpretation of Quad-Model parameters critically depends on the value of OB. For example, AC has sometimes been described as a parameter distinct from the G parameter because only G measures response biases that occur when controlled processing fails (Sherman, 2006). However, as OB approaches 1 (as shown in the top half of Fig. 3), both G and AC represent biases that matter only when control fails.

There is only one previously published comparison of multinomial models of weapon misidentification. Using data from Lambert et al., 2003, Conrey et al., 2005 compared the Process Dissociation, Quad-Model, and Stroop model. The Quad-Model showed the smallest G2 value, and so it was concluded that the Quad-Model “provides a better description of the responses in the weapon identification task than less complex models” (Conrey et al., 2005; p. 482).

However, a major challenge to model comparison is that models that are complex (e.g., have a large number of free parameters) can capitalize on chance (Busemeyer and Wang, 2000, Pitt et al., 2002). That is, complex models can sometimes appear to fit well for spurious reasons. To take an extreme example, if a model were so complex that it had as many free parameters as there were data points, the model could spuriously provide a perfect fit to the data with a G2 of 0. More generally, G2 will tend to decrease as the number of model parameters increases even if the model is inaccurate. This tendency is analogous to the tendency in multiple regression for R2 to improve as the number of predictors increases even if those predictors are random and arbitrary.

Because the Quad-Model is more complex than other models, does the Quad-Model better account for the data, or just better capitalize on chance? Furthermore, is there a model that is generally supported by the data? We address these questions by using several model testing approaches that adjust for complexity differences, including nested tests to examine the OB parameter, goodness-of-fit tests relative to a saturated model, and more direct model comparison using both the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC).

One approach is to compare two models to see if the model with additional free parameters fits significantly better than the model with fewer free parameters. This approach is only valid when the models are nested. For example, the Quad-Model can be compared to the Stroop/G model, because the latter is nested within the former. When OB = 0, the Quad-Model reduces to the Stroop/G model, and so a significant difference from 0 would be taken as evidence against the Stroop/G model. Similarly, the Quad-Model can be compared to the Process Dissociation/G model by determining whether the OB parameter was significantly different from 1. Unfortunately, there are limitations to nested testing. One limitation is that the Process Dissociation and Stroop models without guessing parameters cannot be compared to other models. A second limitation is that the OB parameter can lack precision in the Weapon-Bias paradigm (e.g., Conrey et al., 2005), and so it may rarely be significantly different from either 1 or 0. Finally, while nested tests can potentially provide evidence against the Process Dissociation/G and Stroop/G models, the Quad-Model is insulated from falsification in such tests.

The G2 statistic can be computed for each model of interest relative to a saturated model, with G2 representing badness of fit (i.e., higher is worse). G2 is biased to be low for models with more free parameters (e.g., the Quad-Model), and so models should not be directly compared to one another via G2. Each model of interest can, though, be separately compared to the saturated model. Each model of interest has a particular critical value based on the model’s degrees of freedom. When G2 is below the critical value, the model of interest is said to “fit” the data. When a model of interest fits, the model did not perform significantly worse than the saturated model. G2 is limited because, even when a model fits, all it indicates is a failure to reject the null hypothesis. Results can be ambiguous when multiple models fit the data. Furthermore, G2 does not allow for direct comparisons among non-nested models.

To directly compare all models, two adjustments to G2 that correct for complexity differences among models were examined. The two particular adjustments were chosen because they are known to have different biases. The first adjustment, AIC (Akaike, 1974), is biased to favor more complex models especially when the sample size is large. The second adjustment, BIC (Schwarz, 1978), is based on asymptotic principles from Bayesian model comparison. BIC tends to favor simpler models. Because AIC and BIC have opposing biases, when both complexity adjustments agree, there is good evidence that one model is performing better than another (Burnham and Anderson, 2004, Kuha, 2004). For both AIC and BIC, smaller values indicate better model performance.

We extend previous work in three ways in order to thoroughly test the generality of results. First, we use four datasets (including the one used by Conrey et al.) in order to determine whether results are consistent across studies. The studies, described below, were selected because they included enough conditions to provide the degrees of freedom needed to directly compare all models. Second, we tested additional models that are nested within the Quad-Model: the Process Dissociation/G and Stroop/G models. These comparisons allow us to consider whether the use of an OB parameter or Guessing parameter (or both) are justified by the available data. Third, data are analyzed at both the group and the subject level. The group level collapses together all responses from all subjects in an experimental group. This approach is common in multinomial process tree models (for a review see Batchelder & Riefer, 1999), but it relies on the assumption that parameter values are identical across participants within a group. Alternatively, models can be fit separately to each subject, with each subject having his or her own set of unique parameter values. The two levels of analysis can produce very different results (e.g., Estes & Maddox, 2005) and so we examine both separately (for alternative approaches, see Klauer, 2006, Riefer and Batchelder, 1991).

Section snippets

Lambert et al. (2003), Study 2

Participants (N = 127) were randomly assigned to a Private or Anticipated Public condition. Those in the Private condition were assured that their responses would be confidential. Participants in the Anticipated Public condition were told that, following the task, they would be asked to share and discuss their performance with other participants in the experiment session. The primes included four Black and four White faces, including two male and two female faces of each race. Target photos

OB parameter

As shown in Fig. 4, the OB parameter at the group level was sometimes significantly different from 0, suggesting that the Stroop/G model was inaccurate. Otherwise, the OB parameter was largely ambiguous.

Using nested comparisons, 5 of the 14 OB parameters were significantly different from 0, ps < .05. These 5 OB parameters are the same 5 in Fig. 4 that do not have confidence intervals overlapping with 0. In contrast, none of the OB parameters were significantly different from 1.0, ps > .05. In other

Discussion

Among the five multinomial models tested, the Process Dissociation model was best supported by the data. It was supported by AIC and BIC at both the group level and the subject level, and also G2, particularly at the subject level, where more participants were successfully fit by the Process Dissociation model than by any other model. Analyses of OB parameters and the Group level G2 were less clear, but generally suggested that the Stroop and Stroop/G models did not accurately represent the

Acknowledgments

We thank Alan Lambert for generously sharing data, Elizabeth Collins, Riki Conrey, Melanie Green, and Larry Jacoby for helpful feedback, and Nicole Beckage and Michael Schachter for assisting with references. This research was supported in part by the National Science Foundation (0615478).

References (37)

  • S. Chaiken et al.

    Dual-process theories in social psychology

    (1999)
  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1977)
  • F.R. Conrey et al.

    Separating multiple processes in implicit social cognition: The quad model of implicit task performance

    Journal of Personality and Social Psychology

    (2005)
  • J. Correll et al.

    The police officer’s dilemma: Using ethnicity to disambiguate potentially threatening individuals

    Journal of Personality and Social Psychology

    (2002)
  • P.G. Devine

    Stereotypes and prejudice: Their automatic and controlled components

    Journal of Personality and Social Psychology

    (1989)
  • W.K. Estes et al.

    Risks of drawing inferences about cognitive processes from model fits to individual versus average performance

    Psychonomic Bulletin & Review

    (2005)
  • F. Faul et al.

    G∗Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences

    Behavior Research Methods

    (2007)
  • D.T. Gilbert et al.

    On cognitive busyness: When person perceivers meet persons perceived

    Journal of personality and social psychology

    (1988)
  • Cited by (40)

    • Race, weapons, and the perception of threat

      2020, Advances in Experimental Social Psychology
      Citation Excerpt :

      Comparability between the PDP and other models allows researchers to estimate, evaluate, and compare the fit of alternative models, including those with larger numbers of parameters. A variety of other models have been applied to the WIT, such as a version of PDP in which the ordering of automatic and controlled parameters in the processing tree are reversed (thus, people exert control only in the absence of an automatic response tendency; Bishara & Payne, 2009). The results suggest that the control-dominant model (depicted above) is preferable.

    • RT-MPTs: Process models for response-time distributions based on multinomial processing trees with applications to recognition memory

      2018, Journal of Mathematical Psychology
      Citation Excerpt :

      The proposed approach is also likely to be valuable in areas in which the incorporation of response times into existing MPT models represents a major development. One such area of research is implicit social cognition – which includes prominent paradigms such as the Implicit Association Test or the Weapon Identification Task – where several MPT models capturing a diverse set of automatic and controlled processes have been proposed (e.g., Bishara & Payne, 2009; Meissner & Rothermund, 2013; for reviews, see Hütter & Klauer, 2016; Sherman, Klauer, & Allen, 2010). Despite their merits, these models currently ignore response times, a key aspect of participants’ judgments (e.g., decide as quickly as possible whether the object being held by a Black or White person is a gun) and a key aspect in traditional analyses of such data.

    • Multinomial processing trees as theoretical bridges between cognitive and social psychology

      2018, Psychology of Learning and Motivation - Advances in Research and Theory
      Citation Excerpt :

      That said, MPTs can be compared quantitatively to evaluate which provides best fit to a given data set. For example, Bishara and Payne (2009) applied five different MPTs to data from a weapons identification task and calculated AIC and BIC as model-selection indices to determine which MPT best describes data from this task. In this way, MPTs can help to advance both theory (i.e., validating or falsifying the assumptions articulated by the model) and methodology (i.e., which model provides best fit to data from a given task) by providing rigorous quantitative standards for both model fit and selection.

    View all citing articles on Scopus
    View full text