Deception is a methodological technique in which participants are intentionally misinformed or are not fully aware of the specific purpose of a study. Hertwig and Ortmann (2008b) stated that although deception in research is not easily defined, a consensus has emerged across disciplines defining deception as “intentional and explicit provision of erroneous information … whereas withholding information about research hypotheses, the range of experimental manipulations, or the like ought not to count as deception” (p. 222). Deception is commonly employed when researchers cannot obtain viable or desired information through other means, based on the notion that awareness of the exact nature of the study will compromise certain psychological or behavioral processes. Thus, use of deception enables researchers to collect uncompromised data. Although deception is most commonly employed within social psychological research, it is a methodological technique used in other subdisciplines of psychology, such as personality, cognitive, and clinical psychology, as well as in research outside of psychology (e.g., medicine, marketing).

Researchers employing deception assume that participants will have no knowledge of the deception, that the experimental design will not elicit awareness of the deception by participants, that the methodology is sound, and that participant behavior is not a result of suspiciousness (Golding & Lichtenstein, 1970). In order to determine whether these assumptions are met, it has been suggested that participants complete a postexperimental inquiry before being debriefed (Aronson, Ellsworth, Carlsmith, & Gonzales, 1990). The purpose of this inquiry is to assess whether participants were suspicious of the cover story or any feedback/information given, thought they were being deceived, or had knowledge about any aspect of the study before participating. The postexperimental inquiry is necessary because demand characteristics are often present in deception research and because participant crosstalk (participants divulging important experimental details with other individuals who may participate in the study) is a common phenomenon in psychology participant pools (see Edlund, Sagarin, Skowronski, Johnson, & Kutter, 2009). It is vital that the information collected during the postexperimental inquiry be accurate, because if the data collected from participants are based on potential knowledge of deception and not on natural behavior, those data are invalid and must be excluded.

Unfortunately, the ability of the postexperimental inquiry to detect whether participants had knowledge about a study, believed the ostensible purpose of the study, or were suspicious of any aspect of the study is questionable. Several studies have employed a technique in which a confederate gave participants information about the study prior to participation. Two studies (McMillen & Austin, 1971; Nichols & Maner, 2008) found that none of the participants admitted to having received information about the study during the postexperimental inquiry. Likewise, Taylor and Shepperd (1996) found that although participants discussed feedback they had received before interacting as a group, uncovering the deception in the study, none of the participants admitted to knowing that the feedback they had received was false during the postexperimental interview. Sagarin, Rhoads, and Cialdini (1998) had similar results, finding that only 1 out of 81 participants admitted to having received the answer for the task from a confederate, and Levy (1967) found that 1 of 16 participants admitted to having received information about the study before participating.

Golding and Lichtenstein (1970) designed a study to determine how likely participants were to admit to having prior information about the study and to possessing suspicion and/or awareness of the study purpose. These researchers also examined whether certain factors increased participants’ admission rates. They found that although awareness and admission rates were low, participants who were told that their answers were important for the sake of the scientific integrity of the study had higher admission rates. Newberry (1973) further examined factors that increased admission to having received information about the study before participating. Overall, he found that 30%–80% of participants lied about receiving information from a confederate. Admission rates were lower, however, when participants were questioned by an experimenter rather than on a questionnaire.

The present research

Although participant honesty on postexperimental inquiries is vitally important to research employing deception, little research has been conducted since the 1970s that has examined factors that may improve participant honesty in response to postexperimental inquiries. The present research therefore aimed to examine (1) how willingly participants admit to suspiciousness, awareness of deception, and prior knowledge during a postexperimental inquiry; (2) methods that are already in use by some researchers to improve participant honesty, but that have yet to be empirically tested; and (3) why participants either are or are not forthcoming during the postexperimental inquiry.

Study 1

In order to assess the adequacy of postexperimental inquiries in their proper context, we first need to know how prevalent their use is. The purpose of Study 1 was therefore to assess how frequently researchers employing deception in their research include a postexperimental inquiry. Although Aronson et al. (1990) suggested that a postexperimental inquiry be employed in any study in which participants are deceived, it is not clear whether researchers follow this advice. We chose to survey social psychologists because deception is most commonly employed in social psychological research.

Members of the Society for Personality and Social Psychology (SPSP) listserv were sent an e-mail eliciting participation in the survey. We chose to sample members of the SPSP listserv because SPSP is the largest organization of social psychologists in the world, with over 7,000 members, of whom approximately 1,900 are on the listserv. The e-mail sent to listserv members requested that any researchers who had conducted research involving deception of human participants complete the survey. A total of 77 individuals completed the survey. These researchers were asked whether they typically use a postexperimental inquiry when they conduct research involving deception and whether they administer the inquiry before debriefing, after debriefing, before and after debriefing, or other. They were also asked what form the postexperimental inquiry took: interview with the experimenter, interview with a third person (e.g., another experimenter, but not the one participants had interacted with during the study), a paper-and-pencil survey, a survey administered on a computer, or another form. The researchers were also asked how they typically use the data gathered from the postexperimental inquiry (discard data if a participant indicates awareness of the deception or purpose of the study, discard data if a participant indicates suspicion, use statistics to control for suspicion and/or awareness, or other). Finally, researchers were asked about the questions that they typically ask participants during the postexperimental inquiry.

Of the 77 researchers who completed the survey, only 2 said that they do not use a postexperimental inquiry to probe for suspicion and/or awareness. Most of the respondents (84%) said that they administer the postexperimental inquiry before debriefing (though 1 said that such an inquiry was only used in the beginning stages of the research). Ten percent of the respondents indicated having participants complete the postexperimental inquiry both before and after debriefing. Only 1 researcher indicated that he or she administers the postexperimental inquiry after debriefing, and 1 researcher said that he or she asks open-ended questions during the study.

A majority of the respondents (57%) indicated that they conduct the postexperimental inquiry as an interview with the experimenter, while 23% administer the inquiry on the computer, and 21% administer the inquiry as a paper-and-pencil survey (the percentages sum to over 100% because 4 researchers indicated using more than one method to administer the postexperimental inquiry). No one indicated having a third person (e.g., another experimenter) administer the postexperimental inquiry. Four of the respondents did not answer this question.

In terms of how the data collected from the postexperimental inquiry are used, 58% of the respondents indicated that they discard the data (i.e., they do not include the data in statistical analyses) when participants indicate full awareness of the deception or purpose of the study. Eight percent said that they discard data when a participant indicates suspiciousness, and 27% indicated using statistical procedures to control for suspiciousness and/or awareness. Three percent indicated that they used the data in other ways, and 3 researchers did not answer.

Most of the respondents who completed the survey and indicated using a postexperimental inquiry in research involving deception provided us with the questions that they typically ask of participants during the inquiry (only 7 participants who indicated using postexperimental inquiries did not provide the questions that they typically ask of participants). At the most basic level, most researchers ask participants what they thought the study was about (13% only asked participants this one question). Researchers might also include a question asking participants whether they thought that anything was weird, strange, odd, or out of place during the study (21% indicated asking participants these first two questions). A majority of those who completed the survey (55%) indicated using a funnel debriefing, beginning with very basic questions (e.g., “Do you have any questions about the study?”) and ending with more specific questions about the study before debriefing participants.

Based on this sample, it would appear that most research involving the element of deception does include a postexperimental inquiry of some form. Two limitations should be noted, however. First, only 77 individuals completed the survey, but it is likely that many more social psychologists at least occasionally include deception in their research. Second, the sample might have been biased in that researchers who do use a postexperimental inquiry might have been more likely to complete the survey than those who do not use a postexperimental inquiry. Nonetheless, these results are useful in terms of understanding how frequently researchers do employ a postexperimental inquiry and how it is commonly structured.

Study 2

The purpose of Study 2 was to assess how willingly participants admit to suspiciousness, awareness of deception, and prior knowledge during the postexperimental inquiry. Further, the present study assessed whether mood state, reward, and the format of the postexperimental inquiry altered participant responses to postexperimental inquiry questions.

Mood state

Previous research has shown that mood state affects participant engagement in a study. For instance, Nichols and Maner (2008) found that happier participants were more likely to engage in behavior that provided possible support for the hypothesis. Given that past research suggests that positive mood increases participant willingness to aid the research process, it was predicted that participants in a more positive mood would be less inclined to acquiesce with the demands of the postexperimental inquiry, since they are likely to think that divulging such information would hinder the research.

Reward

Reward has been used in previous research to encourage participant honesty on postexperimental inquiries. For instance, Cryder, Lerner, Gross, and Dahl (2008) told participants that they would receive a $5 gift certificate for Amazon.com if they correctly guessed the study hypothesis, in order to encourage truthful responses on the demand-awareness questions. Past research has not, however, examined whether offering a reward improves participant honesty on the postexperimental inquiry. We predicted that offering a reward would increase participant willingness to admit suspicion or awareness of the deception and to having received information about the study.

Postexperimental inquiry format

Although Aronson et al. (1990) suggested that the postexperimental inquiry should take place as an interview between participants and the experimenter, Newberry (1973) found that participants were more likely to admit to having received information about the study when questions were presented on a questionnaire rather than by the experimenter. Further, Nichols and Maner (2008) suggested that reduced experimenter–participant interaction could diminish demand on the participants. As a result, we predicted that participants would be more likely to admit to suspicion and/or awareness of the deception, as well as to having prior knowledge about the study, when completing a computerized questionnaire than when interviewed by the experimenter. Although Newberry examined whether this factor impacted participants’ willingness to admit to having information about a study, we wanted to determine whether this would apply to participants divulging any suspicion or awareness of the deception as well.

Method

Participants

The participants were 624 Introduction to Psychology students (219 males, 405 females) with ages ranging from 16 to 57 years (M = 20.76, SD = 5.31). Participants received course credit.

Materials and procedure

Participants who had been led to believe the study was about social perceptiveness were greeted by an experimental confederate in the hallway upon arrival at the lab. After initiating a conversation with the participant, the confederate either informed the participant of the study’s ostensible hypothesis (informed condition, n = 313) or withheld this information during the interaction (naïve condition, n = 311). The experimenter was blind to this condition, reducing the possibility of experimenter bias. Participants informed of the ostensible purpose were told

Oh, I did that study a couple of weeks ago. They have you do some sort of social perception task, and then say you either did really good or really bad on the task. None of it is true, though; they are just trying to trick you. They are trying to manipulate your mood.

Following the interaction with the confederate, an experimenter brought participants into the lab, and after signing the consent form, participants engaged in a timed social perception task. They were told that social perception involves the ability to accurately judge others’ moods, personality traits, and motives, and that they would read descriptions of two individuals and then answer several questions about each person’s personality traits and characteristics. Participants were given 10 min to complete this task. The experimenter then ostensibly scored the social perception task and informed participants of either success or failure on the task. Participants in the success condition (n = 316) were told, “On average, college students get around 36 out of the 60 questions correct, around 60%. In your case, you did far above average, getting 54 out of 60, or an impressive 90%, correct.” Those in the failure condition (n = 308) were told, “On average, college students get around 36 out of the 60 questions correct, around 60%. In your case, you did far below average, getting only 18 out of 60, or 30%, correct.” The purpose of the success or failure feedback was to induce a positive or a negative mood state, respectively (see Nummenmaa & Niemi, 2004). Participants then completed the Brief Mood Introspection Scale (BMIS; Mayer & Gaschke, 1988).

Participants next completed the postexperimental inquiry before being debriefed either alone on a computer-based survey (n = 316) or by being interviewed by an experimenter (n = 308). In addition, half of the participants were offered a small reward ($5 gift certificate to Amazon.com) for correctly stating the purpose of the study when completing the postexperimental inquiry (n = 309), whereas the other participants were offered no such reward (n = 315). Prior to being given the postexperimental inquiry, all participants were told

We are interested in assessing participants’ perceptions of the study, and would like your feedback to several questions. Your answers to these questions will in no way affect the research credit you will receive, nor how we use your data. We just want to ensure that the design of the study is sound, and we need your feedback about the study in order to do so.

This statement was designed to promote scientific integrity (see Golding & Lichtenstein, 1970), as well as to enlist the aid of participants (see Aronson et al., 1990). Participants were then asked to answer several questions about the study and were instructed to elaborate on any question they answered “yes” to. Participants were then debriefed by the experimenter and told the true purpose of the study. Before leaving the lab, participants were asked to answer a few more questions about the study on a computer-based survey. (Refer to the Appendix for a list of the questions participants were asked before and after debriefing.)

Participants’ answers before and after debriefing were coded for awareness of the experimental manipulation—defined as admitting to possessing knowledge and/or suspicion about the manipulation, the purpose of the study, and/or the experimental deception—and for admission of prior information—defined as admitting to having received prior information about the study. These two dependent variables were coded on a 1–5 scale (1 = no awareness or admission, 5 = complete awareness or admission) by two trained raters unaware of the conditions to which participants were randomly assigned (see Table 1 for details about the rating scales used to code the dependent measures). A third independent rater resolved any discrepancies between the two raters.

Table 1 Rating scales used to code dependent measures in Studies 2 and 3
Full size table

Results

Intraclass correlations were calculated for the four dependent variables (DVs) to assess interrater reliability. Chronbach’s α ranged from .74 – .96, indicating good interrater reliability. Next, the means, standard deviations, and bivariate correlation coefficients were calculated (see Table 2).

Table 2 Overall zero-order correlations, means, standard deviations, and intraclass correlation coefficients for the four dependent measures in Study 2
Full size table

A 2 (informed or naïve of the study’s ostensible purpose) × 2 (success or failure feedback on the social perception task) ANOVA was conducted with mood valence from the BMIS as the DV, to determine whether the social perception task successfully altered participants’ mood state. The results showed a main effect of success/failure on mood, F(1, 620) = 13.18, p < .001, Cohen’s d = 0.29, indicating that participants who received success feedback reported significantly more positive mood (M = 25.23, SD = 3.52) than those who received failure feedback (M = 24.16, SD = 3.84). There was not, however, a significant interaction between the informed/naïve and success/failure conditions, nor a main effect for informed/naïve, suggesting that being informed of the ostensible purpose did not alter participants’ mood following the success/failure manipulation. Mood was not significantly correlated with any of the DVs, all rs < .10.

A 2 (informed or naïve) × 2 (success or failure) × 2 (reward or no reward) × 2 (computer survey or experimenter interview) MANOVA was conducted with awareness and admission before debriefing entered as the DVs. For awareness, there were no significant interactions; however, there were significant main effects of informed/naïve, F(1, 608) = 9.62, p < .01, d = 0.24; reward/no reward, F(1, 608) = 11.49, p < .01, d = 0.27; and computer survey/experimenter interview, F(1, 608) = 12.25, p < .001, d = 0.28. These results show that participants who were informed of the ostensible purpose, were told that they would receive a reward for correctly stating the purpose of the study, and completed the postexperimental survey on a computer reported greater awareness of the experimental manipulation (see Table 3 for means and standard deviations). For admission, there was a significant two-way interaction between success/failure and reward/no reward, F(1, 608) = 4.61, p < .05, ω2 = .001, indicating that of the participants who received a reward, those in the failure condition were more likely to admit to having prior information than those in the success condition, whereas the opposite results were found for participants who did not receive a reward (see Fig. 1). There was also a significant main effect of informed/naïve, F(1, 608) = 18.09, p < .001, d = 0.18, showing that participants informed of the ostensible purpose of the study (M = 1.21, SD = 0.72) were more likely to admit to having received prior information about the study than those naïve as to the ostensible study purpose (M = 1.10, SD = 0.44). Although mood was not significantly correlated with any of the dependent measures, mood was entered as a covariate into a 2 × 2 × 2 × 2 MANCOVA to determine whether mood could account for these findings. The results remained unchanged.

Table 3 Means and standard deviations for awareness of the experimental manipulation before debriefing for the informed/naïve, reward/no reward, and computer survey/experimenter interview conditions in Study 2
Full size table
Fig. 1
figure 1

Interaction between success/failure and reward/no reward for awareness of the experimental manipulation before debriefing in Study 2

Full size image

A 2 (informed or naïve) × 2 (success or failure) × 2 (reward or no reward) MANOVA was conducted with awareness and admission after debriefing entered as the DVs (the Computer Survey/Experimenter Interview factor was not included in these analyses because all participants completed the postdebriefing questions on the computer). There was a significant main effect of informed/naïve for the awareness analysis, F(1, 616) = 8.52, p < .01, d = 0.23 (informed M = 2.50, SD = 1.63; naïve M = 2.14, SD = 1.56), and for the admission analysis, F(1, 616) = 57.47, p < .001, d = 0.60 (informed M = 1.62, SD = 1.23; naïve M = 1.07, SD = 0.38). There were not, however, any significant interactions or other main effects.

We also conducted two 2 × 2 ANOVAs with Time (before and after debriefing) as a repeated measures factor and Informed/Naïve as a between-subjects factor for both awareness and admission. For awareness, there was a significant main effect of time, F(1, 622) = 70.33, p < .001, d = 0.40, indicating that awareness scores were significantly higher after (M = 2.32, SD = 1.60) than before (M = 1.78, SD = 1.00) debriefing. There was not, however, a significant interaction effect. For admission, there was a significant interaction between time and informed/naïve, F(1, 622) = 25.54, p < .001, ω2 = .04, indicating that naïve participants’ admission scores were similar before and after being debriefed, whereas informed participants’ admission scores were higher after than before debriefing (see Fig. 2). There was also a significant main effect of time, F(1, 622) = 16.13, p < .001, d = 0.14, showing that participants’ admission scores were significantly lower before (M = 1.22, SD = 0.71) than after (M = 1.34, SD = 0.95) debriefing.

Fig. 2
figure 2

Interaction between time (before and after debriefing) and informed/naïve for admission of prior information in Study 2

Full size image

Discussion

As expected, participants informed of the ostensible study purpose had higher awareness and admission scores than those naïve to the deception. Informed participants, however, had relatively low awareness and admission scores, certainly lower than expected. In fact, the awareness and admission scores we found in the present study were even lower than those found by Golding and Lichtenstein (1970). For example, the average awareness score in the Golding and Lichtenstein study for informed participants was 2.70 (SD = 1.03) on a 1–5 scale, and the average admission score for informed participants was 2.40 (SD = 1.05) on a 1–4 scale. The present findings suggest that, in general, participants do not readily confess to possessing suspiciousness or awareness of the experimental deception or true purpose of the study. Furthermore, participants appear to be even less likely to admit to having received information about the study from another individual than they are to share potential suspicions or awareness. Our findings are consistent with previous research and, unfortunately, suggest that the typical postexperimental inquiry employed in deception research does not appropriately detect suspiciousness, awareness of deception, or prior knowledge about the study (also see Nichols & Maner, 2008).

Offering participants a reward for correctly stating the purpose of the study appeared to increase participants’ willingness to indicate suspiciousness/awareness about the deception. Participants who were offered a small reward before completing the postexperimental inquiry had higher awareness scores before debriefing than did participants not offered a reward. The difference between the two groups was, however, quite small. This suggests that offering a reward to participants may increase participants’ willingness to admit suspicion/awareness of the deception, but perhaps other types of rewards could be stronger reinforcers for such behavior (e.g., offering additional course/extra credit to participants from a general psychology participant pool).

There was also a significant interaction between reward/no reward and success/failure on admission prior to debriefing. Specifically, participants who received success feedback and were offered a reward were less likely to admit to having received information from the confederate than those who received success feedback but were not offered a reward, whereas the opposite results were found for participants who received failure feedback. It is not immediately apparent why this was so, and future research should examine this finding further.

Although we did offer a reward to half of the participants for correctly stating the purpose of the study, we did not specifically offer a reward for admitting to having received information about the study from the confederate. We speculated that even if participants were not willing to implicate another, they would at least admit to awareness of the deception. Our results suggest that this did not necessarily occur. Although offering such a reward would likely lead to an overestimate of the number of participants who received prior information, this should be tested empirically in future research.

The results also indicated that participants completing a computerized postexperimental survey were more likely to admit to suspicion/awareness of the deception before debriefing than were participants interviewed by the experimenter. The difference between the two groups was, however, relatively small, which might be explained by the fact that participants were not required to spend a predetermined amount of time on the postexperimental inquiry. This might have caused those completing the computer survey to rush through the survey, decreasing the likelihood that they would divulge any suspicion/awareness that they had about the study.

Participants completing the computerized postexperimental survey were not more likely to admit to receiving information about the study from the confederate, which conflicts with Newberry’s (1973) findings. This might be explained by the fact that we did not phrase the question positively (e.g., “participants were paid to tell others about this study—did you hear anything about it?”) or negatively (e.g., “it is necessary for us to know whether participants had any knowledge about this study before participating”), as did Newberry. Rather, we simply asked participants whether they had any knowledge about the study before participating. Aronson et al. (1990) suggests that asking for this information in such a direct manner may deter participants from divulging such information.

Although the success/failure manipulation did successfully manipulate mood, mood did not appear to influence participants’ answers on the postexperimental inquiry. The difference in self-reported mood between those in the success and failure conditions was rather small, though. Future research should therefore employ a stronger mood manipulation in order to determine whether mood state has any impact on participants’ willingness to divulge information during the postexperimental inquiry.

Furthermore, self-reported mood did not differ as a function of whether participants were informed or naïve as to the ostensible study purpose, which was surprising. These results might be explained by one of three possibilities. The most likely is that these results are due to the participant expectancy effect. That is, participants informed of the ostensible study purpose reported the mood state that they thought was expected of them. A second possibility is that even though informed participants knew that the feedback was false, the success/failure feedback might still have affected participants’ moods. Third, perhaps informed participants either did not attend to the information given by the confederate or did not believe this information, although it is unlikely that this was the case for most of the participants in the informed group.

Both informed and naïve participants had higher awareness scores after debriefing than before. This suggests that conducting the postexperimental inquiry after debriefing may have increased participants’ admissions of suspicion/awareness, but that they also may have been more likely to admit to knowledge of deception, even if they had none, in order to increase credibility after being deceived by the experimenter (see Aronson et al., 1990). Informed participants also had higher admission scores after than before debriefing; however, admission scores were still low for informed participants after debriefing.

In addition to the limitations discussed above, other weaknesses should be mentioned. One, although participants were told that we wanted their assistance to improve the methodology of the study before they completed the postexperimental inquiry, we believe that this statement could have used stronger language to stress the importance of their cooperation for the scientific integrity of the study (see Aronson et al., 1990; Golding & Lichtenstein, 1970). Additionally, we did not prompt participants when asking them whether they had received prior information about the study. Aronson et al. suggested that experimenters should encourage participants to admit to being given information about the study, such as by stressing that although information is occasionally shared mistakenly, the experimenter is not interested in how or from whom this information was received (see Aronson et al., 1990). As Aronson and colleagues pointed out, participants are unlikely to admit to having received information about a study if not prompted for fear of implicating others who shared information.

Study 3

Since several limitations were noted in Study 2, Study 3 was designed to improve upon the methods employed in Study 2 and to further examine the impact of rewards on participants’ willingness to admit to suspicion, awareness, and/or prior information on the postexperimental inquiry. Specifically, we offered participants a different reward in Study 3: extra research credit. Because Introduction to Psychology students at our university are required to earn research credits (either by participating in research or by completing alternative activities), we believed that extra research credit might be a more powerful incentive to participants than a $5 gift certificate.

Several other improvements were made. First, the interaction between the experimental confederate and the participant was moved from the hallway outside of the lab into the laboratory. This was done in order to minimize distractions in an attempt to increase participants’ attention to the information the confederate shared with the participant. Second, all participants completed the postexperimental inquiry on a computer without the experimenter present in the room, and they were told that they had to spend at least 5 min on the questions. Third, a stronger scientific integrity statement was included with the postexperimental inquiry. Finally, more questions were asked during the postexperimental inquiry, and, following Aronson et al.’s (1990) suggestions, we improved the wording on the question assessing for prior information about the study.

Method

Participants

A group of 139 Introduction to Psychology students (102 females, 37 males) with ages ranging from 18 to 48 years (M = 21.91, SD = 6.31) volunteered to participate in the study. Participants received course credit for taking part in the study.

Materials and procedure

Participants were led to believe that the study was exploring the link between personality type and writing ability/style. When participants arrived at the lab, an experimental confederate was also waiting in the hallway outside of the lab, but did not initiate any conversation. The experimenter brought the participant and the confederate into the lab and seated them at separate desks within the same room. The experimenter asked them to read and sign different informed consent forms, and said he/she would return in a few minutes. After the experimenter left the room, the confederate initiated a conversation with the participant, either informing the participant of the study’s ostensible purpose (informed condition, n = 69) or withholding this information during the interaction (naïve condition, n = 70). Participants informed of the ostensible purpose were told: “Oh, I did that study a couple of weeks ago. They have you fill out a personality questionnaire, and then they give you a fake personality profile. It’s all just a trick used to manipulate your feelings.” The experimenter was blind to whether participants were informed or naïve, reducing the possibility of experimenter bias.

After the interaction with the confederate, an experimenter took the participant and the confederate into separate, smaller rooms within the lab. Participants were first asked to complete the Eysenck Personality Questionnaire (EPQ; Eysenck & Eysenck, 1975) and were told that they would receive some information on what the survey said about their personality. After participants completed the EPQ, the experimenter left the room for a few minutes to score the Extraversion scale. The experimenter returned and gave participants accurate feedback on their Extraversion score from the EPQ, explaining its significance. Then the experimenter told participants that either introversion or extraversion was a bad thing for interpersonal relationships, and that they were the type of person who would end up alone later in life. This protocol was adapted from Twenge, Baumeister, Tice, and Stucke (2001; refer to this previous study for the actual feedback given to participants) and has been employed previously in several studies (e.g., Baumeister, DeWall, Ciarocco, & Twenge, 2005; Baumeister, Twenge, & Nuss, 2002; Twenge et al., 2001; Twenge, Catanese, & Baumeister, 2002, 2003).

After receiving this feedback, participants completed a mood questionnaire (adapted from Buckley, Winkel, & Leary, 2004) and were then instructed to write a short essay reflecting their opinions of the smoking policy at the university. Participants were given 10 min to write the essay. They next completed the postexperimental inquiry on the computer (see the Appendix for the instructions and questions). Half of the participants were told that they would receive one additional research credit if they correctly stated the purpose of the study (reward condition n = 69; no-reward condition n = 70). Participants were then debriefed and asked to answer a few more questions about the study. Finally, participants were thanked for participating in the study, received their research credit, and left the lab.

Participants’ answers on the postexperimental inquiry before debriefing were coded for awareness of the experimental manipulation—defined as admitting to possessing knowledge and/or suspicion about the manipulation, the purpose of the study, and/or the experimental deception—and admission of prior information, defined as admitting to having received prior information about the study. The two DVs were coded on a 1–5 scale (1 = no awareness or admission, 5 = complete awareness or admission) by two trained raters unaware of the conditions to which participants were randomly assigned (see Table 1 for details about the rating scales used to code the dependent measures). A third independent rater resolved any discrepancies between the two raters.

Results

Intraclass correlations were calculated for awareness and admission prior to debriefing to assess interrater reliability. For awareness, Chronbach’s α was .96, and for admission, Chronbach’s α was .95, indicating excellent interrater reliability for both variables. Means, standard deviations, and bivariate correlation coefficients were next calculated for awareness, admission, and self-reported mood. Scores were calculated for positive mood, feelings of belonging, depressed mood, hurt feelings, anger, anxiety, and self-conscious emotions from the mood scale. The overall mean (for all participants) for awareness was 2.99 (SD = 1.64), and the overall mean for admission was 1.44 (SD = 1.07). The correlation between these two variables was small, r = .08 (n.s.). Positive mood was negatively correlated with awareness, r = −.22, p < .01, indicating that participants with greater self-reported positive mood had lower awareness scores. Self-conscious emotion (r = .27, p < .01) was positively correlated with admission, indicating that those with greater self-reported self-conscious emotions had higher admission scores. See Table 4 for all of the means, standard deviations, and correlation coefficients.

Table 4 Means, standard deviations, and zero-order correlations for the dependent variables and mood subscale scores in Study 3
Full size table

A one-way MANOVA was conducted to examine whether there were differences in self-reported mood between informed and naïve participants. The results showed no significant differences between the two groups on any of the mood scores. A 2 (informed or naïve) × 2 (reward or no reward) MANOVA was then conducted with awareness and admission before debriefing entered as the DVs. There were significant main effects of informed/naïve and reward/no reward on both awareness and admission (see Table 5 for F scores, p values, Cohen’s d effect sizes, means, and standard deviations for each main effect). The results showed that informed participants were more likely to indicate awareness of the purpose of the study and to admit to having had prior information about the study than those who were naïve. Participants receiving a reward were also more likely to indicate awareness of the purpose of the study than those not receiving a reward, though they were less likely to admit to having prior information about the study than those who did not receive a reward. There was also a marginally significant interaction between the informed/naïve and reward/no reward on admission, F(1, 135) = 3.75, p = .05. The results showed that informed participants not offered a reward were more likely to admit to prior information than those in the other three conditions (see Fig. 3). Because some of the mood scores were significantly correlated with the dependent measures, the mood subscale scores were entered as covariates into a 2 × 2 MANCOVA to determine whether mood could account for these findings. The results remained unchanged.

Table 5 Main effects for the informed/naïve and reward/no reward factors on awareness of the experimental manipulation and admission of prior information for Study 3
Full size table
Fig. 3
figure 3

Interaction between informed/naïve and reward/no reward for admission of prior information in Study 3

Full size image

Discussion

Consistent with the results of Study 2, participants informed of the purpose of the study by an experimental confederate had higher awareness and admission scores than those naïve to the deception. Participants in Study 3 were more likely to admit to suspicion and/or awareness of the deception or purpose of the study than were participants in Study 2 (Study 3 M = 2.99, Study 2 M = 1.78). This indicates that the improvements made in Study 3 may have helped encourage participants to be more forthcoming in communicating any suspiciousness or awareness that they had about the study. Awareness scores in Study 3 were still quite low, however; only 36 of 69 participants in the informed condition received a score of 4 or 5 on awareness. Nearly half of the participants with compromising information about the study did not reveal that information. Admission scores were largely identical between Studies 2 and 3 (Study 3 M = 1.44, Study 2 M = 1.22). These improvements do not appear to have had any impact on participants’ willingness to admit to receiving detailed information about the study prior to participating.

Study 3 also showed that participants offered a reward (one extra research credit) for correctly stating the purpose of the study had higher awareness scores, which is consistent with the results of Study 2. In fact, it appears that offering the extra research credit was more successful than offering a $5 gift certificate, because awareness scores were quite a bit higher for those offered a reward in Study 3 (M = 3.30) than for those offered a reward in Study 2 (M = 1.94). Unlike in Study 2, however, participants who received a reward in Study 3 had lower admission scores than participants who did not receive a reward. In addition, there was a marginally significant interaction between informed/naïve and reward/no reward for admission, showing that informed participants not offered a reward were more likely to admit to having received prior information about the study than were informed participants offered a reward and naïve participants who either were or were not offered a reward. These results suggest that offering a reward decreases the likelihood that participants will share that someone else (e.g., another participant) gave them information about the study. This is perhaps because participants fear that they will not receive the reward if they reveal such information, though this is a speculative explanation. It is therefore suggested that researchers offer rewards to participants during the postexperimental inquiry with caution, as the reward may increase participants’ willingness to admit to suspicion or awareness but decrease admission of prior information.

The results showed that participants who reported more positive mood after receiving rejection feedback were less likely to report suspicion or awareness of the purpose of the study. These results are somewhat surprising, because it might be expected that after receiving feedback that they would be alone later in life, participants who were suspicious of this feedback, or who knew it to be false, would report a more positive mood. This result suggests that perhaps participants in a more positive mood are less willing to admit to suspicion or awareness of the purpose of the study. This is somewhat consistent with the Nichols and Maner (2008) finding that happier participants were more likely to engage in behavior that provided possible support for the hypothesis. Alternatively, these results might also suggest that participants less suspicious or aware of the deception or true purpose of the study had a more positive mood state. That is, perhaps being suspicious of the feedback or aware of the fact that one is being deceived could result in reduced positive affect.

The results also showed that participants who reported greater self-conscious emotions (e.g., embarrassed, humiliated, ashamed) were more likely to admit to having received prior information about the study. As in Study 2, self-reported mood did not differ as a function of participants being informed or naïve as to the ostensible study purpose. The most likely explanation for this result is that the lack of difference between the two groups is due to the participant expectancy effect: Participants informed of the ostensible study purpose reported the mood state they thought was expected of them.

Study 4

Even with the improvements made in Study 3, participant awareness and admission rates remained lower than what we had hoped to find, although awareness scores were much improved over Study 2. Unfortunately, the designs of Studies 2 and 3 did not give much indication as to why some participants were unwilling to share their suspicions and/or awareness with the researcher during the postexperimental inquiry. Study 4 was designed to assess why participants might be reticent to share their suspicions or knowledge about a study during the postexperimental inquiry.

Method

Participants

A total of 456 undergraduate students volunteered to participate in the study. Information about the participants’ ages and gender was not collected. Participants received course credit for completion of the study.

Materials and procedure

All participants completed the study online. After reading the informed consent document and proceeding to the next page, participants completed the Big Five Inventory (BFI; John, Naumann, & Soto, 2008). Participants were then asked to read seven scenarios depicting situations that participants might encounter when participating in psychological research and to indicate (1) what they would do in that situation, and why, and (2) what they think the average participant would do in that situation, and why (see the Appendix for the scenarios presented to participants). Specifically, participants were asked how likely they (and the average participant) would be to tell the experimenter about their suspicions and/or awareness, on a 1–5 Likert scale (1 = not at all likely, 5 = very likely), if they were not asked or prompted by the experimenter to do so, if they were asked directly by the experimenter, if they were asked by a third person (e.g., another experimenter, but not the one they interacted with during the study), if they were asked on a paper-and-pencil survey, and if they were asked on a computer survey. They were also asked to indicate which of the following statements would influence their (and the average participant’s) willingness to either tell or not tell the experimenter about their suspicions and/or awareness: “I would be worried about ruining the study if I did/did not tell,” “I would be worried about not receiving credit or payment for participating in the study if I did/did not tell,” “I would be worried about looking like a fool if I did/did not tell,” “I would be worried about getting into trouble if I did/did not tell,” “I would be worried about getting another person into trouble if I did/did not tell (e.g., another student, another participant, roommate, friend, the experimenter),” or “other.” Participants were asked to select all of the options that applied.

Results and discussion

Because the seven scenarios were similar, responses for each scenario were combined. On average, ratings were lower for each situation (i.e., not prompted, asked directly by an experimenter, asked by a third person, asked on a paper-and-pencil survey, or asked on a computer survey) when participants were rating the average participant than when they were rating themselves. Participants may have been engaging in impression management, so they rated themselves as more likely to share this information than they rated others. If this is true, participants’ ratings of the average participant might be more accurate than participants’ ratings of themselves.

Participants indicated that the willingness to share their suspicions and/or awareness with the experimenter would be lowest when they were not asked or prompted by the experimenter to do so (M = 2.62 for themselves, M = 2.24 for the average participant), and highest when they were asked on a computer questionnaire (M = 3.49 for themselves, M = 3.11 for the average participant). Comparing means between the two situations (not prompted and asked on a computer survey), when rating themselves, Cohen’s d for the comparison was 0.66; when rating the average participant, Cohen’s d was 0.72. When comparing means for completing the postexperimental inquiry on a computer and the other three situations (asked directly by the experimenter, asked by a third person, or asked on a paper-and-pencil survey), the differences between the means were quite small. See Table 6 for all of the means and standard deviations.

Table 6 Overall means and standard deviations for how likely participants (and the average participant) would be to share their suspicions and/or awareness with the experimenter in the five situations presented in the different scenarios in Study 4
Full size table

We examined the zero-order bivariate correlations (Pearson’s r) between the big five personality characteristics on the BFI and participants’ willingness to share their suspicions and/or awareness in the seven scenarios. The only personality trait that was significantly correlated with participants’ willingness to share this information was openness, r = .14–.20, p < .05, for all five situations (see Table 7 for all correlation coefficients). These correlations, although somewhat small, indicate that people with higher openness scores on the BFI might be somewhat more willing to share their suspicions and/or awareness with the researcher during the postexperimental inquiry.

Table 7 Zero-order correlations between personality traits on the Big Five Inventory and how likely participants would be to share their suspicions and/or awareness in the five situations presented in the different scenarios in Study 4
Full size table

When participants were asked what would influence them to either tell or not tell the experimenter about their suspicions and/or awareness, 44% said that they were concerned about ruining the study. Likewise, 39% said that they were concerned about not receiving credit/payment for participating, 35% were concerned about getting another person into trouble, 31% were concerned about getting into trouble, and 26% said that they were concerned about looking like a fool. It therefore appears that participants are most concerned with ruining the study and not receiving credit or payment for participation, although a considerable proportion of participants were concerned about getting themselves or others into trouble if they shared the information. Other responses for influences on whether they would or would not tell included that they would probably figure it was part of the experiment, so they wouldn’t feel it necessary to say anything; whether or not they felt that the information was important enough to share; that they wouldn’t see the point in saying anything, that they wouldn’t care enough to say anything; that they wouldn’t want to be a snitch; that they would be worried that they would upset someone or hurt their feelings, or make someone angry, or that they would be out of line; that they would be worried about being told something was wrong with them if they shared the information; and the attractiveness of the researcher.

When asked what participants thought would influence the average participant to either tell or not tell the experimenter about their suspicions and/or awareness, 59% thought that they would be concerned about not receiving credit/payment for participating. Also, 42% thought that they would be concerned about getting into trouble, 38% that they would be concerned about getting another person into trouble, 35% that they would be concerned about ruining the study, and 33% that they would be concerned about looking like a fool. Respondents thought other participants would be most concerned about not receiving credit or payment for participation, but a substantial number of respondents also thought others would be concerned about getting themselves and others into trouble and about ruining the study. Other responses included that they probably wouldn’t care enough about the study to share the information, that the view of the experimenter as an authority figure would decrease the likelihood of a subject speaking out against the experiment, that they might not realize the impact of their knowledge or suspicion, and that paranoia might keep them from saying anything.

Based on this information, it appears that participants may be most concerned about not receiving credit or payment for participation and about ruining the study if they were to voice their suspicions and/or awareness to the experimenter. However, they also appear to be concerned about getting themselves and others into trouble, and many participants simply might not care enough to say anything, or might not believe it is important to do so (particularly if they believe that anything odd that occurs is simply part of the experiment).

General discussion

The four studies we conducted assessed how frequently researchers using deception employ postexperimental inquiries; the frequency with which participants admit to suspicion and/or awareness on the postexperimental inquiry; the factors that impact their willingness to do so; and reasons why participants might not be forthcoming on the postexperimental inquiry. Study 1 indicated that, at least within the field of social psychology (the field in which deception is most frequently used in research), most researchers do employ a postexperimental inquiry when including deception in research. The methods used, however, vary quite a bit between researchers, which is likely due to the nature of the research being conducted.

Study 2 confirmed findings from previous research, showing that the majority of participants do not share their suspicions and/or awareness with the experimenter during the postexperimental inquiry. Study 2 did show, however, that offering a reward and administering the postexperimental inquiry as a computer survey may increase admission rates. Study 3 improved on the methods used in Study 2, and further explored the use of rewards to encourage participants to be honest on the postexperimental inquiry. Results from Study 3 suggested that the improvements made (e.g., a stronger statement of scientific integrity prior to the postexperimental inquiry, a timed postexperimental inquiry completed on a computer) did increase participants’ willingness to share suspiciousness and/or awareness of the study over Study 2, but did not affect participants’ willingness to admit to having received information about the study before participating. Consistent with the results of Study 2, Study 3 found that offering a reward to participants for correctly stating the purpose of the study increased participants’ willingness to share any suspicions and/or awareness that they had about the study with the experimenter. Offering a reward, however, actually decreased participants’ willingness to admit to having prior information about the study in Study 3.

Finally, Study 4 found that there are likely several reasons why participants do not share their suspicions and/or awareness with experimenters during the postexperimental inquiry. The most influential reasons appear to be that participants are worried that they might ruin the study or that compensation might be withheld upon admission. Participants may also not think it is important to say anything, especially if they believe that whatever happened is part of the study, or they simply may not care.

As the question of participant honesty on postexperimental inquiries has received little attention in the last several decades relative to the volume of deceptive studies published, this study aimed to further this research by examining other factors that might improve awareness and admission rates on postexperimental inquiries. We chose to examine factors that some researchers are already employing in their studies (i.e., offering a reward and administering the postexperimental inquiry on a computer), presumably to improve participant admission on the postexperimental inquiry (though administering the postexperimental inquiry on the computer might also be done for convenience). Unfortunately, the factors examined only modestly improved awareness and admission rates (if they improved them at all). Although the present research did not identify changes to the postexperimental inquiry process that greatly improved participant honesty, we were able to determine that some of the current methods used during this process actually have little impact. In addition, we identified possible reasons why participants are not forthcoming on the postexperimental inquiry. This will allow researchers to conduct future research to examine factors that may address participants’ concerns and to significantly improve the information participants give on postexperimental inquiries.

As stated previously, participant honesty on postexperimental inquiries is vitally important to research employing deception. Participant awareness of the deception or of the true purpose of the study is likely to shape participants’ behaviors and to skew the data collected. In addition, previous research (see Hertwig & Ortmann, 2008a) has suggested that participant suspiciousness may also impact research outcomes adversely. The irony of this is that researchers who deceive participants are relying on those deceived participants not to deceive the researchers in order to interpret their data.Footnote 1 Interestingly, Studies 2 and 3 found no differences in self-reported mood between informed and naïve participants, which may have been due to the participant expectancy effect. That is, participants may act or behave in a manner that they believe is expected of them. This may, at times, result in no differences between suspicious/aware participants and those who are not suspicious/aware. At other times, it may result in large differences between suspicious/aware participants and those not suspicious/aware, especially when the behavior that participants believe is expected of them may make them look bad (i.e., is in conflict with the desire to manage one’s impression). As a result, it is suggested that researchers at the very least statistically compare participants who indicate suspicion and/or awareness on the postexperimental inquiry with those who do not, and perhaps even eliminate data from participants who indicate suspicion and/or awareness from statistical analyses of the data collected.

There are several ethical and methodological reasons why researchers should use deception sparingly (see Hertwig & Ortmann, 2008a). The present study suggests yet another reason. When conducting research involving deception, researchers must be able to accurately assess participant suspicion and awareness. At present, however, the primary method for doing so, the postexperimental inquiry, does not appear to be able to reliably and accurately assess participant suspicion and awareness. It was previously stated that the use of deception enables researchers to collect uncompromised data. But how accurate are the collected data when researchers cannot reliably assess when a participant is suspicious of, detects, or knows about the deception? The present research suggests that offering a reward and conducting the postexperimental inquiry on a computer may modestly increase participant honesty. Additional research is needed, however, that will identify either additional ways to improve on the postexperimental inquiry or viable alternatives to postexperimental inquiries. This is suggested because although deception should be used as a last resort, there remain instances in which deception is a necessary research tool, and alternatives to deception may not always produce comparable results. For instance, rather than use deception, researchers might have participants recall a past experience, imagine a particular scenario, or engage in role-playing, and then assess participants’ thoughts, feelings, and/or behaviors. Thomas and Diener (1990) showed, however, that people tend to overestimate the severity of their emotions when recalling past experiences. Furthermore, recalled events are subject to distortion based on assumptions, expectations, and other a priori theories (e.g., Ross, 1989). Imaginary experiences may be even more susceptible to influence from such intuitions and expectations than are actual experiences.

Conclusion

Deception in research is a widely used and accepted practice in contemporary experimental psychology. The difficulty of accurately detecting participant suspicion and knowledge, as well as the fact that accuracy was only trivially improved by our procedure changes, is alarming. An inability to determine whether participants are aware of experimental deception, either because of demand characteristics present in the study or because of knowledge obtained before participating, compromises the internal validity of the study. It is therefore vitally important that postexperimental inquiries adequately assess whether participants had prior knowledge of the study, were suspicious of any of its aspects, or were aware of the deception. The effectiveness of postexperimental inquiries relies on participants reliably and accurately reporting any suspiciousness or awareness of deception—if participants do not, this could jeopardize the legitimacy of deception as a methodological technique. Unfortunately, results from these and previous studies have suggested that the typical postexperimental inquiry does not adequately assess participant suspicion, awareness, or prior knowledge. Despite testing some new possible influences (i.e., participant mood, offering a reward, administering the postexperimental inquiry on a computer), we are still not certain how to improve the postexperimental inquiry. The consequence is that the data collected in studies that include an element of deception might not always be accurate. Additional research is warranted in this area in order to continue to improve the reliability and validity of this process. In addition, because admission rates in these and previous studies were relatively low, it would be beneficial to do everything possible to reduce or eliminate the incidence of crosstalk (see Edlund et al., 2009, for suggestions).