Abstract
The intentional distortion of test results presents a fundamental problem to self-report-based psychiatric assessment, such as screening for depressive symptoms. The first objective of the study was to clarify whether depressed patients like healthy controls possess both the cognitive ability and motivation to deliberately influence results of commonly used screening measures. The second objective was the construction of a method derived directly from within the test takers’ responses to systematically detect faking behavior. Supervised machine learning algorithms posit the potential to empirically learn the implicit interconnections between responses, which shape detectable faking patterns. In a standardized design, faking bad and faking good were experimentally induced in a matched sample of 150 depressed and 150 healthy subjects. Participants completed commonly used questionnaires to detect depressive and associated symptoms. Group differences throughout experimental conditions were evaluated using linear mixed-models. Machine learning algorithms were trained on the test results and compared regarding their capacity to systematically predict distortions in response behavior in two scenarios: (1) differentiation of authentic patient responses from simulated responses of healthy participants; (2) differentiation of authentic patient responses from dissimulated patient responses. Statistically significant convergence of the test scores in both faking conditions suggests that both depressive patients and healthy controls have the cognitive ability as well as the motivational compliance to alter their test results. Evaluation of the algorithmic capability to detect faking behavior yielded ideal predictive accuracies of up to 89%. Implications of the findings, as well as future research objectives are discussed. Trial Registration The study was pre-registered at the German registry for clinical trials (Deutsches Register klinischer Studien, DRKS; DRKS00007708).
Similar content being viewed by others
References
van Hooft EA, Born MP (2012) Intentional response distortion on personality tests: using eye-tracking to understand response processes when faking. J Appl Psychol 97:301
Ziegler M, MacCann C, Roberts R (2011) New perspectives on faking in personality assessment. Oxford University Press, Oxford
Viswesvaran C, Ones DS (1999) Meta-analyses of fakability estimates: implications for personality measurement. Educ Psychol Meas 59:197–210
Boon J, Gozna L, Hall S (2008) Detecting ‘faking bad’ on the Gudjonsson suggestibility scales. Personal Individ Differ 44:263–272
Dorsch F, Wirtz M, Strohmer J (2013) Lexikon der Psychologie (16. Aufl.). Bern Huber
Tett RP, Simonet DV (2011) Faking in personality assessment: a “multisaturation” perspective on faking as performance. Hum Perform 24:302–321
Ziegler M, Buehner M (2009) Modeling socially desirable responding and its effects. Educ Psychol Meas 69:548–565
Schmidt FL, Le H, Ilies R (2003) Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychol Methods 8:206
Tourangeau R, Rasinski KA (1988) Cognitive processes underlying context effects in attitude measurement. Psychol Bull 103:299
Marazziti D, Consoli G, Picchetti M et al (2010) Cognitive impairment in major depression. Eur J Pharmacol 626:83–86
Austin M-P, Mitchell P, Goodwin GM (2001) Cognitive deficits in depression: possible implications for functional neuropathology. Br J Psychiatry 178:200–206
McCullough JP Jr (2003) Treatment for chronic depression: Cognitive behavioral analysis system of psychotherapy (CBASP). Educational Publishing Foundation, Denmark
Rehm LP (1977) A self-control model of depression. Behav Ther 8:787–804
Derry PA, Kuiper NA (1981) Schematic processing and self-reference in clinical depression. J Abnorm Psychol 90:286
Hollon SD, Kendall PC (1980) Cognitive self-statements in depression: development of an automatic thoughts questionnaire. Cogn Ther Res 4:383–395
Henry GM, Weingartner H, Murphy DL (1973) Influence of affective states and psychoactive drugs on verbal learning and memory. Am J Psychiatry 130:966–971
Bulbena A, Berrios GE (1993) Cognitive function in the affective disorders: a prospective study. Psychopathology 26:6–12
Pauls CA, Crost NW (2005) Cognitive ability and self-reported efficacy of self-presentation predict faking on personality measures. J Individ Differ 26:194–206
Kaplan HI, Sadock BJ (1998) Kaplan and Sadock’s synopsis of psychiatry: behavioral sciences/clinical psychiatry. Williams & Wilkins Co, Baltimore
Smallberg M (1982) Effort and cognition in depression. Arch Gen Psychiatry 39:593–597
Wolkenstein L, Schönenberg M, Schirm E, Hautzinger M (2011) I can see what you feel, but I can’t deal with it: impaired theory of mind in depression. J Affect Disord 132:104–111
Wang Y, Wang Y, Chen S et al (2008) Theory of mind disability in major depression with or without psychotic symptoms: a componential view. Psychiatry Res 161:153–161
Wilbertz G, Brakemeier E-L, Zobel I et al (2010) Exploring preoperational features in chronic depression. J Affect Disord 124:262–269
Ziegler M (2011) Applicant faking: a look into the black box. Ind Organ Psychol 49:29–36
Widder B (2011) Beurteilung der Beschwerdenvalidität. Begutacht Neurol 2:64–92
Van Egmond J, Kummeling I, aan Balkom T (2005) Secondary gain as hidden motive for getting psychiatric treatment. Eur Psychiatry 20:416–421
Van Egmond J, Kummeling I (2002) A blind spot for secondary gain affecting therapy outcomes. Eur Psychiatry 17:46–54
Aronoff GM, Mandel S, Genovese E et al (2007) Evaluating malingering in contested injury or illness. Pain Pract 7:178–204
Lieb M, Palm U, Meyer S et al (2014) Risikofaktoren für Suizide und Suizidversuche an einem Universitätsklinikum. Psychiatr Prax 41:195–199
Griffin GA, Normington J, May R, Glassmire D (1996) Assessing dissimulation among social security disability income claimants. J Consult Clin Psychol 64:1425
Cima M, Hollnack S, Kremer K et al (2003) Strukturierter Fragebogen Simulierter Symptome. Nervenarzt 74:977–986
Vossler-Thies E, Stevens A, Engel RR, Licha C (2013) Erfassung negativer Antwortverzerrungen mit der deutschen Fassung des “Personality Assessment Inventory”, dem “Verhaltens-und Erlebensinventar”. Diagnostica
Stevens A, Friedel E, Mehren G, Merten T (2008) Malingering and uncooperativeness in psychiatric and psychological assessment: prevalence and effects in a German sample of claimants. Psychiatry Res 157:191–200
Greve KW, Ord J, Curtis KL et al (2008) Detecting malingering in traumatic brain injury and chronic pain: a comparison of three forced-choice symptom validity tests. Clin Neuropsychol 22:896–918
Donovan JJ, Dwight SA, Schneider D (2014) The impact of applicant faking on selection measures, hiring decisions, and employee performance. J Bus Psychol 29:479–493
Deutsche R Erwerbsminderungsrente im Zeitablauf 2016. Stat Dtsch Rentenversicher
Hough LM, Eaton NK, Dunnette MD et al (1990) Criterion-related validities of personality constructs and the effect of response distortion on those validities. J Appl Psychol 75:581
Kuncel NR, Borneman MJ (2007) Toward a new method of detecting deliberately faked personality tests: the use of idiosyncratic item responses. Int J Sel Assess 15:220–231
Holden RR, Lambert CE (2015) Response latencies are alive and well for identifying fakers on a self-report personality inventory: a reconsideration of van Hooft and Born (2012). Behav Res Methods 47:1436–1442
Stark S, Chernyshenko OS, Chan K-Y et al (2001) Effects of the testing situation on item responding: cause for concern. J Appl Psychol 86:943
Sjöberg L (2015) Correction for faking in self-report personality tests. Scand J Psychol 56:582–591
Youngjohn JR, Lees-Haley PR, Binder LM (1999) Comment: warning malingerers produces more sophisticated malingering. Arch Clin Neuropsychol 14:511–515
Converse PD, Oswald FL, Imus A et al (2008) Comparing personality test formats and warnings: effects on criterion-related validity and test-taker reactions. Int J Sel Assess 16:155–169
Heggestad ED, Morrison M, Reeve CL, McCloy RA (2006) Forced-choice assessments of personality for selection: evaluating issues of normative assessment and faking resistance. J Appl Psychol 91:9
Komar S, Komar JA, Robie C, Taggar S (2010) Speeding personality measures to reduce faking. J Pers Psychol 9:126–137
Fine S, Pirak M (2016) Faking fast and slow: within-person response time latencies for measuring faking in personnel testing. J Bus Psychol 31:51–64
Holden RR, Book AS (2009) Using hybrid Rasch-latent class modeling to improve the detection of fakers on a personality inventory. Personal Individ Differ 47:185–190
DGPPN B, KBV A, AkdÄ B et al et al (2009) für die Leitliniengruppe Unipolare Depression (2009): S3-Leitlinie/Nationale VersorgungsLeitlinie Unipolare Depression-Langfassung, 1. Aufl DGPPN ÄZQ AWMF–Berlin Düsseld
Kohavi R, Provost F (1998) Glossary of terms. Mach Learn 30:271–274
Russell S, Norvig P, Intelligence A (1995) A modern approach. Artif Intell Prentice-Hall Egnlewood Cliffs 25:27
Kotsiantis SB, Sotiris B, Zaharakis I et al (2007) Supervised machine learning: a review of classification techniques. Emerg Artif Intell Appl Comput Eng 160:3–24
Urbanowicz RJ, Moore JH (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1
Mohri M, Rostamizadeh A, Talwalkar A (2012) Foundations of machine learning. MIT press, Cambridge
Gopinathan KM, Biafore LS, Ferguson WM et al. (1998) Fraud detection using predictive modeling. U.S. Patent No. 5, 819, 226, U.S. Patent and Trademark Office, Washington, DC
Bolton RJ, Hand DJ (2002) Statistical fraud detection: a review. Stat Sci 235–249
Davatzikos C, Ruparel K, Fan Y et al (2005) Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. Neuroimage 28:663–668
Ormerod T, Morley N, Ball L et al (2003) Using ethnography to design a Mass Detection Tool (MDT) for the early discovery of insurance fraud. In: CHI’03 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 650–651
Ortega PA, Figueroa CJ, Ruz GA (2006) A medical claim fraud/abuse detection system based on data mining: a case study in chile. DMIN 6:26–29
Dua P, Bais S (2014) Supervised learning methods for fraud detection in healthcare insurance. In: Machine learning in healthcare informatics. Springer, Berlin, 261–285
Cruz JA, Wishart DS (2006) Applications of machine learning in cancer prediction and prognosis. Cancer Inform 2:59
Koutsouleris N, Meisenzahl EM, Davatzikos C et al (2009) Use of neuroanatomical pattern classification to identify subjects in at-risk mental states of psychosis and predict disease transition. Arch Gen Psychiatry 66:700–712
Thrun S, Pratt L (2012) Learning to learn. Springer Science & Business Media, Berlin
Wittchen HU, Zaudig M, Fydrich T (1997) SKID-I und SKID-II. Strukt Klin Interview Für DSM-IV Hogrefe Gött
Hautzinger M, Keller F, Kühner C (2006) Beck depressions-inventar (BDI-II). Harcourt Test Services Frankfurt
Franke GH, Derogatis LR (2002) Symptom-Checkliste von LR Derogatis: SCL-90-R; deutsche Version. Beltz Test
Borkenau P, Ostendorf F (2008) NEO-FFI: NEO-Fünf-Faktoren-Inventar nach Costa und McCrae, Manual
Brickenkamp R, Schmidt-Atzert L, Liepmann D (2010) Test d2-Revision: Aufmerksamkeits-und Konzentrationstest. Hogrefe Göttingen
Dahlstrom WG, Welsh GS, Dahlstrom LE (1975) An MMPI handbook: research applications. University of Minnesota Press, Minneapolis
Greene RL (2000) The MMPI-2: an interpretive manual. Allyn & Bacon, Boston
Hiller W, Zaudig M, Mombour W (1995) ICD-10 Checklisten. Internationale Diagnosen Checklisten für ICD-10. Hans-Huber, Bern
Grieve R, De Groot HT (2011) Does online psychological test administration facilitate faking? Comput Hum Behav 27:2386–2391
Raudenbush SW, Bryk AS (2002) Hierarchical linear models: applications and data analysis methods. Sage, Thousand Oaks
Bates D, Maechler M, Bolker B, Walker S (2014) lme4: linear mixed-effects models using Eigen and S4. R Package Version 1:1–23
Cortes C, Vapnik V (1995) Support vector machine. Mach Learn 20:273–297
Breiman L (2001) Random forests. Mach Learn 45:5–32
Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38:367–378
Chen T, Guestrin C (2016) Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM, pp 785–794
Bischl B, Mersmann O, Trautmann H (2010) Resampling methods in model validation. In: Workshop on Experimental Methods for the Assessment of Computational Systems (WEMACS 2010), held in conjunction with the International Conference on Parallel Problem Solving From Nature (PPSN 2010), Krakow, Poland, Sept. p 14
Altmann A, Toloşi L, Sander O, Lengauer T (2010) Permutation importance: a corrected feature importance measure. Bioinformatics 26:1340–1347
Bischl B, Lang M, Kotthoff L et al (2016) mlr: machine learning in R. J Mach Learn Res 17:1–5
Carretero-Dios H, Pérez C (2007) Standards for the development and review of instrumental studies: considerations about test selection in psychological research. Int J Clin Health Psychol 7
Insel T, Cuthbert B, Garvey M et al (2010) Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am Psychiatric Assoc 167:748–751
Dillman DA, Redline CD (2004) Testing paper self-administered questionnaires: cognitive interview and field test comparisons. Methods Test Eval Surv Quest 299–317
Ones DS, Viswesvaran C, Reiss AD (1996) Role of social desirability in personality testing for personnel selection: the red herring. Am Psychol Assoc 81:660–679
Ones DS, Viswesvaran C (1998) The effects of social desirability and faking on personality and integrity assessment for personnel selection. Hum Perform 11:245–269
Beck AT, Steer RA, Carbin MG (1988) Psychometric properties of the Beck depression inventory: twenty-five years of evaluation. Clin Psychol Rev 8:77–100
Bagby RM, Nicholson RA, Buis T, Bacchiochi JR (2000) Can the MMPI-2 validity scales detect depression feigned by experts? Assessment 7:55–62
Dodd LE, Pepe MS (2003) Partial AUC estimation and regression. Biometrics 59:614–623
Walter SD (2005) The partial area under the summary ROC curve. Stat Med 24:2025–2040
Pannone RD (1984) Predicting test performance: a content valid approach to screening applicants. Pers Psychol 37:507–514
Hargittai E (2009) An update on survey measures of web-oriented digital literacy. Soc Sci Comput Rev 27:130–137
Schmidt–Atzert L, Bühner M (1998) Fehlertypen im Aufmerksamkeits-Belastungs-Test d2. Diagnostica
Fried EI, Epskamp S, Nesse RM et al (2016) What are’good’depression symptoms? Comparing the centrality of DSM and non-DSM symptoms of depression in a network analysis. J Affect Disord 189:314–320
Messick S (1988) Meaning and values in test validation: the science and ethics of assessment. ETS Res Rep Ser 1988
Acknowledgements
The authors would like to acknowledge the kind assistance of Sabrina Immisch and Karoline Ziener (Department of Psychiatry and Psychotherapy, Ludwig-Maximilians-University, Munich, Germany).
Funding
This study was funded by the Dr.-Karl-Wilder-Foundation (German Insurance Association, GDV, Germany). The sponsors had no role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
Author information
Authors and Affiliations
Contributions
All authors have made substantive intellectual contributions to the submitted work in form of conception of the study, and/or acquisition of data, and/or analysis and interpretation of data, and/or drafting or revising the article. NS, SH, SG, FP and AJ participated in acquisition of data and manuscript drafting; TE, PF, CS and MB participated in manuscript editing and interpretation of data; FP and NS were responsible for the concept and conduction of the study; SG, SH, BB, SC and MB were responsible for the integrity of the data and the accuracy of the data analysis; all authors approved the final version of the manuscript and take public responsibility for its content.
Corresponding author
Ethics declarations
Conflict of interest
All authors reported no direct or indirect financial or personal relationships, interests and affiliations relevant to the subject matter of the manuscript that have occurred over the last 3 years, or that are expected in the foreseeable future.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Goerigk, S., Hilbert, S., Jobst, A. et al. Predicting instructed simulation and dissimulation when screening for depressive symptoms. Eur Arch Psychiatry Clin Neurosci 270, 153–168 (2020). https://doi.org/10.1007/s00406-018-0967-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00406-018-0967-2