Skip to main content

Advertisement

SpringerLink
Log in

Predicting instructed simulation and dissimulation when screening for depressive symptoms

  • Original Paper
  • Published:
European Archives of Psychiatry and Clinical Neuroscience Aims and scope Submit manuscript

Abstract

The intentional distortion of test results presents a fundamental problem to self-report-based psychiatric assessment, such as screening for depressive symptoms. The first objective of the study was to clarify whether depressed patients like healthy controls possess both the cognitive ability and motivation to deliberately influence results of commonly used screening measures. The second objective was the construction of a method derived directly from within the test takers’ responses to systematically detect faking behavior. Supervised machine learning algorithms posit the potential to empirically learn the implicit interconnections between responses, which shape detectable faking patterns. In a standardized design, faking bad and faking good were experimentally induced in a matched sample of 150 depressed and 150 healthy subjects. Participants completed commonly used questionnaires to detect depressive and associated symptoms. Group differences throughout experimental conditions were evaluated using linear mixed-models. Machine learning algorithms were trained on the test results and compared regarding their capacity to systematically predict distortions in response behavior in two scenarios: (1) differentiation of authentic patient responses from simulated responses of healthy participants; (2) differentiation of authentic patient responses from dissimulated patient responses. Statistically significant convergence of the test scores in both faking conditions suggests that both depressive patients and healthy controls have the cognitive ability as well as the motivational compliance to alter their test results. Evaluation of the algorithmic capability to detect faking behavior yielded ideal predictive accuracies of up to 89%. Implications of the findings, as well as future research objectives are discussed. Trial Registration The study was pre-registered at the German registry for clinical trials (Deutsches Register klinischer Studien, DRKS; DRKS00007708).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. van Hooft EA, Born MP (2012) Intentional response distortion on personality tests: using eye-tracking to understand response processes when faking. J Appl Psychol 97:301

    PubMed  Google Scholar 

  2. Ziegler M, MacCann C, Roberts R (2011) New perspectives on faking in personality assessment. Oxford University Press, Oxford

    Google Scholar 

  3. Viswesvaran C, Ones DS (1999) Meta-analyses of fakability estimates: implications for personality measurement. Educ Psychol Meas 59:197–210

    Google Scholar 

  4. Boon J, Gozna L, Hall S (2008) Detecting ‘faking bad’ on the Gudjonsson suggestibility scales. Personal Individ Differ 44:263–272

    Google Scholar 

  5. Dorsch F, Wirtz M, Strohmer J (2013) Lexikon der Psychologie (16. Aufl.). Bern Huber

  6. Tett RP, Simonet DV (2011) Faking in personality assessment: a “multisaturation” perspective on faking as performance. Hum Perform 24:302–321

    Google Scholar 

  7. Ziegler M, Buehner M (2009) Modeling socially desirable responding and its effects. Educ Psychol Meas 69:548–565

    Google Scholar 

  8. Schmidt FL, Le H, Ilies R (2003) Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychol Methods 8:206

    PubMed  Google Scholar 

  9. Tourangeau R, Rasinski KA (1988) Cognitive processes underlying context effects in attitude measurement. Psychol Bull 103:299

    Google Scholar 

  10. Marazziti D, Consoli G, Picchetti M et al (2010) Cognitive impairment in major depression. Eur J Pharmacol 626:83–86

    CAS  PubMed  Google Scholar 

  11. Austin M-P, Mitchell P, Goodwin GM (2001) Cognitive deficits in depression: possible implications for functional neuropathology. Br J Psychiatry 178:200–206

    CAS  PubMed  Google Scholar 

  12. McCullough JP Jr (2003) Treatment for chronic depression: Cognitive behavioral analysis system of psychotherapy (CBASP). Educational Publishing Foundation, Denmark

    Google Scholar 

  13. Rehm LP (1977) A self-control model of depression. Behav Ther 8:787–804

    Google Scholar 

  14. Derry PA, Kuiper NA (1981) Schematic processing and self-reference in clinical depression. J Abnorm Psychol 90:286

    CAS  PubMed  Google Scholar 

  15. Hollon SD, Kendall PC (1980) Cognitive self-statements in depression: development of an automatic thoughts questionnaire. Cogn Ther Res 4:383–395

    Google Scholar 

  16. Henry GM, Weingartner H, Murphy DL (1973) Influence of affective states and psychoactive drugs on verbal learning and memory. Am J Psychiatry 130:966–971

    CAS  PubMed  Google Scholar 

  17. Bulbena A, Berrios GE (1993) Cognitive function in the affective disorders: a prospective study. Psychopathology 26:6–12

    CAS  PubMed  Google Scholar 

  18. Pauls CA, Crost NW (2005) Cognitive ability and self-reported efficacy of self-presentation predict faking on personality measures. J Individ Differ 26:194–206

    Google Scholar 

  19. Kaplan HI, Sadock BJ (1998) Kaplan and Sadock’s synopsis of psychiatry: behavioral sciences/clinical psychiatry. Williams & Wilkins Co, Baltimore

    Google Scholar 

  20. Smallberg M (1982) Effort and cognition in depression. Arch Gen Psychiatry 39:593–597

    PubMed  Google Scholar 

  21. Wolkenstein L, Schönenberg M, Schirm E, Hautzinger M (2011) I can see what you feel, but I can’t deal with it: impaired theory of mind in depression. J Affect Disord 132:104–111

    PubMed  Google Scholar 

  22. Wang Y, Wang Y, Chen S et al (2008) Theory of mind disability in major depression with or without psychotic symptoms: a componential view. Psychiatry Res 161:153–161

    PubMed  Google Scholar 

  23. Wilbertz G, Brakemeier E-L, Zobel I et al (2010) Exploring preoperational features in chronic depression. J Affect Disord 124:262–269

    PubMed  Google Scholar 

  24. Ziegler M (2011) Applicant faking: a look into the black box. Ind Organ Psychol 49:29–36

    Google Scholar 

  25. Widder B (2011) Beurteilung der Beschwerdenvalidität. Begutacht Neurol 2:64–92

    Google Scholar 

  26. Van Egmond J, Kummeling I, aan Balkom T (2005) Secondary gain as hidden motive for getting psychiatric treatment. Eur Psychiatry 20:416–421

    PubMed  Google Scholar 

  27. Van Egmond J, Kummeling I (2002) A blind spot for secondary gain affecting therapy outcomes. Eur Psychiatry 17:46–54

    PubMed  Google Scholar 

  28. Aronoff GM, Mandel S, Genovese E et al (2007) Evaluating malingering in contested injury or illness. Pain Pract 7:178–204

    PubMed  Google Scholar 

  29. Lieb M, Palm U, Meyer S et al (2014) Risikofaktoren für Suizide und Suizidversuche an einem Universitätsklinikum. Psychiatr Prax 41:195–199

    PubMed  Google Scholar 

  30. Griffin GA, Normington J, May R, Glassmire D (1996) Assessing dissimulation among social security disability income claimants. J Consult Clin Psychol 64:1425

    CAS  PubMed  Google Scholar 

  31. Cima M, Hollnack S, Kremer K et al (2003) Strukturierter Fragebogen Simulierter Symptome. Nervenarzt 74:977–986

    CAS  PubMed  Google Scholar 

  32. Vossler-Thies E, Stevens A, Engel RR, Licha C (2013) Erfassung negativer Antwortverzerrungen mit der deutschen Fassung des “Personality Assessment Inventory”, dem “Verhaltens-und Erlebensinventar”. Diagnostica

  33. Stevens A, Friedel E, Mehren G, Merten T (2008) Malingering and uncooperativeness in psychiatric and psychological assessment: prevalence and effects in a German sample of claimants. Psychiatry Res 157:191–200

    PubMed  Google Scholar 

  34. Greve KW, Ord J, Curtis KL et al (2008) Detecting malingering in traumatic brain injury and chronic pain: a comparison of three forced-choice symptom validity tests. Clin Neuropsychol 22:896–918

    PubMed  Google Scholar 

  35. Donovan JJ, Dwight SA, Schneider D (2014) The impact of applicant faking on selection measures, hiring decisions, and employee performance. J Bus Psychol 29:479–493

    Google Scholar 

  36. Deutsche R Erwerbsminderungsrente im Zeitablauf 2016. Stat Dtsch Rentenversicher

  37. Hough LM, Eaton NK, Dunnette MD et al (1990) Criterion-related validities of personality constructs and the effect of response distortion on those validities. J Appl Psychol 75:581

    Google Scholar 

  38. Kuncel NR, Borneman MJ (2007) Toward a new method of detecting deliberately faked personality tests: the use of idiosyncratic item responses. Int J Sel Assess 15:220–231

    Google Scholar 

  39. Holden RR, Lambert CE (2015) Response latencies are alive and well for identifying fakers on a self-report personality inventory: a reconsideration of van Hooft and Born (2012). Behav Res Methods 47:1436–1442

    PubMed  Google Scholar 

  40. Stark S, Chernyshenko OS, Chan K-Y et al (2001) Effects of the testing situation on item responding: cause for concern. J Appl Psychol 86:943

    CAS  PubMed  Google Scholar 

  41. Sjöberg L (2015) Correction for faking in self-report personality tests. Scand J Psychol 56:582–591

    PubMed  Google Scholar 

  42. Youngjohn JR, Lees-Haley PR, Binder LM (1999) Comment: warning malingerers produces more sophisticated malingering. Arch Clin Neuropsychol 14:511–515

    CAS  PubMed  Google Scholar 

  43. Converse PD, Oswald FL, Imus A et al (2008) Comparing personality test formats and warnings: effects on criterion-related validity and test-taker reactions. Int J Sel Assess 16:155–169

    Google Scholar 

  44. Heggestad ED, Morrison M, Reeve CL, McCloy RA (2006) Forced-choice assessments of personality for selection: evaluating issues of normative assessment and faking resistance. J Appl Psychol 91:9

    PubMed  Google Scholar 

  45. Komar S, Komar JA, Robie C, Taggar S (2010) Speeding personality measures to reduce faking. J Pers Psychol 9:126–137

    Google Scholar 

  46. Fine S, Pirak M (2016) Faking fast and slow: within-person response time latencies for measuring faking in personnel testing. J Bus Psychol 31:51–64

    Google Scholar 

  47. Holden RR, Book AS (2009) Using hybrid Rasch-latent class modeling to improve the detection of fakers on a personality inventory. Personal Individ Differ 47:185–190

    Google Scholar 

  48. DGPPN B, KBV A, AkdÄ B et al et al (2009) für die Leitliniengruppe Unipolare Depression (2009): S3-Leitlinie/Nationale VersorgungsLeitlinie Unipolare Depression-Langfassung, 1. Aufl DGPPN ÄZQ AWMF–Berlin Düsseld

  49. Kohavi R, Provost F (1998) Glossary of terms. Mach Learn 30:271–274

    Google Scholar 

  50. Russell S, Norvig P, Intelligence A (1995) A modern approach. Artif Intell Prentice-Hall Egnlewood Cliffs 25:27

    Google Scholar 

  51. Kotsiantis SB, Sotiris B, Zaharakis I et al (2007) Supervised machine learning: a review of classification techniques. Emerg Artif Intell Appl Comput Eng 160:3–24

    Google Scholar 

  52. Urbanowicz RJ, Moore JH (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1

    Google Scholar 

  53. Mohri M, Rostamizadeh A, Talwalkar A (2012) Foundations of machine learning. MIT press, Cambridge

    Google Scholar 

  54. Gopinathan KM, Biafore LS, Ferguson WM et al. (1998) Fraud detection using predictive modeling. U.S. Patent No. 5, 819, 226, U.S. Patent and Trademark Office, Washington, DC

  55. Bolton RJ, Hand DJ (2002) Statistical fraud detection: a review. Stat Sci 235–249

  56. Davatzikos C, Ruparel K, Fan Y et al (2005) Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. Neuroimage 28:663–668

    CAS  PubMed  Google Scholar 

  57. Ormerod T, Morley N, Ball L et al (2003) Using ethnography to design a Mass Detection Tool (MDT) for the early discovery of insurance fraud. In: CHI’03 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 650–651

  58. Ortega PA, Figueroa CJ, Ruz GA (2006) A medical claim fraud/abuse detection system based on data mining: a case study in chile. DMIN 6:26–29

    Google Scholar 

  59. Dua P, Bais S (2014) Supervised learning methods for fraud detection in healthcare insurance. In: Machine learning in healthcare informatics. Springer, Berlin, 261–285

    Google Scholar 

  60. Cruz JA, Wishart DS (2006) Applications of machine learning in cancer prediction and prognosis. Cancer Inform 2:59

    Google Scholar 

  61. Koutsouleris N, Meisenzahl EM, Davatzikos C et al (2009) Use of neuroanatomical pattern classification to identify subjects in at-risk mental states of psychosis and predict disease transition. Arch Gen Psychiatry 66:700–712

    PubMed  PubMed Central  Google Scholar 

  62. Thrun S, Pratt L (2012) Learning to learn. Springer Science & Business Media, Berlin

    Google Scholar 

  63. Wittchen HU, Zaudig M, Fydrich T (1997) SKID-I und SKID-II. Strukt Klin Interview Für DSM-IV Hogrefe Gött

  64. Hautzinger M, Keller F, Kühner C (2006) Beck depressions-inventar (BDI-II). Harcourt Test Services Frankfurt

  65. Franke GH, Derogatis LR (2002) Symptom-Checkliste von LR Derogatis: SCL-90-R; deutsche Version. Beltz Test

  66. Borkenau P, Ostendorf F (2008) NEO-FFI: NEO-Fünf-Faktoren-Inventar nach Costa und McCrae, Manual

  67. Brickenkamp R, Schmidt-Atzert L, Liepmann D (2010) Test d2-Revision: Aufmerksamkeits-und Konzentrationstest. Hogrefe Göttingen

  68. Dahlstrom WG, Welsh GS, Dahlstrom LE (1975) An MMPI handbook: research applications. University of Minnesota Press, Minneapolis

    Google Scholar 

  69. Greene RL (2000) The MMPI-2: an interpretive manual. Allyn & Bacon, Boston

    Google Scholar 

  70. Hiller W, Zaudig M, Mombour W (1995) ICD-10 Checklisten. Internationale Diagnosen Checklisten für ICD-10. Hans-Huber, Bern

    Google Scholar 

  71. Grieve R, De Groot HT (2011) Does online psychological test administration facilitate faking? Comput Hum Behav 27:2386–2391

    Google Scholar 

  72. Raudenbush SW, Bryk AS (2002) Hierarchical linear models: applications and data analysis methods. Sage, Thousand Oaks

    Google Scholar 

  73. Bates D, Maechler M, Bolker B, Walker S (2014) lme4: linear mixed-effects models using Eigen and S4. R Package Version 1:1–23

    Google Scholar 

  74. Cortes C, Vapnik V (1995) Support vector machine. Mach Learn 20:273–297

    Google Scholar 

  75. Breiman L (2001) Random forests. Mach Learn 45:5–32

    Google Scholar 

  76. Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38:367–378

    Google Scholar 

  77. Chen T, Guestrin C (2016) Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM, pp 785–794

  78. Bischl B, Mersmann O, Trautmann H (2010) Resampling methods in model validation. In: Workshop on Experimental Methods for the Assessment of Computational Systems (WEMACS 2010), held in conjunction with the International Conference on Parallel Problem Solving From Nature (PPSN 2010), Krakow, Poland, Sept. p 14

  79. Altmann A, Toloşi L, Sander O, Lengauer T (2010) Permutation importance: a corrected feature importance measure. Bioinformatics 26:1340–1347

    CAS  PubMed  Google Scholar 

  80. Bischl B, Lang M, Kotthoff L et al (2016) mlr: machine learning in R. J Mach Learn Res 17:1–5

    Google Scholar 

  81. Carretero-Dios H, Pérez C (2007) Standards for the development and review of instrumental studies: considerations about test selection in psychological research. Int J Clin Health Psychol 7

  82. Insel T, Cuthbert B, Garvey M et al (2010) Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am Psychiatric Assoc 167:748–751

    Google Scholar 

  83. Dillman DA, Redline CD (2004) Testing paper self-administered questionnaires: cognitive interview and field test comparisons. Methods Test Eval Surv Quest 299–317

  84. Ones DS, Viswesvaran C, Reiss AD (1996) Role of social desirability in personality testing for personnel selection: the red herring. Am Psychol Assoc 81:660–679

    Google Scholar 

  85. Ones DS, Viswesvaran C (1998) The effects of social desirability and faking on personality and integrity assessment for personnel selection. Hum Perform 11:245–269

    Google Scholar 

  86. Beck AT, Steer RA, Carbin MG (1988) Psychometric properties of the Beck depression inventory: twenty-five years of evaluation. Clin Psychol Rev 8:77–100

    Google Scholar 

  87. Bagby RM, Nicholson RA, Buis T, Bacchiochi JR (2000) Can the MMPI-2 validity scales detect depression feigned by experts? Assessment 7:55–62

    CAS  PubMed  Google Scholar 

  88. Dodd LE, Pepe MS (2003) Partial AUC estimation and regression. Biometrics 59:614–623

    PubMed  Google Scholar 

  89. Walter SD (2005) The partial area under the summary ROC curve. Stat Med 24:2025–2040

    CAS  PubMed  Google Scholar 

  90. Pannone RD (1984) Predicting test performance: a content valid approach to screening applicants. Pers Psychol 37:507–514

    Google Scholar 

  91. Hargittai E (2009) An update on survey measures of web-oriented digital literacy. Soc Sci Comput Rev 27:130–137

    Google Scholar 

  92. Schmidt–Atzert L, Bühner M (1998) Fehlertypen im Aufmerksamkeits-Belastungs-Test d2. Diagnostica

  93. Fried EI, Epskamp S, Nesse RM et al (2016) What are’good’depression symptoms? Comparing the centrality of DSM and non-DSM symptoms of depression in a network analysis. J Affect Disord 189:314–320

    PubMed  Google Scholar 

  94. Messick S (1988) Meaning and values in test validation: the science and ethics of assessment. ETS Res Rep Ser 1988

Download references

Acknowledgements

The authors would like to acknowledge the kind assistance of Sabrina Immisch and Karoline Ziener (Department of Psychiatry and Psychotherapy, Ludwig-Maximilians-University, Munich, Germany).

Funding

This study was funded by the Dr.-Karl-Wilder-Foundation (German Insurance Association, GDV, Germany). The sponsors had no role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Author information

Author notes

  1. Stephan Goerigk, Sven Hilbert, Frank Padberg and Nina Sarubin authors contributed equally to this work.

Authors and Affiliations

  1. Department of Psychological Methodology and Assessment, Ludwig-Maximilians-University Munich, Munich, Germany

    Stephan Goerigk, Sven Hilbert, Markus Bühner, Clemens Stachl & Nina Sarubin

  2. Department of Psychiatry and Psychotherapy, Ludwig-Maximilians-University, Nussbaumstrasse 7, 80336, Munich, Germany

    Stephan Goerigk, Andrea Jobst, Peter Falkai, Frank Padberg & Nina Sarubin

  3. Hochschule Fresenius, University of Applied Sciences, Munich, Germany

    Stephan Goerigk & Nina Sarubin

  4. Faculty of Psychology, Educational Science and Sport Science, University of Regensburg, Regensburg, Germany

    Sven Hilbert

  5. Department of Statistics, Ludwig-Maximilians-University, Munich, Germany

    Bernd Bischl & Stefan Coors

  6. Department of Clinical Psychology and Psychotherapy, Ludwig-Maximilians-University, Munich, Germany

    Thomas Ehring

Authors
  1. Stephan Goerigk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sven Hilbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Jobst
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Falkai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Bühner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Clemens Stachl
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bernd Bischl
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stefan Coors
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Thomas Ehring
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Frank Padberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Nina Sarubin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantive intellectual contributions to the submitted work in form of conception of the study, and/or acquisition of data, and/or analysis and interpretation of data, and/or drafting or revising the article. NS, SH, SG, FP and AJ participated in acquisition of data and manuscript drafting; TE, PF, CS and MB participated in manuscript editing and interpretation of data; FP and NS were responsible for the concept and conduction of the study; SG, SH, BB, SC and MB were responsible for the integrity of the data and the accuracy of the data analysis; all authors approved the final version of the manuscript and take public responsibility for its content.

Corresponding author

Correspondence to Stephan Goerigk.

Ethics declarations

Conflict of interest

All authors reported no direct or indirect financial or personal relationships, interests and affiliations relevant to the subject matter of the manuscript that have occurred over the last 3 years, or that are expected in the foreseeable future.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goerigk, S., Hilbert, S., Jobst, A. et al. Predicting instructed simulation and dissimulation when screening for depressive symptoms. Eur Arch Psychiatry Clin Neurosci 270, 153–168 (2020). https://doi.org/10.1007/s00406-018-0967-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00406-018-0967-2

Keywords

Search

Navigation