Evaluation of the correlation between gaze avoidance and schizophrenia psychopathology with deep learning-based emotional recognition

Highlights

• We aimed to investigate the correlation between gaze avoidance and psychopathology in patients with schizophrenia through eye movement measurements in interpersonal situations.

• The neutral facial expression of real life objects was standardized and presented as a stimulus using the deep learning-based emotional inference method.

• When observing a neutral facial expression stimulus, patients with schizophrenia with more positive symptoms and general psychopathology symptoms, show more gaze avoidance.

Abstract

Objective

To investigate the correlation between gaze avoidance and psychopathology in patients with schizophrenia through eye movement measurements in real-life interpersonal situations.

Methods

We enrolled 52 clinically stable patients with schizophrenia. Psychopathology was evaluated using the Positive and Negative Syndrome Scale (PANSS), Hamilton Depression Rating Scale, and Hamilton Anxiety Rating Scale. After presenting a visual stimulus, eye movements were measured with Tobii Pro Wearable Glasses 2, and deep learning-based emotional recognition using the residual masking network was used for neutral stimulus verification. Statistical analyses were performed using Pearson’s correlation and regression analyses.

Results

Data of 45 participants with verified stimulus neutrality by deep learning image recognition were used for analysis. The first dwelling time was negatively correlated with the PANSS positive syndrome subscale (p = 0.028), general psychopathology subscale (p = 0.008), total score (p = 0.008), 5-factor positive symptoms (p = 0.035), and 5-factor depression/anxiety symptoms (p = 0.043). The baseline-area of interest (AOI) pupil diameter change was positively correlated with PANSS 5-factor positive symptom scores (p = 0.039). After adjusting for additional variables, the same items had a significant effect on the first dwelling time and baseline-AOI pupil diameter change.

Conclusions

Psychopathology, particularly positive symptoms, was associated with gaze avoidance and pupil diameter in patients with schizophrenia. Evaluating the characteristics of eye movements in patients with schizophrenia will enable better understanding of their symptoms.

Introduction

Schizophrenia is a disease characterized by distortion of thoughts, cognition, and emotion, and is accompanied by a serious decline in social function (Wiersma et al., 2000). The latter is mainly associated with a decrease in social cognition and quality of life (Maat et al., 2012). Social cognition refers to the ability to understand others’ intentions and perform social interactions (Brothers, 1990). To understand the intentions of others, the ability to process information from the others’ faces is important (Roux et al., 2014). Direct gaze is the most important mediator of social interaction and communication. When a picture expressing a neutral emotion is presented, patients with schizophrenia often interpret it as negative emotions (Kohler et al., 2003, Loughland et al., 2002). Further, they show a high frequency of gaze avoidance of facial areas important for social interaction, such as the eyes and nose (Kohler et al., 2003, Loughland et al., 2002).

Several studies have examined the causes of gaze avoidance in patients with schizophrenia. First, positive symptoms in patients with schizophrenia might be associated with gaze avoidance. Patients with delusions fix their gaze less on someone else’s face compared to those without delusion or healthy controls. The perception of threat-related stimuli increases by delusion, which may cause anxiety, resulting in vigilance avoidance (Loughland et al., 2002). Second, several studies have shown that negative symptoms in patients with schizophrenia are associated with gaze avoidance. Tso et al. (2012) analyzed the relationship between eye movements and clinical symptoms in patients with schizophrenia using an eye contact perception task, finding a significant correlation between negative symptoms and gaze avoidance. Third, several studies have shown that social anxiety can lead to gaze avoidance. Patients with social anxiety disorder present an active gaze avoidance of attentional bias to stimuli due to low self-esteem and fear of evaluation (Schneier et al., 2011, Weeks et al., 2019). About 38–65% of patients with schizophrenia are known to have social anxiety disorders (Achim et al., 2011, Temmingh and Stein, 2015).

Eye trackers are widely used in various fields because they are cheaper, simpler to manage, and have no complex task requirements compared to other methods (Beedie et al., 2012). Recently, new wearable eye trackers, which allow to check eye movements and pupil changes in various environments without spatial constraints, are applied to research in various fields, such as psychology, advertising marketing, and psychiatry (Babcock et al., 2003).

With an eye tracker, it is possible to find the areas where the subject's gaze lingers and perform heatmap and scanpath analysis. Simultaneously, changes in pupil diameter are measurable. The change in pupil diameter is caused by the contraction of the pupil sphincter muscle by light (Diamond, 2001). Even under conditions where light intensity is constant, pupil dilation can be caused by signal changes in the brain associated with cognitive and autonomic nerves (Costa and Rudebeck, 2016, Samuels and Szabadi, 2008). Pupil dilation is associated with noradrenaline secretion, and can occur together with an arousal reaction in a threat-related stimulus, accompanied by feelings of fear and anxiety (Leuchs et al., 2017). Therefore, pupil dilation can be used as an indicator of emotional and cognitive processes (Eckstein et al., 2017).

Previous studies using eye trackers stimulated patients with standardized emotional state photos and videos. However, these experimental conditions may not fully reflect real-world situations (Beedie et al., 2011b, Li et al., 2020). To compensate for the limitations of these studies, recent research has tried to present more natural situations. In some studies, natural social scenes were presented as photographs, and experiments conducted using real-time videos such as Skype (Beedie et al., 2011a, Li et al., 2020).

Although several studies have investigated the causes of gaze avoidance in patients with schizophrenia, no clear conclusions have been drawn. In addition, existing studies have evaluated eye movements of patients with schizophrenia by presenting stimuli using photographs or pre-recorded videos, but few directly investigated gaze avoidance in humans in real-world situations.

In this study, we investigated the relationship between psychopathology and gaze avoidance and pupil diameter change using a wearable eye movement tracking device after presenting real-world neutral facial stimuli in patients with schizophrenia.