Behavioral and electrophysiological signatures of word translation processes

Highlights

• Behavioral responses are faster for backward than forward translation.

• Responses are faster in simultaneous translation than within-language word generation.

• Neural attentional and self-monitoring effects are stronger in translation.

• Lexical access interference needs more control in within-language word generation.

• Late language asymmetry effects reflect differences in conceptual processing.

Abstract

Translation is a demanding process during which a message is analyzed, translated and communicated from one language to another. Despite numerous studies on translation mechanisms, the electrophysiological processes underlying translation with overt production remain largely unexplored. Here, we investigated how behavioral response patterns and spatial-temporal brain dynamics differ in a translation compared to a control within-language word-generation task. We also investigated how forward and backward translation differs on the behavioral and electrophysiological level. To address these questions, healthy late bilingual subjects performed a translation and a within-language control task while a 128-channel EEG was recorded. Behavioral data showed faster responses for translation compared to within-language word generation and faster responses for backward than forward translation. The ERP-analysis revealed stronger early ( < 200 ms) preparatory and attentional processes for between than within word generation. Later (424–630 ms) differences were characterized by distinct engagement of domain-general control networks, namely self-monitoring and lexical access interference. Language asymmetry effects occurred at a later stage (600 ms), reflecting differences in conceptual processing characterized by a larger involvement of areas implicated in attention, arousal and awareness for forward versus backward translation.

Introduction

As multilingualism plays a crucial role in an increasingly globalized and multicultural world, studies investigating the representation and neural processing of multiple languages have gained substantial interest. A particularly demanding process is translation, where encoding of words (or sentences) takes place in the original language followed by the selective retrieval of the target language.

In a comprehensive meta-analysis, Indefrey and Levelt (2004) (updated Indefrey, 2011) identified spatio-temporal correlates for the core processes of within language word generation: accessing the lexical concept of words and lexical selection retrieval around 150–350 ms after stimulus onset in anterior middle temporal regions, is followed by posterior middle-temporal phonological code retrieval, posterior inferior-frontal phonological encoding, and articulatory preparation in supplementary motor areas between 350 and 600 ms initiating the articulation of the word. In addition, the model includes self-monitoring processes mainly mediated by superior temporal regions, but also an involvement of a larger network including the cingulate and insular cortex, supplementary and primary motor areas, cerebellum, thalamus and the basal ganglia (Christoffels et al., 2007, van de Ven et al., 2009).

In contrast to within language processing, less is known about cross language processing, namely the interaction between the first (L1) and second (L2) language in bilingual word processing. Numerous models of bilingual word access have been brought forward to explain how speakers select words in the target language and how co-activation and thus intrusion from the unintended language can be suppressed. One of the most influential models is the Revised Hierarchical Model (RHM; Kroll and Stewart, 1994), which proposes a hierarchical organization of the lexical and the conceptual level, and elaborates the implications of this hierarchical organization for bilingual word production. The RHM distinguishes between two lexicons - one for words of L1 and one for words of L2. These two lexicons are linked to a common conceptual system, which contains the meaning of the words. According to the RHM, L1-L2 translation is slower as compared to L2-L1 translation in bilinguals who acquired their L2 after early childhood and for whom the L1 remains the dominant language. In this model, “both lexical and conceptual links are active in bilingual memory, but the strengths of the links differ as a function of fluency in L2 and relative dominance of L1 to L2” (Kroll and Stewart, 1994, p. 157). As such, the model proposes an asymmetry in the strength of the connections between words and their concepts in the two languages, characterized by stronger links and thus faster access to meaning for L1 words. In bilinguals with a low L2-proficiency, the L2 is assumed to require mediation via the L1 translation equivalent, which in turn leads to slower responses (e.g. Kroll and Stewart, 1994; Sholl, Sankaranarayanan and Kroll, 1995). The Inhibitory Control Model (IC model, Green, 1998), which includes executive-attentional control mechanisms within models of bilingual processing – also referred to as “language control mechanisms” - offers an alternative explanation for this translation-asymmetry. It has been postulated that translation from L1 to L2 requires the inhibition of L1 lemmas in order to produce L2 words. In unbalanced bilinguals, L1 lemmas are assumed to be more active than L2 lemmas, requiring higher attentional resources for the L1 lemmas to be suppressed. Consequently, L1-L2 translation would be slower as compared to L2-L1 translation because the two tasks require differential inhibitory demands (see Sunderman and Kroll, 2006; Kroll et al., 2010 for a review). Basic assumptions of the RHM have been called into question (Brysbaert and Duyck, 2010) and alternative models of bilingual language selection have been suggested (e.g. Bilingual Interactive Activation BIA+ Model, Dijkstra and Van Heuven, 2002; Conceptual Selection Model, CSM, Bloem and La Heij, 2003), proposing non-selective lexical access across languages. In these models, the presentation of a word leads to automatic activation of lexico-semantic information regarding that word in both languages. The similarity of the input word to the internal lexical representations determines their activation, not the word's language membership. Despite large debate about which linguistic models best explain bilingual word production, there seems to be converging evidence that the parallel activation of two languages gives rise to high demands on cognitive control (e.g. Luk et al., 2012; Abutalebi et al., 2013; Kroll and Bialystok, 2013; Kroll et al., 2014).

Together, language models on word processing suggest that early word recognition processes may be similar for first and second language and that the interaction across L1 and L2 may affect higher lexico-semantic, form encoding and attentional processes. La Heij et al. (1996) state that differences between L1-L2 and L2-L1 translation “can be accounted for by assuming that to a large extent both tasks are conceptually mediated, but that second-language words are less efficient in activating their concepts than first-language words” (La Heij et al., 1996, p.663). Dijkstra and Van Heuven (2002, p. 193) state that “until the target word is identified and its language tag retrieved for responding, cross-linguistic interactions can arise within the mental lexicon”. In his IC model, Green (1998) emphasizes the role of the “supervisory attentional system”, especially in cases where “automatic control is insufficient” (p. 69). Furthermore, the models suggest that the spatio-temporal characteristics of the networks underlying translation production could depend on language proficiency and that unbalanced proficiency may lead to asymmetric bilingual word processing. As Kroll and Biyalistok concluded, “language comprehension and production depend on the absolute levels of proficiency of both languages” (Kroll and Bialystok, 2013, p. 2). Green further states that “even relatively fluent bilinguals may continue to show an asymmetry in accessing meaning in their two languages” (Green, 1998, p. 72).

Neuroimaging studies suggest that the involvement of specific neural substrates in translation depends on the source unit (namely words, sentences, supra-sentential text) and the direction of translation (for a review see García, 2013; Hervais-Adelman et al., 2011). For single-word translation, studies using spatially sensitive methods have shown inconsistent results. In their PET study, Klein et al. (1995) found that left inferior and dorsolateral frontal and prefrontal regions are activated for both L1-L2 and L2-L1 translation. Similar regions were also activated during synonym generation and rhyme generation tasks. This involvement of the dorsolateral prefrontal cortex was not replicated in the PET-study of Price et al. (1999), who found increased activity in the anterior cingulate, subcortical structures and regions associated with articulation (anterior insula, cerebellum and supplementary motor area) and decreased activation in several other temporal and parietal language areas associated with the meaning of words. According to Price et al. (1999), one reason for these diverging results could be the subjects’ different L2-expertise levels, assuming that prefrontal involvement would be stronger when proficiency decreases. Moreover, the studies by Klein et al. (1995) and Price et al. (1999) differed in terms of type of translation (overt vs. silent translation) and baseline task (repetition vs. reading). However, both studies revealed exclusively left hemisphere activation for word translation.

There is also a large number of studies using temporally sensitive methods investigating the electrophysiological processes involved in translation. However, most of these studies used a translation recognition paradigm, in which word pairs are presented in two languages and participants have to indicate whether the second word is the correct translation of the first word (e.g. de Groot, 1992; Altarriba and Mathis, 1997; Sundermann and Kroll, 2006; Guo et al., 2012; Ma et al., 2017). Only few EEG studies have been conducted using an overt translation production task (Janyan et al., 2009, Christoffels et al., 2013). Janyan et al. (2009) investigated the degree of the involvement of semantics in cognate versus non-cognate processing in oral translation of visually presented single words, but for L2-L1 translation only. Christoffels and colleagues (2013) investigated the temporal course of single word translation for both forward (L1-L2) and backward (L2-L1) translation. Their participants were unbalanced but proficient bilinguals, with Dutch as their L1 and English as L2 learned at ~10years of age. Behavioral results showed no reliable differences between translation directions in the reaction times. ERP results indicated that neural processing was different depending on translation direction (L1-L2 or L2-L1) at around 200 ms and 400 ms following word presentation with larger P2 amplitudes for L1-L2 translation and larger N400 amplitudes for L2-L1 translation. The authors proposed two explanations for this result. As a first option, they suggested that the P2 may be more sensitive to processing the source language whereas the N400 could be associated to lexico-semantic processing related to selecting the target. The authors further postulated that the higher ERP amplitudes, which are often interpreted as reflecting more extensive processing, were associated to higher difficulty to access the L1 input (P2) and produce L1 in translation (N400). They related this last point to the “inversed” language effect also called “paradoxical cost asymmetry” (Meuter and Allport, 1999) found in several language-switching studies, which stands for faster responses in L2 than L1 and which has been interpreted in terms of globally reduced access or inhibition of L1 (Costa and Santesteban, 2004, Christoffels et al., 2007, Gollan and Ferreira, 2009). In the alternative explanation, the authors suggest that the larger P2 amplitude for L1-L2 translation could index the lexical retrieval of the L2 word while the larger N400 for the L2-L1 translation could reflect more effortful lexical and semantic processing of the L2 word. In order to differentiate better between translation processing and reading processes, Christoffels et al. (2013) also conducted an experiment for which the material and analyses were based on the translation task, but where participants had to read the words aloud instead of translating them while ERPs were recorded. In contrast to the translation task, no effects of language or interlingual homographs were found. The authors concluded that translation seems to involve conceptual processing in both translation directions and that the task goal influences how words are processed. To our knowledge, the study by Christoffels et al. (2013) is the only one investigating the time course of translation using ERPs and looking at effects of translation direction using an overt production paradigm.

The few studies targeting spatio-temporal mechanisms underlying word translation production suggest that the exact nature of the networks may crucially depend on language proficiency. The bigger the asymmetry across L1 and L2, the more brain regions linked to cognitive control may be engaged during word production. In contrast, very little is known about mere spatio-temporal mechanisms underlying translation effects.

The current study takes these previous findings a step further by adding a within-language word generation task, thus providing the possibility to disentangle processes specific for translation from other within-language lexical-semantic processing. Importantly, we chose a word generation task as a control task and not a reading task because in word production, the selection of a lemma is a conceptually driven process, while in reading it is part of the perception process (Indefrey and Levelt, 2004). Moreover, the type of ERP analysis used in the present study separates topographic and electrical field strength measures. Differentiating between GFP and topographic effects is meaningful, as different topographies directly indicate different source configurations, whereas different GFP in the absence of a topographic effect indicates different activation strength of the same sources (Michel et al., 2004). This allows a more physiologically oriented interpretation (Murray et al., 2008) of the underlying effects of task and language.

Here, we recorded high-density EEG data during a translation task (L1-L2 and L2-L1) in unbalanced bilinguals with an intermediate proficiency in L2. As a control task, we used a within-language word generation task where participants had to say a word aloud that is semantically related to the presented word. This control task was conducted in both L1 and L2 aiming at disentangling the processes that are specific for translation from within-language processing.

The aim of this study is two-fold:

• Aim 1: We investigated which spatial-temporal brain dynamics are different in translation processing compared to within-language processing by comparing behavioral and electrophysiological responses from a translation task to the responses from a within-language word-generation task. On the behavioral level, we expected faster responses for the word-generation task (control task) as compared to the translation task, as no switching from the input to the target language is necessary in the control task. On the electrophysiological level and based on picture naming studies, we hypothesized that the translation task leads to larger amplitudes at around 200–300 ms as well as around 400–600 ms, as these time windows have previously been associated to lexical retrieval and form encoding, respectively (Indefrey and Levelt, 2004). However, considering that response times in translation tasks are slower than in picture naming tasks (e.g. Kroll and Stewart, 1994; Cheung and Chen, 1998; Francis et al., 2003) lexical retrieval and form encoding might be delayed and occur at later stages. In addition, we expected that lexical retrieval is more difficult in the translation task compared to the control task and that it necessitates a higher load of attentional control mechanisms to switch from the input language to the production language.

• Aim 2: Based on studies showing differential effects depending on direction of translation, we investigated the influence of translation direction by comparing the responses of forward translation to responses from backward translation. On the behavioral level we expected that translation would be faster and more accurate for L2-L1 translation than for L1-L2 translation, as the participants in this study had an intermediate level of L2-production skills. At the electrophysiological level we expected similar results as found by Christoffels et al. (2007), namely differential neural processing around 200 ms and 400 ms following word presentation.