Figures
Abstract
In this study, we applied the text-change paradigm to investigate whether and how discourse hierarchy affected the memory representation of a discourse. Three kinds of three-sentence discourses were constructed. In the hierarchy-high condition and the hierarchy-low condition, the three sentences of the discourses were hierarchically organized and the last sentence of each discourse was located at the high level and the low level of the discourse hierarchy, respectively. In the linear condition, the three sentences of the discourses were linearly organized. Critical words were always located at the last sentence of the discourses. These discourses were successively presented twice and the critical words were changed to semantically related words in the second presentation. The results showed that during the early processing stage, the critical words were read for longer times when they were changed in the hierarchy-high and the linear conditions, but not in the hierarchy-low condition. During the late processing stage, the changed-critical words were again found to induce longer reading times only when they were in the hierarchy-high condition. These results suggest that words in a discourse have better memory representation when they are located at the higher rather than at the lower level of the discourse hierarchy. Global discourse hierarchy is established as an important factor in constructing the mental representation of a discourse.
Citation: Wu Y, Yang X, Yang Y (2016) Eye Movement Evidence for Hierarchy Effects on Memory Representation of Discourses. PLoS ONE 11(1): e0147313. https://doi.org/10.1371/journal.pone.0147313
Editor: Kevin Paterson, University of Leicester, UNITED KINGDOM
Received: September 7, 2015; Accepted: December 31, 2015; Published: January 20, 2016
Copyright: © 2016 Wu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All EDD files of the current study are available from the figshare database (http://dx.doi.org/10.6084/m9.figshare.1597587).
Funding: This work was supported by the National Natural Science Foundation of China (http://www.nsfc.gov.cn/) to YY (61433018) and XY (31400968). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Human language is represented in the brain in hierarchical ways [1], and the processing of hierarchical structures is a core feature that differentiates human language from other animal communication systems e.g., [2, 3]. Previous studies have found that processing different levels of hierarchy may involve different cognitive and neural mechanisms during sentence comprehension e.g., [4, 5, 6]. It is therefore of great interest to investigate whether and how hierarchy affects cognitive processing, such as the construction of mental representations during discourse comprehension.
Studies at the sentence level have repeatedly shown that the hierarchical organization of a sentence plays an important role in sentence comprehension. First, hierarchy affects online syntactic analysis. Compared with subordinate clauses, syntactic error in main clauses induces smaller P600 effect with earlier latency, indicating that syntactic analysis in main clauses was less difficult than that in subordinate clauses [7]. Second, hierarchy interacts with inter-clause semantic relation and clause order, affecting the availability of information. In particular, information in main clauses is more accessible than that in subordinate clauses, especially when the clauses are linked by temporal conjunction, e.g., after [8–10]. Third, hierarchy affects the processing depth of information. For example, Baker and Wagner [6] found that false information was more likely to be reported when it was conveyed by main clauses than by subordinate clauses. This result was replicated by another experiment with text-change paradigm [5]. In addition to the studies of complex sentences, Jiang and Zhou [4] investigated neural dynamics in processing different levels of syntactic hierarchy during simple sentence processing. The results showed that violating the low-level constraints elicited a left-lateralized, anteriorly maximized negativity, whereas violating the high-level constraints elicited a right anterior negativity (RAN) and a right centro-parietal negativity (N400), suggesting that different levels of syntactic hierarchy may involve different neural mechanisms.
Equally important, but has received far less attention, is the effect of hierarchy on a larger language unit, i.e., discourse. According to Rhetorical Structure Theory (RST), coherence of a discourse is supported by a hierarchical, connected structure of the discourse [11, 12]. Two sentences in a discourse that share the same intended function are joined to create a span and to convey the purpose of the writer. This span may, in turn, be related to another sentence or span to create a new effect on the reader. This process continues until the end of a discourse, resulting in a hierarchical structure of the discourse. Discourse hierarchy can affect the depth of semantic integration during comprehension. Discourse-incongruous words were read longer than discourse-congruous words only when the critical sentence and the preceding sentence were located at the same discourse level, but not when they were located at different discourse levels [13]. In addition, studies of prosody have shown that people tend to insert longer preceding pauses for clauses located at the high level than for those located at the low level [14, 15], which indicates that the representation of discourse hierarchy is kept in mind during the planning of speech production. Despite these studies, whether and how hierarchy affects cognitive mechanisms during discourse processing, such as the construction of mental representations, remains unresolved.
To address this issue, we constructed three-sentence discourses with various structures according to RST. In the hierarchy-high and hierarchy-low conditions, the three sentences of the discourses were hierarchically organized and the last sentence of each discourse was located at the high level and the low level of the discourse hierarchy respectively. In the linear condition, the three sentences of the discourses were organized linearly. The stimuli in Table 1 are examples.
In the hierarchy-low condition, the first sentence briefly describes the goal of Yanli, and both the second and the third sentences elaborate what Yanli did to achieve this goal. The latter two sentences are closely related to each other and share the same intention. In terms of their effects on readers, they should be joined into a span before they are connected to the first sentence. According to RST, these two elaboration sentences occupy the lower levels of the discourse hierarchy, and the first goal sentence occupies the higher level of the discourse hierarchy, as shown in Fig 1. In the hierarchy-high condition, the previous two sentences specifically depict the actions of Yanli, whereas the last sentence describes the purpose of Yanli’s actions. Therefore, the previous two sentences are sub-units for the same discourse intention, and they create a span first before they are related to the last sentence. In this case, the final sentence is located at the higher level of the discourse hierarchy (central in Fig 1). In the linear condition, all the three sentences describe the actions of Yanli, such that all of them share the common discourse intention and are located at the same level of the discourse hierarchy (right in Fig 1). The critical word grasshoppers is always located in the last sentence of the discourses. These discourses are successively presented twice with an interposing blank screen, and the critical word is changed to the semantically related word centipedes on the second presentation.
Previous studies have shown that higher-level information receives deeper processing during sentence comprehension [5, 6] and that humans are sensitive to hierarchical structure during discourse processing [13]. Therefore, we hypothesize that hierarchy will modulate the memory representation of the discourses, such that the change of the critical words in the hierarchy-high condition will be detected most successfully, whereas the change of those critical words in the hierarchy-low condition will be detected least successfully.
Method
Ethics Statement
The experiment was approved by the Institutional Review Board of the Institute of Psychology, Chinese Academy of Sciences. The participants signed informed consent forms before the experiment and had the right to quit the experiment at any time.
Participants
Thirty undergraduate students (16 women; mean age = 21.0 years, SD = 1.9) participated in the experiment. All were native Chinese speakers with normal or corrected-to-normal vision. All were paid for participation.
Apparatus
Eye movements were monitored using EyeLink 1000, which had a sampling rate of 1000 Hz. The participants viewed the stimuli with both eyes; however, only the right eye was recorded. Stimuli were presented in 28 Chinese Songti font on a 21-inch CRT monitor with a refresh rate of 150 Hz and a resolution of 1024×768 pixels. Participants read the discourses approximately 58 cm from the monitor.
Materials
Thirty-six sets of three-sentence discourses were constructed (see Table 1 for an example). Sentences in a discourse were connected by commas to ensure that the effects of syntactic markings would not confound the effects of discourse structure [13]. The critical words (all two-character words) appeared in the middle of the final sentences, followed by two two-character words. The words that immediately preceded and followed the critical words were identical across all the three conditions. The critical words in the two displays were matched on stroke number [first display, M = 17.72, SD = 4.07; second display, M = 17.94, SD = 3.75] and log frequency [first display, M = 3.24, SD = .83; second display, M = 3.31, SD = .86]; all ts<−.72. The semantic relatedness of the critical words between the two displays was 3.26 (.22) on a scale ranging from 1 (no relevance) to 5 (very strong relevance), indicating moderate relatedness of critical words between the two displays.
Three offline pretests of the experimental materials were conducted to assess the structures, the predictability of the critical words, and the coherence of the discourses. In the structure pretest, 24 participants were asked to choose one structure for each discourse from the three structures as shown in Fig 1. The results showed that the approval rates for the discourse structure were high, and there was no difference among the three structures, F<1 (as shown in Table 2). In the cloze probability pretest, another 24 participants were asked to complete the discourses that were truncated before the critical words. The results revealed that the probabilities for predicting the critical words on both displays were low, and there was no difference among the three conditions, FS<1. In the coherence pretest, another 24 participants were asked to rate the coherence of the discourses on a scale of 1 (not coherent at all) to 5 (very coherent). The results showed that there was no difference across the conditions and displays, FS<1.
The experimental discourses were divided into three versions via Latin square design. Each version contained 36 discourses (12 per condition). Besides, 108 filler discourses were added to the experimental materials. A total of 36 fillers matched the experimental items in discourse structures, with 12 discourses per structure. These fillers remained unchanged on the second presentation. The 72 other fillers had different discourse structures from the experimental materials. For half of these fillers, words changed at a variety of locations. For the other half of these fillers, words remained identical. In total, the participants read 144 discourses during the experimental phase, half of which included changes. There were 8 practice discourses before each experimental session.
Procedure
Participants were instructed that each discourse would be presented twice, and their task was to read for comprehension. In addition, they were asked to read at their normal pace and read through each discourse only once if they already understood the discourse [16, 17].
A calibration was conducted at the beginning of the experiment. Each trial started with a drift calibration, after which a small square appeared at the position where the first character of upcoming discourses would be located. The participants were asked to fixate on the square until it disappeared and was replaced by a discourse. The discourses were presented as a whole, with each sentence in one separate line. The participants were asked to press a handheld button after they finished reading the discourses. Once they pressed the button, the discourses disappeared and were replaced by a blank screen for 500 ms. The second display of the discourses then appeared on the screen. For 1/3 of the trials, the participants were asked to answer a comprehension question by pressing marked buttons. The entire experiment lasted for approximately 60 minutes.
Data analysis
Data obtained from one female subject were eliminated from the data analysis because the trials in which reading paths were chaotic or in which she blinked were over one-third of her total trials. The average accuracy rate of the 29 other subjects was 90.03%. Further 6.9% of the remaining trials were excluded because of the fixation disruptions or blinks. Fixations less than 80 ms were merged into the nearest fixation point if they were less than one character at a distance. Other fixations less than 80 ms or more than 800 ms were omitted. This procedure excluded a total of 1.22% fixation points. Finally, reading times that were three standard deviations above or below the mean for the critical words of a given subject in a given condition were deleted, which removed 0.07% of the total data.
Four eye movement measures were examined: first fixation duration (the duration of the first first-pass fixation on a region), gaze duration (the time a reader spends on a region before the reader leaves it), go-pass time (the sum of the time spent on a region and to the left of the region before moving to the right of it), and total time (the sum of all fixation durations on a region). The first two measures were associated with early processing, and the last two measures were associated with later processing [18, 19]. We did not count the instances of skipping for the first fixation duration, gaze duration, go-pass time, and total time; thus, 10.34%, 10.60%, 10.70%, and 4.58%, respectively, of the total data were removed from final analysis.
For each eye movement measure, a 3 (hierarchy: hierarchy-high, hierarchy-low, and linear) × 2 (text change: first-display, second-display) repeated ANOVA was conducted, with participants (F1) and items (F2) as random factors.
Results
Table 3 shows the means and standard deviations for the critical words.
For the first fixation duration, the main effect of text change was found. The changed-critical words in the second display were read longer than the original-critical words in the first display, F1(1, 28) = 11.03, MSE = 981, p < .01, = .28; F2(1, 35) = 19.90, MSE = 711, p < .001,
= .36. Neither a main effect of hierarchy nor an interaction was found, FS<2.15.
No main effect of hierarchy was found on gaze duration, FS<1. However, a main effect of text change was found; the changed-critical words were read longer than the original-critical words, F1(1, 28) = 7.65, MSE = 7013, p < .05, = .22; F2(1, 35) = 7.77, MSE = 6976, p < .01,
= .18. More importantly, the text change effect was modulated by hierarchy, F1(2, 56) = 5.22, MSE = 3590, p < .01,
= .16; F2(2, 70) = 4.94, MSE = 5530, p < .05,
= .12. In the hierarchy-low condition, no significant difference was found between the two displays, FS<1. However, in the hierarchy-high condition, the gaze durations were longer for the changed-critical words than for the original-critical words, F1(1, 28) = 12.58, MSE = 4843, p < .01,
= .31; F2(1, 35) = 13.58, MSE = 4523, p < .01,
= .28. This was also the case for the linear condition, F1(1, 28) = 5.79, MSE = 5164, p < .05,
= .17; F2(1, 35) = 5.51, MSE = 8063, p < .05,
= .14.
For go-pass time, no significant differences were found among the three hierarchical levels or between the two displays, FS<1.20. However, an interaction effect was found between them, F1(2, 56) = 5.42, MSE = 5339, p < .01, = .16; F2(2, 70) = 4.28, MSE = 8884, p < .05,
= .11. Simple effect analysis revealed that in the hierarchy-high condition, the changed-critical words were read longer than the original-critical words, F1(1, 28) = 4.99, MSE = 8853, p < .05,
= .15; F2(1, 35) = 8.54, MSE = 5355, p < .01,
= .20, whereas in the hierarchy-low condition or in the linear condition, the times spent on the two displays had no significant differences, FS<2.56.
The results for total time showed a similar pattern as the go-pass time. Neither the main effect of hierarchy nor the main effect of text change was significant, Fs<2.05. However, the interaction was remarkable, F1(2, 56) = 6.47, MSE = 4928, p < .01, = .19; F2(2, 70) = 4.95, MSE = 6878, p < .05,
= .12. In the hierarchy-high condition, the total times were longer in the second display than in the first display, F1(1, 28) = 6.47, MSE = 8610, p < .05,
= .19; F2(1, 35) = 8.21, MSE = 5429, p < .01,
= .19. However, no significant differences were found between the two displays in the hierarchy-low condition or in the linear condition, FS<2.19.
Discussion
Our findings showed that during the early processing stage, the critical words were read longer when they were changed in the hierarchy-high and linear conditions, but not in the hierarchy-low condition. Moreover, during the late processing stage, the changed-critical words induced longer reading times, but only in the hierarchy-high condition.
The changed-critical words induced longer first fixation duration for each condition, indicating the same rapid and automatic processing of lexical access for the changed-critical words across all the three conditions [20, 21]. More importantly, longer gaze duration for the changed-critical words was observed in the hierarchy-high and linear condition, but not in the hierarchy-low condition. This result suggests that information at the higher level of the discourse hierarchy is represented in finer granularity and that readers are more sensitive to the change at the higher level than that at the lower level of the discourse hierarchy. Note that of the hierarchy-high and the linear condition, the changes in the hierarchy-high condition are more salient than those in the linear condition for readers. As evidenced by the fact that for the hierarchy-high condition, longer reading times were observed both on the early processing measure (gaze duration) and the late processing measures (go-pass time and total time), whereas for the linear condition, longer reading times were found only on the early processing measure. These results suggest that the memory representation of the original-critical words vary as a function of discourse hierarchy.
The finding that information at the higher level of discourse hierarchy is represented in finer granularity extends previous studies at phrase and sentence levels, which suggested that information at the higher level had a better memory representation than information at the lower level [5, 22]. In addition, our results compare favorably with the previous finding that central sentences of discourse had a better memory representation than sentences that were loosely related with text theme e.g., [23].
The moderate effect of discourse hierarchy on memory representation of a discourse may be related to the uneven prominence of different hierarchical levels. According to RST [11, 12], information at the higher level played a more prominent role in expressing a writer’s intention, which was supported by acoustic studies showing that speakers inserted a longer pause at boundaries of higher hierarchy [14, 15]. In studies using the change-detection paradigm, more successful detection of the changes was taken as an indication that more attention resources was allocated to the elements being changed e.g., [24, 25, 26]. Therefore, our finding that the changed-critical words in the hierarchy-high condition were more successfully detected could be due to the fact that information at the higher level of the discourse hierarchy received more attention and was processed to a deeper extent.
Another possibility may be related to the activation of propositions. In the framework of construction-integration theory for discourse comprehension, highly activated propositions would predicate a better memory representation [27]. Possibly, the information located at the higher level was more highly activated than that at the lower level. Therefore, the critical words in the hierarchy-high condition had a better memory representation than those in the linear or hierarchy-low condition.
Givón [28] argued that linguistic cues could function as mental processing instructions, instructing readers in the importance of concepts and how to allocate attention during discourse comprehension. This has been evidenced by numerous studies that suggested that linguistic cues as lexical devices and syntactic devices influenced the processing depth and memory representation of discourses e.g., [29, 30–32]. However, all of these studies focused on the linguistic cues related to lexical and syntactic devices at the sentence level. Thus, our findings contribute to the literature by showing that linguistic cues as signaled by hierarchical organization at the discourse level also play an important role in discourse comprehension.
In summary, the current study provides evidence that information has a better memory representation when it is located at the higher level than at the lower level of the discourse hierarchy, suggesting that information in a discourse is represented in various granularities as a function of hierarchical levels. Our results extend previous findings on hierarchy processing at the phrase and sentence levels and establish discourse hierarchy as an important linguistic device that modulates the memory representation of a discourse.
Acknowledgments
We thank XiuPing Zhang and Yang Cao who helped in preparing the experimental materials.
Author Contributions
Conceived and designed the experiments: YW XY YY. Performed the experiments: YW. Analyzed the data: YW. Contributed reagents/materials/analysis tools: YW XY YY. Wrote the paper: YW XY YY.
References
- 1.
Chomsky N. Syntactic structures. The Hague: Mouten 1957.
- 2. Hauser MD, Chomsky N, Fitch WT. The faculty of language: what is it, who has it, and how did it evolve? Science. 2002;298(5598):1569–79. pmid:12446899
- 3. Fitch WT, Hauser MD. Computational constraints on syntactic processing in a nonhuman primate. Science. 2004;303(5656):377–80. pmid:14726592
- 4. Jiang X, Zhou X. Processing different levels of syntactic hierarchy: An ERP study on Chinese. Neuropsychologia. 2009;47(5):1282–93. pmid:19428392
- 5. Sanford AJ. Context, attention and depth of processing during interpretation. Mind & language. 2002;17(1–2):188–206.
- 6. Baker L, Wagner JL. Evaluating information for truthfulness: The effects of logical subordination. Memory & Cognition. 1987;15(3):247–55.
- 7. Münte TF, Szentkuti A, Wieringa BM, Matzke M, Johannes S. Human brain potentials to reading syntactic errors in sentences of different complexity. Neuroscience Letters. 1997;235(3):105–8. pmid:9406880
- 8. Townsend DJ, Bever TG. Interclause relations and clausal processing. Journal of Verbal Learning and Verbal Behavior. 1978;17(5):509–21.
- 9. Morrow DG. Prominent characters and events organize narrative understanding. Journal of Memory and Language. 1985;24(3):304–19.
- 10.
Cooreman A, Sanford AJ. Focus and syntactic subordination in discourse: Citeseer; 1996.
- 11. Mann WC, Thompson SA. Rhetorical structure theory: Toward a functional theory of text organization. Text. 1988;8(3):243–81.
- 12. Taboada M, Mann WC. Rhetorical Structure Theory: looking back and moving ahead. Discourse Studies. 2006;8(3):423–59.
- 13. Yang X, Chen L, Yang Y. The effect of discourse structure on depth of semantic integration in reading. Memory & cognition. 2014;42(2):325–39.
- 14. Ouden Hd, Noordman L, Terken J. Prosodic realizations of global and local structure and rhetorical relations in read aloud news reports. Speech Communication. 2009;51(2):116–29.
- 15. Yang X, Yang Y. Prosodic realization of rhetorical structure in Chinese discourse. Audio, Speech, and Language Processing, IEEE Transactions on. 2012;20(4):1196–206.
- 16. Raney GE, Rayner K. Word frequency effects and eye movements during two readings of a text. Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale. 1995;49(2):151–72. pmid:9183975
- 17. Ward P, Sturt P. Linguistic focus and memory: An eye movement study. Memory & cognition. 2007;35(1):73–86.
- 18. Johnson RL. The Quiet Clam Is Quite Calm: Transposed-Letter Neighborhood Effects on Eye Movements During Reading. J Exp Psychol Learn. 2009;35(4):943–69.
- 19. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological bulletin. 1998;124(3):372–422. pmid:9849112
- 20. Inhoff AW. Two stages of word processing during eye fixations in the reading of prose. Journal of verbal learning and verbal behavior. 1984;23(5):612–24.
- 21. Lima SD, Inhoff AW. Lexical access during eye fixations in reading: Effects of word-initial letter sequence. Journal of Experimental Psychology: Human Perception and Performance. 1985;11(3):272. pmid:3159838
- 22. Kintsch W, Keenan J. Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology. 1973;5(3):257–74.
- 23. de Villiers PA. Imagery and theme in recall of connected discourse. Journal of Experimental Psychology. 1974;103(2):263–8.
- 24. Rensink RA, O'Regan JK, Clark JJ. To see or not to see: The need for attention to perceive changes in scenes. Psychological science. 1997;8(5):368–73.
- 25. Simons DJ, Rensink RA. Change blindness: Past, present, and future. Trends in cognitive sciences. 2005;9(1):16–20. pmid:15639436
- 26. Rensink RA. The dynamic representation of scenes. Visual cognition. 2000;7(1–3):17–42.
- 27. Kintsch W. How readers construct situation models for stories: The role of syntactic cues and causal inferences. From learning processes to cognitive processes: Essays in honor of William K Estes. 1992;2:261–78.
- 28. Givón T. The grammar of referential coherence as mental processing instructions. Linguistics. 1992;30(1):5–55.
- 29. Sturt P, Sanford AJ, Stewart A, Dawydiak E. Linguistic focus and good-enough representations: An application of the change-detection paradigm. Psychonomic bulletin & review. 2004;11(5):882–8.
- 30. Klin CM, Weingartner KM, Guzman AE, Levine WH. Readers' sensitivity to linguistic cues in narratives: How salience influences anaphor resolution. Memory & Cognition. 2004;32(3):511–22.
- 31. Birch S, Rayner K. Effects of syntactic prominence on eye movements during reading. Memory & cognition. 2010;38(6):740–52.
- 32. Magliano JP, Schleich MC. Verb aspect and situation models. Discourse Process. 2000;29(2):83–112.