The process of forming object representations in visual short-term memory (VSTM) from different visual features of a stimulus, such as color, shape, size, orientation, location, movement, and so forth, is referred to as feature binding. There are ongoing debates regarding the nature of the representations that are formed— specifically, whether they comprise integrated objects or individual features that are linked on a temporary basis (e.g., Luck & Vogel, 1997; Treisman, 2006; Vogel, Woodman, & Luck, 2006; Wheeler & Treisman, 2002; Xu, 2002)–and the extent to which visual attention might be used in forming and/or maintaining those representations in VSTM (Allen, Baddeley, & Hitch, 2006; Allen, Hitch, & Baddeley, 2009; Brown & Brockmole, 2010; Fougnie & Marois, 2009; Gajewski & Brockmole, 2006; Johnson, Hollingworth, & Luck, 2008; Logie, Brockmole, & Vandenbroucke, 2009). There has been less discussion in this area concerning the possible differential role of various categories of stimulus features in the emergence of bound objects in VSTM. Although this last topic has been addressed extensively within the literature on visual attention and visual perception (e.g., Treisman & Gelade, 1980), it is unclear whether the principles that govern visual attention and perception also apply in the operation of VSTM or visual working memoryFootnote 1 (e.g., Logie, 1995, 2003; Logie & van der Meulen, 2009; Zhaoping, 2008) in the few seconds after stimulus offset. This was the primary focus of the experimental work reported here.

Within the domain of visual perception, feature integration theory (FIT) holds that, in contrast to other stimulus properties such as color and shape, location plays a key role in binding by providing the spatial map to which individual features are then attached and are thus combined to form objects (Treisman, 2006; Treisman & Gelade, 1980). According to FIT, although individual visual features are detected relatively automatically, participants cannot know which of these features go together unless attention is focused on particular locations. Treisman and Gelade noted that directing attention to a point in space precedes the identification of information at that location. This leads to the conclusion that focusing attention on a particular spatial location then allows the features at that location to be bound together so that an item can be identified.

In applying FIT to VSTM, Treisman (2006) argued that location has an important role to play in providing the reference frame for bindings that are formed between features. She argues further that both individual features and bound objects are held in a visual temporary memory system but that attention is required to maintain the bindings between features. Wheeler and Treisman (2002) viewed location as one of the many possible descriptive properties of an object. Their experiments showed that participants were better at remembering locations than colors, in that memory for location remained at ceiling when the number of stimuli increased from three to six, whereas memory for color decreased. Further evidence that location might be “special” comes from studies showing that cognitive aging is linked with impairments of binding when location is one of the features to be bound (e.g., location–shape) but that binding that does not involve location (e.g., shape–color) appears to be insensitive to cognitive aging (Brockmole, Parra, Della Sala, & Logie, 2008; Brown & Brockmole, 2010; Olson et al., 2004; Parra, Abrahams, Logie, & Della Sala, 2009).

In one of the few studies to directly address the “special role” of location in immediate memory for feature binding, Treisman and Zhang (2006) used change detection to examine memory for color–shape bindings when the locations of the items differed in the presentation and test displays. Changing location disrupted memory for bindings with a 100-ms delay between study and test. This suggested that feature bindings were largely automatic, with obligatory inclusion of locations that were presented but designated as irrelevant. The disruptive effect was much smaller when the study–test interval was 900 ms, and there was no disruptive effect of changing locations with study–test intervals of 3 or 6 s. This pattern of results suggests that location is crucial for initial detection and encoding of feature bindings but that bound surface features might be stored independently of location after those representations are transferred to VSTM. However, Treisman and Zhang (2006) considered only the impact of changing locations on VSTM. Therefore, it is not clear whether location is special in comparison with other features or whether stimulus features such as color and shape can likewise be excluded from VSTM representations when they become task irrelevant.

Our major aim in the experiments reported here was to explore whether or not location has a special role in short-term memory for bindings (rather than perceptual binding), as compared with the role of other features—specifically, shape and color. Color and shape are typically stable properties of an object (especially shape) that allow the object to be identified regardless of its location. In contrast, location is a transient property of an object, defining its momentary position in space. Therefore, it is possible that location, but not color and shape, can be easily removed when a representation of the object is formed in VSTM. On the other hand, it is possible that unlike perceptual processes, the VSTM system is relatively immune to the differences between various features and operates according to different principles than those operating in visual attention and visual perception. Indeed, such distinctions have been made in a variety of other contexts (e.g., Irwin, 1991; Phillips, 1974). VSTM might be sufficiently flexible to allow the formation of bindings between task-relevant features in the face of major changes in task-irrelevant object properties, regardless of what those features may be.

The experiments reported here assessed the extent to which consistency in shape, color, and location is required for forming and retaining bound representations in VSTM over different study–test intervals. Each experiment varied the task relevance of these three features. In Experiment 1, location was task irrelevant, and memory for shape–color binding was tested in the face of changing location information. This experiment broadly followed the procedures used by Treisman and Zhang (2006, Experiment 5) but used shorter and more fine-grained variations in the study–test intervals. We expected to replicate the Treisman and Zhang finding that randomizing location is disruptive at short study–test intervals, but not at longer delays, as compared with a condition in which locations remain unchanged between study and test. By using finer grained variations in study–test intervals, we explored whether there is a gradual or a sudden loss of the disruptive effect, to identify the point at which the disruptive effect disappears and to consider the characteristics of the memory system that might retain the bound task-relevant features. In Experiment 2, shape was task irrelevant, and memory was tested for color–location binding in the face of changing shape information. In Experiment 3, color was task irrelevant and was randomized between study and test, with memory tested for shape–location binding. If location has a preeminent role in the binding of other features, as was suggested by Treisman and Sato (1990), then, in Experiment 1, changing locations randomly across short study–test intervals should prevent, or at least disrupt, retention of shape–color bindings. In addition, by this argument, changing shape or changing color should have little or no effect on retention of color–location or shape–location binding, respectively, in Experiments 2 and 3. However, if any feature can be excluded from bound object representations in VSTM, then at longer delays, no differential impact on memory should be observed whether location, color, or shape information is designated as task irrelevant and is randomized between study and test.

Experiment 1

Location is such an overwhelming cue for encoding and retrieving stimuli and/or their features that it is invariably used if present (Hollingworth, 2007; Jiang, Olson, & Chun, 2000; Mitroff & Alvarez, 2007), but it can be made irrelevant to the task through randomization. Experiment 1 used a change detection task in which observers judged whether a study and a subsequent test display contained the same six colored shapes, irrespective of their locations. To assess memory for bindings, the color–shape pairings of two objects in the display were swapped between study and test on half of the trials. On the other half of the trials, the color–shape pairings were identical between study and test for all six objects. Critically, for half of the trials, the locations of items in the test display were randomized. as compared with the study display, to make location noninformative and irrelevant as a cue for task performance. On the other half of the trials, the locations of the objects were unchanged between study and test. Given that iconic memory is spatiotopic (e.g., Irwin, 1991; Phillips, 1974), we would expect randomization of location to be disruptive at very short study–test intervals, because of a mismatch between the test array and the content of the icon. The question of theoretical importance here is whether this disruption is maintained when study–test intervals are sufficiently long to allow the sensory trace in the icon to decay and for consolidation of the to-be-remembered object (i.e., color–shape binding) into VSTM. If the disruption continues with long study–test intervals, this would suggest that features, even if task irrelevant, remain bound in the representations. If the disruption is found to be absent at longer intervals, this would suggest that, over time, task-irrelevant features are no longer included in the representation, leaving only the binding between relevant features. Given wide variability in estimates of the speed of consolidation into VSTM (Brockmole, Wang, & Irwin, 2002; Irwin, 1991; Jiang et al., 2000; Phillips, 1974; Vogel et al., 2006) and Treisman and Zhang’s (2006, Experiment 5) finding that consolidation in VSTM of shape–color binding might occur between 900 and 6,000 ms, we varied the study–test intervals between 0 and 2,500 ms, with increments of 500 ms.

Finally, we explored the extent to which participants could ignore an irrelevant feature when forming bindings of relevant features from the same display. Treisman and Zhang (2006) used a mixed design in which participants were unaware whether each trial would involve a change of the irrelevant feature. In order to maximize the opportunity for participants to ignore the irrelevant feature, we therefore used a blocked design, so that participants knew in advance whether or not the irrelevant feature would change or remain the same between study and test displays. This allowed for a direct test of whether encoding of an irrelevant feature is obligatory or whether it can be ignored when participants know that it is irrelevant and not helpful for task performance.

Method

Participants

Twelve students (3 men and 9 women) in the age range of 18–25 years participated in the experiment and received an honorarium of £10. All participants provided informed consent and reported normal color vision and normal or corrected-to-normal visual acuity. They were naïve as to the experimental hypotheses.

Apparatus and stimuli

On each trial, stimuli consisted of displays of six objects randomly placed within an imaginary 3 × 4 square grid subtending 6.1º × 7.8º horizontally and vertically. The items were created by randomly combining six shapes (circle, plus, right triangle, horseshoe, diamond, parallelogram) and six colors (yellow, cyan, magenta, blue, red, green) without replacement. These items were displayed on a gray background. Each item subtended 1.6º × 1.7º of visual angle. Observers viewed these stimuli from an unconstrained distance of approximately 1 m. All the stimuli were displayed on a gray background on a 43-cm (41 cm viewable) CRT screen. Participants could move their eyes freely during the task. An example stimulus display is shown in top left panel of Fig. 1.

Fig. 1
figure 1

Sequence of events in Experiments 1, 2, and 3 (left panels) and results (right panels) shown as mean d′ scores for change detection of bindings. Stimuli are not drawn to scale

Full size image

Design and procedure

The experiment was a two (unchanged/randomized locations) × six (study–test intervals) repeated measures factorial design. In the unchanged-location condition, stimuli in the study display and test display for a given trial were presented in the same locations. On 50% of the trials, the color–shape combinations were unchanged between study and test, whereas on the remaining 50% of the trials, two of the stimuli swapped features between study and test displays. Participants judged the two displays to be same or different by pressing one of two keys on a response box. For half of the different trials, two shapes swapped locations while colors remained in the same location. For the other half of the different trials, two colors swapped locations, while shapes remained in the same locations.

In the randomized-location condition, the stimuli in the test display were presented in six randomly selected positions without reference to the study display on that same trial. The task was to detect whether or not there was a change between study and test arrays in the binding of color and shape between any two items despite the fact that the locations of the items had changed across displays. The same sets of six colors and six shapes were used for all trials; therefore, successful task performance could not be based on memory for individual features and was dependent only on the detection of correct combinations of features. Because the positions in the test display were selected at random, it was possible that one or more of the same locations were used between study and test. However, since target items could appear anywhere on the screen at test, the randomization ensured that location was never an informative cue.

Participants were tested on 2 consecutive days at the same time of the day, half being tested first with randomized locations, and the other half being tested first with unchanged locations. Each study display was presented for 200 ms, and the six study–test intervals were 0, 500, 1,000, 1,500, 2,000, and 2,500 ms. Three blocks of 20 trials each were created for each study–test interval in each location condition, yielding 36 blocks totaling 720 trials. Block order was counterbalanced within and across participants. A brief rest was given after every six blocks. In each of the two test sessions, participants initially practiced 8 trials of each of the study–test intervals, starting from the longest (2,500 ms) and working through all the study–test intervals to the shortest (0 ms), giving 48 practice trials in each session.

The test display was presented until the participants responded. Accuracy was the dependent variable of interest, and participants were asked to ensure accuracy, rather than speed of response. Articulatory suppression (saying the word “the” repeatedly at about two to four utterances per second) was used from fixation until after the response had been given, to prevent the participants from encoding or rehearsing the stimuli verbally.

Results

Mean change detection performance (as measured with d′) across study–test intervals and for the unchanged-/randomized-location conditions is shown in the top right panel of Figure (see the Supplemental materials for hits and false alarms for each condition). Analysis of the d′ values showed a significant main effect of location condition, F(1, 11) = 44.958, MSE = 0.606, p < .001, partial η2 = .803. Change detection for color–shape bindings was significantly reduced when the task-irrelevant feature of stimulus location was changed between study and test. The main effect of study–test intervals was also significant, F(5, 55) = 27.903, MSE = 0.151, p < .001, partial η2 = .717, indicating that change detection for color–shape bindings differed across the study–test intervals. Crucially, there was a significant interaction between these variables, F(5, 55) = 33.517, MSE = 0.228, p < .001, partial η2 = .753. Pairwise t tests with Bonferroni adjustment for six comparisons (p < .008) showed that the difference between the means for the unchanged and the randomized conditions was significant at 0 ms, t(11) = 12.784, p < .001, 500 ms, t(11) = 3.419, p < .003, and 1,000 ms, t(11) = 2.823, p < .008, but not at 1,500, 2,000, or 2,500 ms (p > .05 in all cases).

Separate single degree-of-freedom polynomial tests within each condition were conducted to investigate further the nature of the interaction. In the unchanged-location condition, performance across study–test intervals was characterized by negative-slope linear, F(1, 11) = 131.030, MSE = 0.288, p < .001, partial η2 = .923, quadratic, F(1, 11) = 53.584, MSE = 0.220, p < .001, partial η2 = .830, and cubic F(1, 11) = 53.422, MSE = 0.126, p < .001, partial η2 = .829, trends. In contrast, for the randomized locations, performance was characterized by a positive-slope quadratic trend, F(1, 11) = 7.114, MSE = 0.156, p < .022, partial η2 = .393. One slope being negative and the other being positive, performance converged between the two conditions at 1,500 ms. The significant higher order trends indicate a nonlinear relationship between study–test intervals and performance.

Discussion

The results obtained at short study–test intervals (e.g., less than or equal to 1,000 ms) offer support for the predictions of FIT (Treisman, 2006; Treisman & Gelade, 1980) and the broader literature on iconic or visual sensory memory (e.g., Irwin, 1991; Phillips, 1974) in suggesting that location has an important role in the initial processing and retention of visual displays. Randomizing locations between study and test, with the test occurring immediately or up to 1,000 ms after stimulus offset, was highly disruptive of performance, as compared with the unchanged-location condition. This result occurred even though location was irrelevant for the task of detecting changes in shape–color binding. Retaining the same locations between study and test resulted in ceiling performance with an immediate test but progressively poorer performance with the longer study–test intervals, reaching asymptote at around 1,500 ms. These results suggest that location was stored in the initial temporary representation following stimulus offset. Therefore, in the unchanged condition, location appears to be an effective cue at these short study–test intervals, and the location information remaining in the mental representation within those intervals might have aided search in the test display for changes in color–shape combinations. However, in the randomized condition, location would have been an irrelevant cue, resulting in a mismatch between a stored representation that incorporated the locations at presentation and the changed locations used in the test array. Given that the representation might have driven the search in the test array, this mismatch would disrupt the detection of changes in color–shape bindings. In striking contrast, results obtained with longer study–test intervals (1,500 ms or more) show that randomizing locations at test resulted in little or no disruption of change detection in color–shape bindings, as compared with the unchanged-location condition. The last result is consistent with previous experimental findings with mixed trial types (Treisman & Zhang, 2006) for longer study–test intervals than those used here. It is also consistent with the suggestion that the irrelevant feature of location is not included in the mental representation at these longer intervals: Leaving location unchanged does not aid performance, and changing location does not impair performance.

The results indicate that location is initially important for forming and retaining temporary representations of stimulus arrays, but this is true only for study–test intervals shorter than 1,500 ms. At or beyond 1,500 ms, there is no evidence here to suggest that location has an obligatory role in the maintenance of temporary representations of color–shape bindings. Nor does location provide an effective cue at these delays, even when locations are identical between study and test. Furthermore, extending the study–test interval beyond 1,500 ms does not appear to result in any reliable change in performance, suggesting that the bindings available at 1,500 ms can be maintained at the same level for at least 2.5 s following presentation, regardless of whether or not location changes.

One striking feature of the results is the rapid reduction in change detection performance in the unchanged condition over the first 1,000 ms. This suggests some form of decay of the memory trace for the stimulus set before the test display is presented. One possibility is that performance in this condition is initially based on a representation in iconic memory that automatically retains location information. These locations could then aid the search in the test array for changes to the color–shape bindings until the icon decays along with the location information it contains. Clearly, location is not used as an effective cue beyond 1,500 ms, since performance is no different from performance in the randomized condition for the longer intervals, suggesting that location is not included in the more stable representation in VSTM that comprises only color–shape bindings.

Another striking feature of the results is that performance in the randomized-location condition was better when tested at longer study–test intervals, as compared with the shorter intervals. One possible interpretation is that within the first second after stimulus offset, the high-fidelity spatial information available in a sensory trace begins to rapidly decay, while a representation in VSTM is formed of the task-relevant feature bindings. The decaying sensory trace would contain both task-relevant features and the task-irrelevant feature of location for items in the study array, and so location acts as an effective cue for the unchanged-location condition, whereas changes in location are disruptive in the randomized-location condition. As the task-relevant features are transferred into a representation in VSTM, performance would rely progressively less on the location-bound icon and progressively more on a stable representation of the task-relevant bound features in VSTM. As a result, because location is not task relevant, it becomes progressively less effective as a memory cue in the unchanged condition (contributing to progressively poorer performance) and progressively less disruptive in the randomized condition (resulting in progressively better performance). This idea that performance is supported concurrently by two changing, and possibly conflicting, memory codes over the first 1,000 ms is supported by the observation that performance in the two conditions converges as the study–test interval is increased, and at 1,500 ms, performance is no different for the randomized and the unchanged conditions when, on this account, performance is supported solely by VSTM.

The exact mechanism underlying the removal of location information from object representations is not clear from this experiment alone. One possibility is that task-irrelevant information passively decays as stimulus information is consolidated in VSTM. A related possibility is that there is an active process of removal or inhibition of the disruptive task-irrelevant feature of location while object representations comprising bindings of color and shape are formed in VSTM. The disruptive effects of randomization for the immediate test condition demonstrate clearly that location cannot be inhibited by processes of selective attention during the presentation of the study array or immediately following its offset. Therefore, any such process of inhibition of locations requires a period of 1,000–1,500 ms after stimulus offset to complete. This is a plausible account but raises questions as to precisely how such a selective inhibitory process operates postperceptually. Another possibility is that there is active visual rehearsal of the task-relevant bindings of shape and color and that there is decay over around 1,500 ms of the location information, which is not rehearsed. However, it seems unlikely that very much selective rehearsal of this nature could occur within a period of 1,500 ms. We will attempt to distinguish between these possibilities in Experiments 2 and 3.

In summary, whereas the results from previous studies have suggested that location is the most important cue in perceptual binding (Fahle & Koch, 1995; Jiang et al., 2000; Keele, Cohen, Ivry, Liotti, & Yee, 1988; Phillips, 1974; Treisman & Gelade, 1980), the results of Experiment 1 suggest that this may be true only during the stimulus display and for a short period thereafter. At longer intervals, changing location induced no more forgetting of color–shape binding than did having location consistent between study and test. Performance at 1,500 ms and beyond is the same as if no change in location had taken place. This suggests that location has no special status in VSTM for bindings in the cases in which location is not relevant and after those bindings have been formed. The result is also consistent with the view that location information is not retained in any memory representation that is available at these longer intervals—or at least, any such information does not affect the search for color–shape changes in the test display. Moreover, these results point to a delay of between 1,000 and 1,500 ms as an indication of the time after which change detection performance appears to rely solely on a memory representation comprising the bindings of task-relevant features.

Experiment 2

Having observed that changing location between study and test is highly disruptive of performance at short (0, 500, or 1,000 ms), but not at longer (1,500 ms and more) intervals, we next asked whether this pattern is specific to changing location. Can other task-irrelevant features of objects be excluded from bindings over time? In Experiment 2, we asked whether changing shape randomly between study and test disrupts the binding of color and location.

As was noted earlier, changing location may affect the search process at test, but it does not typically change the identity of an object, whereas changing shape might well do so. Therefore, there are good reasons to predict that the data pattern for changing shape as a task-irrelevant feature between study and test will be rather different from that for changing location—not least given the body of evidence suggesting that features, such as shape and color, that identify an object are processed via a different neuroanatomical pathway than is location (e.g., Carlesimo, Perri, Turriziani, Tomaiuolo, & Caltagirone, 2001; Funahashi, Takeda, & Watanabe, 2004; Ruchkin, Johnson, Grafman, Canoune, & Ritter, 1997; Smith & Jonides, 1995, 1999; Ungerleider & Mishkin, 1982). This dissociation between the so-called ventral and dorsal processing streams might also suggest that color–shape bindings should be particularly strong given that these features are processed along the same (ventral) pathway and can define an object. Although location allows the object to be detected initially in the stimulus display, it might be much less relevant for defining a target object after it has been perceived. If color–shape bindings in VSTM are normally strong because they define an object, whereas location is not normally a defining feature of an object, changing shape as a task-irrelevant feature might be more disruptive of memory than was changing location in Experiment 1.

Conversely, one possible interpretation of Experiment 1 was that the task-irrelevant feature is held in a rapidly decaying sensory trace and only the task-relevant features are bound in a more stable representation that is being formed in VSTM. On this account, we would expect disruption of location–color binding by randomizing shape and progressively less disruption with longer study–test intervals as the sensory trace decays. Moreover, because locations do not change between study and test, the retention of location information in the representation might aid in the search in the test display even when shape changes. Therefore, the impact of changing shape might be less disruptive and less long-lasting than was the impact of changing location in Experiment 1.

Finally, if location is particularly important in the binding of shape and color and shape has less of a role to play in binding location and color, we might expect location–color binding to be rapid and robust. Therefore, changing shape between study and test might have little impact on location–color binding, and we might expect either no disruption at any interval, or a smaller amount of disruption for the shortest study–test intervals than was found in Experiment 1— for example, at 0 and 500 ms—but perhaps no disruption and convergence between conditions for intervals of 1,000 ms or more. At the longer study–test intervals, the process of removing or inhibiting the irrelevant shape feature from its relatively weak binding to location and color would have been completed, leaving an object file comprising just location and color.

Method

Participants

Twelve students (6 men and 6 women) between the ages of 18 and 25 years participated and were given £10 as an honorarium. All participants provided informed consent and reported normal color vision and normal or corrected-to-normal visual acuity. They were naïve as to the experimental hypotheses, and none had participated in the previous experiment.

Apparatus, stimuli, design, and procedure

The apparatus, stimuli, design, and procedure were identical to those in Experiment 1, except that, on 50% of the trials, 6 shapes were randomly chosen from a set of 12 shapes and reallocated to different items (color–location combinations) between study and test (randomized-shape condition). Participants were to ignore the change in shapes and to remember the combinations of color and location. On half of the randomized-shape trials and on half of the unchanged-shape trials, either two colors swapped locations and all of the shapes remained in the same position between study and test, or two of the color–shape combinations swapped locations (see the middle left panel of Fig. 1). The task was to detect whether the location–color binding had changed, while ignoring the changes to the shapes. This experiment used the same range of study–test intervals as in Experiment 1. Each participant was tested on 2 consecutive days at the same time of the day, half being tested first with randomized shapes, and the other half being tested first with unchanged shapes.

Results

Mean change detection accuracy (d′) across study–test intervals for the unchanged-shape and randomized-shape conditions is shown in the middle right panel of Fig. 1. Hits and false alarms for each condition are given in the Supplemental material. Analysis of the d′ values showed a significant main effect of unchanged/randomized shapes, F(1, 11) = 24.096, MSE = 0.444, p < .001, partial η2 = .687, in that memory for bindings was significantly poorer when the shapes of the stimuli were changed between study and test. The data for the main effect of study–test intervals violated the assumption of sphericity, and so the Greenhouse–Geisser correction was applied. This effect was significant, F(2.744, 30.189) = 13.594, MSE = 0.446, p < .001, partial η2 = .553, indicating that memory for bindings differed across the study–test intervals. Critically, the two variables interacted, F(5, 55) = 24.105, MSE = 0.183, p < .001, partial η2 = .687. As study–test intervals increased, performance decreased in the unchanged-shape condition, whereas it increased in the randomized-shape condition. Paired comparisons using t tests at each of the six study–test intervals, with Bonferroni adjustment (p < .008), showed that the differences between the means for the unchanged and the randomized conditions were significant at 0 ms t(11) = 11.703, p < .001, and approached significance at 500 ms, t(11) = 2.561, p < .013, with no significant differences thereafter, all ps > .05. Given that we were predicting the direction of the difference between conditions (poorer for randomized shapes), a one-tailed test was adopted for significance.

Single degree of freedom polynomial tests showed reliable negative-slope linear, F(1, 11) = 46.931, MSE = 0.532, p < .001, partial η2 = .810, quadratic, F(1, 11) = 30.468, MSE = 0.229, p < .001, partial η2 = .735, and cubic F(1, 11) = 20.213, MSE = 0.193, p < .001, partial η 2 = .648, trends. In the randomized-shape condition, only the positive quadratic trend, F(1, 11) = 17.283, MSE = .084, p < .002, partial η2 = .611, was reliable, indicating a curvilinear function.

Discussion

It seems clear from this experiment that changing shape between study and test was disruptive of memory for location–color bindings for immediate test and for a 500-ms study–test interval, but not for intervals of 1,000 ms or more. At these longer intervals, performance was identical to that in the unchanged condition. From the trend analyses, there was a tendency for performance in the randomized condition to be better when tested at 500 ms than when tested at 0 ms and to match the unchanged condition for intervals of 1,000 ms or longer. The pattern is broadly similar to that shown in Experiment 1, except that the initial level of disruption in the randomized condition is not so dramatic, and the disruptive effect disappeared at a shorter study–test interval. This has the consequence of leaving less scope than there was in Experiment 1 for better performance with each increase in the length of the study–test interval. The opposite, decreasing trend appears in the unchanged condition, leading to the convergence of performance across conditions.

In sum, the results of Experiment 2 indicate that randomizing shapes as a task-irrelevant feature is disruptive of change detection performance for location–color bindings, but only at the shorter study–test intervals of 0 and 500 ms. There is no reliable disruptive effect at intervals of 1,000 ms or more. Analogous to Experiment 1, this indicates that some initial binding of shape to color and location occurs automatically during the 200-ms stimulus presentation and that the effects of shape as a task-irrelevant feature are removed during the process of forming a bound representation of location and color in VSTM. Once that representation has been formed, there is no impact of randomizing shape as a task-irrelevant feature.

The finding that the disruptive effect of randomizing shape is less than the disruptive effect in Experiment 1 of randomizing location as a task-irrelevant feature might indicate that location does indeed have a powerful impact on the initial formation of bindings in visual perception. Shape also contributes to the initial binding, but it has a less dramatic disruptive effect when changed. There is therefore no strong evidence to support the hypothesis that shape–color binding is any stronger than location–color binding simply because both color and shape are thought to be processed by the same ventral neural pathway, or because together they define an object. It is very striking, however, that both shape and location result in disruption at shorter intervals, although they differ in initial, perceptual binding potency and the disruptive influence of shape lasts for about 500 ms less than the influence of location. In other words, location has a much greater initial disruptive effect, and it takes more time to remove its effects than it does to ignore shapes. The results of Experiments 1 and 2 therefore appear to be consistent with the idea that there is some form of active removal or inhibition of the task-irrelevant feature from the initial bound representation when a representation is formed in VSTM. One possible account for the difference between the experiments is that the process of inhibition takes less time to complete for shape than it does for location. An alternative is that the memory trace for location at study in Experiment 2 might aid the search process at test, thereby reducing the disruptive effect of a memory trace for the irrelevant changing shape. We will return to this issue after reporting the results of Experiment 3.

Experiment 3

The motivation for Experiment 3 was broadly the same as that for Experiment 2, except that, here, we considered the impact of randomizing color between study and test and examined memory for binding of location and shape. In keeping with ideas pertaining to differential processing of the boundary feature of form and surface feature of color (Grossberg & Mingolla, 1985; Grossberg & Pessoa, 1998; Humphreys, Cinel, Wolfe, Olson, & Klempen, 2000; Humphreys, Hodsoll, & Riddoch, 2009), it might be expected that randomizing color as a task-irrelevant feature would be disruptive at the shorter study–test intervals, but less disruptive than was shape in Experiment 2. Nevertheless, as for the previous two experiments we expected that performance at the longer study–test intervals would converge between the unchanged and randomized conditions, after the influence of the irrelevant feature had been removed from the initial representation. Because color does not serve to locate the object and is not strictly necessary to define the object, and because location might act as a useful cue for search at test, we expected that the disruptive influence of randomizing color as task irrelevant between study and test would be less than was observed for shape in Experiment 2 and that this influence would be removed at a shorter study–test interval.

Method

Participants

Twelve students (6 men and 6 women) between the ages of 18 and 25 years participated and were given £10 as an honorarium.

Stimuli, design, and procedure

The stimuli, design, and procedure were the same as those in Experiment 2, except that in the randomized condition, instead of changing the shape of all the objects in the display, we changed the color of all the objects by selecting 6 from a set of 12 colors and reallocating colors to different shape–location combinations between study and test. Participants were to ignore the change in colors and to remember the combinations of shape and location. The task was to detect whether any of the location–shape bindings had changed, regardless of color changes. The sequence of events in this experiment is illustrated in the bottom left panel of Fig. 1.

Results

Mean change detection accuracy shown as d′ scores across study–test intervals for the unchanged-color and randomized-color conditions are shown in the bottom right panel of Fig. 1. Hits and false alarms are given in the Supplemental material.

Analysis of the d′ values showed a significant main effect of color condition, F(1, 11) = 23.573, MSE = 0.627, p < .001, partial η2 = .682, in that memory for bindings was significantly reduced when the color of the stimuli was randomized from study to test display. The main effect of study–test interval was also significant, F(2.282, 25.101) = 35.548, MSE = 0.559, p < .001, partial η2 = .764, with Greenhouse–Geisser correction, indicating that memory for shape–location bindings was significantly different for different study–test intervals. There was a significant interaction between these variables, F(5, 55) = 11.493, MSE = 0.237, p < .001, partial η2 = .511. Pairwise comparisons using t tests with Bonferroni adjustment, p < .008, showed that differences between the means for the unchanged and the randomized conditions on one-tailed tests, with direction predicted, were reliable at 0 ms, t(11) = 9.074, p < .001, and approached significance at 500 ms, t(11) = 2.333, p < .020, at 1,000 ms, t(11) = 2.129, p < .029, and at longer intervals, all ps > .025.

Despite the significant interaction, performance is progressively poorer as the study–test interval increases when color is randomized from initial to test display, as well as when color remains unchanged. Single degree-of-freedom polynomial tests for unchanged colors showed negative slope linear, F(1, 11) = 92.285, MSE = 0.423, p < .001, partial η2 = .893, quadratic, F(1, 11) = 50.036, MSE = 0.241, p < .001, partial η2 = .820, and cubic, F(1, 11) = 5.105, MSE = 0.259, p < .045, partial η2 = .317, trends. For the randomized-color condition, single degree-of-freedom polynomial tests also showed a negative slope linear trend, F(1, 11) = 14.549, MSE = .389, p < .003, partial η2 = .569, this slope being shallower than that for the unchanged-color condition. Both slopes are characterized by an initial decline, followed by a leveling out. However, in both cases, the slope is negative. This is in contrast to the location and shape experiments, in which performance improved with increasing study–test intervals when each feature was randomized.

Discussion

From Experiment 3, it is clear that randomizing colors as a task-irrelevant feature between study and test was disruptive of memory for location–shape bindings for the study–test intervals of 0 and 500 ms, but this is much less clear for intervals of 1,000 ms or more. This early stage disruption was much less than that observed for location change or shape change in Experiments 1 and 2. Even for the 0-ms study–test delay, performance in the randomized-color condition is higher than it is for longer delays in the unchanged-color conditions. This leaves no scope for the increase in performance levels for the randomized condition with increasing study–test intervals, as was found in Experiments 1 and 2. However, the gradual loss of the disruptive effect of the task-irrelevant feature as the study–test intervals increase remains clear. Moreover, although the disruptive effect of randomizing color is less than that for randomizing location or shape in the earlier experiments, the convergence of performance levels between the unchanged and randomized conditions still appeared, and, in this case, at the same interval as it appeared for shape in Experiment 2 (1,000 ms). This seems to suggest that color is less important for initial binding than is shape and that it takes less time to remove it from the representation in VSTM. The results are also consistent with the suggestion that location as a relevant feature maintained within the memory representation might facilitate the search process at test, thereby reducing the disruptive impact of changing color as an irrelevant feature.

General discussion

Across three experiments, we examined the extent to which location, color, and shape contribute to the binding of those features in VSTM at varying intervals after stimulus offset. What is clear from these experiments is that designating one presented feature as irrelevant and then randomizing it from study to test is disruptive of detecting changes in bindings between the remaining features at shorter, but not at longer, study–test intervals. Our results for Experiment 1 were predicted from Treisman and Zhang (2006), who demonstrated the reduced importance of location as a cue in binding at 900 ms and thereafter, as compared with 100 ms. However, Treisman and Zhang did not comment in detail regarding the interpretation of their findings at the longer study–test intervals. Also, we used a blocked design and a more fine-grained range of study–test intervals in Experiment 1, showing the period over which this importance of location is reduced when participants know that location will not be informative, and also investigated the change in importance across study–test intervals of shape and color as cues in memory for binding in Experiments 2 and 3. The results showed that shape and color also play a role in initial perceptual integration, even if their influence is weaker than that of location. The results also suggested that after 1,500 ms, location was no longer important when it was task-irrelevant, nor indeed were the other two features after about 500 ms. These results are consistent with the importance of locations postulated by the FIT at the point of perception. However, they are also consistent with the view that when location information is available within the memory trace, it might affect the search process at the time of test, hindering performance when it changes between study and test and aiding performance when it is consistent between study and test. The results run contrary to the assumption of FIT that location is required in VSTM after the object representations comprising bindings of the other relevant features have been formed.

The correct perception of the location of objects in space has survival value in our daily interaction with the world. Location is crucial for finding an object, and changing location of a target immediately after it has been identified is highly disruptive. However, once a target has been found and its features encoded, it is primarily shape and, to a some extent, color that can identify an object and allow us to recognize that object subsequently in different locations. Only if the object is found consistently in the same location (e.g., a building in a city, food in the refrigerator) does location help define that object or aid in its detection. In contrast, a change of color or a change of shape changes the nature of the object, (e.g., making food more or less safe). These broader considerations fit well with the results reported here showing that location is important early in the process of binding and in detecting changes in binding, but not once the representation has already been formed in memory. Location is crucial in visual perception but is treated no differently than other features in the workspace of visual working memory (Logie, 2003; Logie & van der Meulen, 2009).

The paradigm we have used here also offers a means to track the time course over which performance relies on the iconic trace or relies on VSTM. Moreover, whether change detection performance is affected by an irrelevant feature may indicate whether a trace of that irrelevant feature remains in the memory representation at the time of test. The fact that the greatest disruption to performance by randomizing task-irrelevant features between study and test occurs when the test display is presented immediately following stimulus offset suggests that all features participate in the initial representation of features during the 200 ms of the display, and the resulting object file includes the irrelevant feature. We considered one possible account of the disruption due to randomizing one task-irrelevant feature immediately after offset of the study display. Specifically, the representations, based on iconic memory (e.g., Phillips, 1974), comprise both task-relevant and task-irrelevant features. Therefore, the test display fails to match the representation of the study display, and this results in poor detection of changes in the binding of task-relevant features. Different features are more or less disruptive when they fail to match between the icon and the test display. Given that the icon is retinotopic, a shift of location would make it very difficult to find relevant targets in the test display, so randomizing locations would be very disruptive. A shift of color or of shape would not have an impact on search for the target items in the test display but would still result in a mismatch with the contents of the icon. This might account, in part, for the larger effect in Experiment 1 than in Experiments 2 and 3 but does not readily account for the differences observed between Experiments 2 and 3.

The apparent improvement in performance present with increasing study–test intervals when locations were randomized in Experiment 1 was not obtained in Experiment 3 for colors, but in that case, performance in the randomized condition for immediate test was much less disrupted at 0 ms, with performance being higher than the later asymptotic performance in the unchanged, as well as the randomized, condition, This left no scope for better performance at the longer study–test intervals. In Experiment 1, the better performance at the longer intervals was made possible by the much greater disruptive effect of randomizing location at short study–test intervals. Performance in all three experiments in the unchanged condition after 1,500 ms most likely represents the highest level of performance that could be obtained after that study–test interval, regardless of whether or not there was a disruption in the interim. Therefore, only the low performance at 0 ms in the randomized-location condition offers scope for the performance to increase to the asymptotic level when tested at longer study–test intervals.

More important is the fact that performance levels between the unchanged and the randomized conditions converged as study–test interval increased. This convergence is clear in the interactions observed in all three experiments. A possible account for the interactions between the randomized and the unchanged conditions over study–test intervals in all experiments is that as the iconic trace decayed, performance was more reliant on bindings formed only between the task-relevant features in VSTM. The representation in VSTM then allowed a closer match for comparison with the combinations of task-relevant features in the test display, because the mismatched irrelevant feature was not a part of the representation. The process of forming task-relevant bindings as object representations in VSTM was complete after about 500 ms when shape and color were task irrelevant and after 1,500 ms when location was task irrelevant, and thus, gradually, the task-irrelevant feature became irrelevant for performance as well.

The gradual process of deleting or inhibiting a feature from VSTM that is task irrelevant and possibly disruptive has been identified as an important aspect of working memory function (e.g., Friedman & Miyake, 2004; Miyake et al., 2000) that appears to be affected by aging (e.g., Hasher & Zacks, 1988). This inhibitory process is superficially similar to, but rather different from, the processes of attentional selection. Inhibitory processes in short-term memory occur after stimulus presentation and may override automatic capture of attention. For example, in antisaccade tasks in which participants have to inhibit the tendency to fixate the target and have to look in the opposite direction, performance is highly correlated with working memory capacity (e.g., Unsworth, Schrock, & Engle, 2004). Woodman and Luck (2007) also have shown that participants can strategically use the content of visual working memory to facilitate or inhibit their performance on a concurrent visual search task. The experiments reported here suggest that, at least in this paradigm, this process of inhibition can be complete in about 1.5 s and, sometimes, more rapidly, as reflected in the pattern for the randomized condition when performance reaches an asymptote. This period also appears to reflect the pattern of forgetting, with performance in the unchanged condition declining and reaching an asymptote over a period of around 1,500 ms.

In sum, the process of deleting or inhibiting an irrelevant and disruptive feature from VSTM and the forgetting of details from VSTM is common to location, shape, and color but occurs at different rates. It most likely involves executive functions (in this case, selection and inhibition) within working memory acting on the VSTM trace. These aspects of visual working memory appear to be subsequent to and different from the processes involved in visual attention to a stimulus display and initial, rapid perceptual integration of all features. Visual working memory also appears to handle the retention of feature bindings rather differently than does visual attention. Making clear the differences between a visual attentional process and the processes in working memory in the few seconds after a stimulus has disappeared appears to offer important additional insight into the formation of bound object representations.