Introduction

People understand and interact with their environment by using conceptual knowledge about objects and their properties. According to various models of conceptual representation (Caramazza & Shelton, 1998; Farah & McClelland, 1991; Humphreys & Forde, 2001), concepts can be described as distributed patterns of activation across units representing semantic features. Several of these models propose that at least a subset of the features are modality-specific (e.g., Cree & McRae, 2003; Plaut, 2002; Rogers et al., 2004; Vigliocco, Vinson, Lewis, & Garrett, 2004). This is consistent with the embodied cognition view on mental representation (e.g., Barsalou, 1999, 2008a; Grush, 2004), which proposes that concepts are grounded in sensory-motor processing. According to this view, object concepts are learned through recurrent experiences of perceiving and interacting with a particular object. During those experiences, the patterns of activity in sensory-motor brain areas are captured and stored in memory to form elaborated, multimodal knowledge structures, called simulators. Representation is achieved by reactivating a subset of this stored knowledge to construct a specific simulation. The exact content of a particular simulation depends on the individual’s experience with the simulated concept, as well as on situational factors such as current goals and task demands. As a result, each simulation is unique.

A method often used to assess conceptual knowledge about objects and their properties is the property-verification task. In this task, the word for a concept (e.g., banana) is presented together with the word for a property that might be true (e.g., yellow) or false (e.g., blue) for the concept. Participants have to judge whether the property is true or false as quickly as possible. According to the simulation theory, the response time reflects the effort required to form an adequate simulation of the concept and property. Since simulation is analogous to sensory-motor processing, this theory predicts that the response time is affected by factors that also influence perception and action. For example, Solomon and Barsalou (2004) found that property-verification response times depend on perceptual factors such as the size and perceptual salience of a property.

Given that representations are flexible and task-dependent (e.g., Barclay, Bransford, Franks, McCarrel, & Nitsch, 1974; Barsalou, 1993), a simulation may be dominated by one sensory modality, with other modalities engaged to a lesser extent. For example, when performing a property-verification task, one may form different simulations of SAXOPHONE,Footnote 1 depending on the property to be verified. In order to verify that a SAXOPHONE is shiny, a predominantly visual simulation is created, whereas to verify that a SAXOPHONE sounds loud, a predominantly auditory simulation is created. Evidence for such flexible and task-dependent recruitment of modality-specific features in representations is provided by the modality-switch effect. If two representations are simulated by different modality-specific systems, then constructing them consecutively will involve a switch of attention from one modality to another. This will result in a longer reaction time and/or a lower accuracy, similar to modality-switching costs in perceptual tasks (Spence, Nicholls, & Driver, 2001). Conceptual modality-switch effects have been found in various property-verification studies (e.g., Marques, 2006; Pecher, Zeelenberg, & Barsalou, 2003; Pecher, Van Dantzig, & Schifferstein, 2009; Vermeulen, Corneille & Niedenthal, 2008). Other studies demonstrated direct interactions between conceptual processing and perceptual processing, providing even stronger evidence for the theory that the sensory-motor system is involved in conceptual representation (Van Dantzig, Pecher, Zeelenberg, & Barsalou, 2008; Vermeulen et al., 2008). Brain imaging studies have provided corroborative evidence. Goldberg, Perfetti, and Schneider (2006) applied functional magnetic resonance imaging (fMRI) to scan participants during a property-verification task that involved properties from four different sensory modalities (vision, sound, touch, and taste). When participants verified properties from these modalities, activity was recorded in corresponding sensory areas. Thus, verification of visual properties caused activation of visual brain areas, whereas verification of auditory properties caused activation of auditory brain areas. Similar results were found by Simmons et al. (2007) and Kan, Barsalou, Solomon, Minor, and Thompson-Schill (2003). In addition, Cree and McRae (2003), see also Plaut (2002) showed that damage to modality-specific sensory areas may lead to semantic deficits. In sum, multiple studies provide evidence that conceptual representation takes place in the modality-specific brain areas that are also used during actual experience of instances of the represented concepts.

As has become clear, the property-verification task is well suited to the assessment of conceptual knowledge. When applying the task to study modality-specific processing, one should realize that property-verification items vary widely in their degree of modality-exclusivity. Whereas some properties are perceived exclusively by one specific sensory modality (e.g., PIANO-black is purely visual), others may be perceived by multiple modalities (e.g., SHOWER-splashing may be perceived by vision, audition, and touch). It is therefore important to carefully select items based on their modality-exclusivity. Lynott and Connell (2009) found that controlling property-verification items for modality-exclusivity results in a markedly larger modality-switch effect (in terms of effect size). During pilot studies in our own lab, we also found that the size of the modality-switch effect depended on the modality-exclusivity of the items used (e.g., Van Dantzig et al., 2008).

In the present study, in order to select concept-property items on modality-exclusivity, we obtained modality ratings for a large set of items. For each item, participants rated the degree to which the property could be perceived through five different senses (vision, audition, touch, smell, and taste). The modality ratings form a significant extension of those collected by Lynott and Connell (2009). In particular, while they obtained ratings for isolated properties (e.g., yellow), we obtained ratings for concept-property combinations (e.g., BANANA-yellow). It is important to realize that a property is never experienced in isolation, but is always connected to a particular object. In representation, independent simulators may exist for properties (Barsalou, 2008b). However, since simulation is a dynamic process, property-verification does not involve the activation of invariant representations of concepts and properties (Barsalou, 2009). Instead, the representations of the concept and property mutually constrain each other’s content. Thus, the concept WINE is represented differently when one has to verify the property white than when one has to verify the property sweet (Pecher, Zanolie, & Zeelenberg, 2007; Pecher, Zeelenberg, & Barsalou, 2004). Conversely, the property white is represented differently for the concept WINE than for the concept SNOW (Halff, Ortony, & Anderson, 1976; Solomon & Barsalou, 2001). In an experiment by Murphy and Andrew (1993) participants provided synonyms or antonyms for adjectives that were presented in conjunction with a concept, e.g., fresh WATER, fresh AIR or fresh BREAD. The concept strongly influenced which synonyms and antonyms were provided for a particular adjective. The degree to which a specific sensory modality is involved in perceiving a particular property may also differ across concepts. For example, the property hot for the concept LIGHTBULB is perceived almost exclusively by touch, as is well known by anyone who has ever burnt their fingers unscrewing a lamp. Fortunately, in other objects the property hot may also be seen (as in a red-hot GRILL or a steaming mug of COFFEE) or heard (as in a sizzling-hot PAN), so that we do not always have to find out the hard way. Strictly speaking, in these situations it is not the actual property hot that is perceived visually or auditorily, but another visual (steaming, red) or auditory (sizzling) property that merely indicates the hotness of the object. However, according to the simulation theory, property-verification involves simulating a (partial and schematic) sensory-motor experience, by reactivating patterns laid down during earlier experiences. As a result, properties (steaming, sizzling) that frequently co-occurred with the target property (hot) during those experiences may also be included in the simulation and may be used to indicate the presence of the target property. Thus, whereas the property hot in itself is exclusively tactile, in a property-verification task it may be associated with other-modality properties, just as in real experience. The degree to which a particular property is perceived by one or (indirectly) by multiple sensory modalities may vary across concepts.

To capture this variability in property meaning, we paired each property with two concepts, resulting in two different concept-property items. For example, the property plain was paired with the concept FOOD and with the concept FABRIC. For each concept-property item, participants rated the degree to which the property is experienced through five different sensory modalities (vision, audition, touch, smell, and taste). This resulted in a specific modality profile for each item, a five-dimensional vector representing the mean strength ratings on all five modalities. The modality with the highest rating was defined as the dominant modality for that item. Item pairs could be unidominant, with both items having the same dominant modality (e.g., pink FLAMINGO and pink COTTON CANDY are both predominantly visual) or bidominant, with the two items having different dominant modalities (e.g., plain FABRIC and plain FOOD are predominantly visual and gustatory, respectively). Additionally, we computed the angle between the two vectors of each item pair, as well as the cosine of this angle. These measures indicate how similar the modality profiles of the two items of a pair are. Finally, we computed modality-exclusivity scores for each concept-property item, indicating to what extent properties are perceived exclusively by one modality. When comparing our data with those of Lynott and Connell (2009), some interesting patterns arose. In the remainder of this paper, we describe how the norms were collected and we provide the obtained ratings and statistics. Furthermore, the norms were validated in a memory experiment. During the study phase of this experiment, property words were presented together with one of their two normed concepts (e.g., “plain food”). Subsequently, memory for the property words was tested using a cued recognition test in which the property was presented either with the same concept word as in the study phase (“plain food”) or with the other concept of the item pair (“plain fabric”). When the concept was repeated, recognition of the property word was better than when the concept was switched. More interestingly, this effect was influenced by the similarity of the modality profiles of an item pair; the switch cost was larger for items with very different modality profiles (e.g., plain fabric – plain food) than for items with highly similar modality profiles (e.g., grey elephant – grey pebble). This result indicates that our norms reflect actual differences in the representation of properties and concepts.

Norming study

Stimuli

We selected 387 properties that could be experienced through one or more sensory modalities. Of these 387 items, 364 properties were obtained from the set of Lynott and Connell (2009) and the remaining 23 properties had been used in earlier studies in our lab. Each of the properties was paired with two different concepts, yielding a total number of 774 concept-property items. The set of items was divided into ten different lists that consisted of 77 items (six lists) or 78 items (four lists) each. Properties and concepts were never repeated within a list. Participants received only one list, such that they never rated the same property or concept twice. The complete set of items is available in Electronic supplementary material.

Participants

Four hundred thirty-one undergraduate students from the University of California, San Diego participated in the experiment in return for course credits. Given that there were 10 different lists of items, each item was rated by at least 40 participants.

Procedure

The procedure was similar to that used by Lynott and Connell (2009). The E-Prime 2.0 software package (Psychology Software Tools, Pittsburgh, PA) was used to present the items and collect the participants’ responses. Concept-property items were presented in random order. During each trial, a screen was presented with the text “To what extent do you experience [concept] being [property]?” (e.g., “To what extent do you experience a banana being yellow?”). This text was followed by five separate rating scales for each perceptual modality, labeled “by feeling through touch”, “by hearing”, “by seeing”, “by smelling”, and “by tasting”. To avoid any order effects, the order of the rating scales was randomly determined for each participant. Participants rated the extent to which they would experience the property through each of the five senses from 0 (not at all) to 5 (greatly), by clicking on the corresponding number with the computer mouse. Once the property had been rated on all five modalities, the participant clicked on the “Next Item” button to proceed to the subsequent item. If participants were unfamiliar with a concept or property, they were instructed to skip the item by pressing the “Next Item” button. The instruction emphasized that there were no right or wrong answers and that participants could give a high rating for as many or as few of the modalities as they wished, and a low rating for as many or as few of the modalities as they wished. Participants were encouraged to use their first intuitions about the items, and not to think too long about their answers. The experiment was self-paced and lasted approximately 20–30 min. The complete instructions are provided in Appendix.

Results and discussion

The data from fourteen participants were excluded from the analysis because they were non-native English speakers. One other participant was excluded because he/she failed to comply with the instructions, leaving a total of 416 participants.

The ratings provided by the participants were used to compute mean ratings for each concept-property item. The complete set of items and their ratings can be found in Electronic supplementary material.

We first analyzed each item separately. For each item, the modality with the highest mean rating was defined as the dominant modality for that item. Table 1 shows the distribution of items and ratings for each dominant modality. Table 2 shows the correlations between the ratings on each of the five modalities.

Table 1 Number of properties per dominant modality, with modality exclusivity scores and mean strength ratings (0–5) on the five perceptual modalities
Full size table
Table 2 Correlation matrix for mean strength ratings per item on the five perceptual modalities
Full size table

Modality exclusivity

Modality exclusivity is the extent to which a property is perceived exclusively by one modality. Following Lynott and Connell (2009), an item’s modality exclusivity was derived from the mean strength ratings on the five modality scales. Modality exclusivity was computed as the range of scores across modalities divided by the sum of the scores. The resulting modality exclusivity scores range from 0 to 1 (translated to 0% to 100%). A score of 0% is obtained when equal scores are given for each modality, indicating that the property is completely multimodal. A score of 100% is obtained when the property is completely experienced by one single modality. Modality-exclusivity for individual properties ranged from 11 to 98%, with a mean of 50.6% (SD  =  16.2%). Figure 1 displays the distribution of exclusivity scores for the items grouped by dominant modality. Exclusivity scores differed across dominant modalities, F(4, 769) = 34.60, p  <  .001. Posthoc tests with Bonferroni correction revealed that the exclusivity scores of the visual and auditory items (55%) were higher than the scores on the other items (all p values < .001), but did not differ from each other. The olfactory items (46%) and haptic items (44%) did not differ from each other significantly, but both were higher than the gustatory items (37%) (both p values < .05).

Fig. 1
figure 1

Distribution of exclusivity scores, grouped by dominant modality

Full size image

Properties are never experienced in isolation, but are always connected to a particular object. These objects influence the way in which properties are perceived and represented. As a result, a property may receive different modality ratings depending on the concept that it is combined with. This variance can be investigated in our dataset, because each property occurs twice, in combination with two different concepts. The two concept-property items sharing the same property formed an item pair, which could be analyzed in several ways.

Item pairs could be unidominant, with both items of the pair having the same dominant modality (e.g., pink FLAMINGO and pink COTTON CANDY are both predominantly visual) or bidominant, with the two items having different modalities (e.g., plain FABRIC and plain FOOD are predominantly visual and gustatory, respectively). Our set contained 255 unidominant pairs (66%) and 132 bidominant pairs (34%). Table 3 shows how often each combination of dominant modalities occurs and provides examples of each modality combination.

Table 3 Frequencies and examples of unidominant and bidominant item pairs by modality combination
Full size table

Interestingly, even in the unidominant item pairs, our data show that properties are experienced differently across concepts. For example, the property round received high visual ratings for the concepts COIN and PLANET, indicating that this property is experienced predominantly visually in both concepts. The property was also rated as strongly haptic for the concept COIN, but not for the concept PLANET (see Fig. 2). After all, people can (and often do) feel that a coin is round, but they never feel the round shape of a planet. Such subtle differences in the way in which properties are experienced are reflected in our ratings.

Fig. 2
figure 2

Modality strengths of the property round for the concepts COIN and PLANET

Full size image

Similarity of item pairs

Each concept-property item has a specific modality profile, a five-dimensional vector representing the mean strength ratings on the five modalities. In order to determine how similar the two modality profiles of a particular item pair are, we computed the angle between their two vectors. If two items have highly similar modality profiles (e.g., spiky HEDGEHOG – spiky SEA ANEMONE), the angle between the vectors will be small. If the profiles are very different (e.g., honeyed COOKIES – honeyed VOICE), the angle between the vectors will be large. Overall, unidominant item pairs had smaller angles (M  =  15.8 degrees, SD  =  10.6) than bidominant pairs (M  =  40.6 degrees, SD  =  21.0), t(385)  =  15.47, p  <  .001. Note, however, that the two measures do not map completely. The two items of a unidominant pair, while having the same dominant modality, could have very different scores on the remaining modalities, rendering them very dissimilar. Conversely, the two items of a bidominant pair could have very similar modality profiles, while their dominant modalities differ by a very small margin. Because the similarity measure is based on the full modality profile, it is more sensitive to subtle differences between the items of a pair than when one only takes into account the dominant modalities. The angle and the cosine of the angle of each item pair are reported in the data file (see Electronic supplementary material).

The majority of properties used in our dataset (364 out of 387 properties, 94%) were also normed by Lynott and Connell (2009). By comparing our modality exclusivity scores for this subset with the scores of Lynott and Connell (furthermore referred to as L&C exclusivity scores), some interesting patterns become apparent. First, our modality exclusivity scores are slightly higher (M  =  0.51, SD  =  0.14) than the L&C exclusivity scores (M  =  0.47, SD  =  0.17), t(363) = 7.50, p  <  .001. This difference might be because properties were paired with specific concepts in the current study, whereas they were presented in isolation in the study of Lynott and Connell. The latter may have caused participants to represent different concepts for the same property. For example, when asked to rate the property rotten, one participant might form a representation of an apple with brown spots and a mushy texture that indicate its rottenness. Thus, this participant will rate rotten as predominantly visual and haptic. Another participant might instead form a representation of a rotten egg, distinguished by its sulfuric smell. Thus, this participant will rate rotten as predominantly olfactory. It is also possible that a participant may represent multiple concepts for a particular property. This participant may choose to provide modality ratings that average across the represented concepts. The variance in represented concepts, either between participants or within the same participant, will result in lower modality exclusivity scores. Alternatively, participants may form a generic, multimodal representation of the property rotten, based on earlier experience but abstracted from the specific objects with which it was experienced. Such generic representations will result in lower modality exclusivity scores than the more specific representation that is formed when the property is presented together with a particular object.

In any case, we would expect the difference between our exclusivity scores and the L&C scores to be more pronounced in the bidominant item pairs than in the unidominant item pairs. This was tested by splitting the data into unidominant and bidominant items and performing the comparison for both groups separately. In the unidominant items, our exclusivity scores (M  =  .53, SD  =  .14) are similar to the L&C exclusivity scores (M  =  .52, SD  =  .16), t(236) = 1.71, p  >  .05. In the bidominant items, on the other hand, our exclusivity scores are significantly higher (M  =  .47, SD  =  .12) than the L&C scores (M  =  .35, SD  =  .10), t(126) = 9.85, p  <  .001. Figure 3 plots the proportion of unidominant and bidominant item pairs as a function of L&C exclusivity. The proportion of unidominant item pairs increases as exclusivity scores become higher. This pattern indicates that in the study of Lynott and Connell, participants might have represented different concepts for the same property, which increased the variance and lowered the exclusivity score especially for those items in which the different concepts lead to very dissimilar response vectors.

Fig. 3
figure 3

Proportion of unidominant and bidominant item pairs as a function of L&C exclusivity score

Full size image

Memory experiment

Our norms show that when properties are paired with different concepts, the resulting modality profiles can vary widely in similarity. Next, we wanted to investigate whether the similarity in modality profiles in our norms reflects similarity in the mental representations of the properties. For example, the norms indicate that the representation of mild might be very different in the context of CURRY than in the context of WINTER, whereas the representation of grey might not differ so much between ELEPHANT and PEBBLE. Previous studies showed that both implicit (Zeelenberg, Pecher, Shiffrin, & Raaijmakers, 2003) and explicit (Barclay et al., 1974; Tulving & Thomson, 1973; Zeelenberg, 2005) memory for concept nouns are sensitive to the overlap in contextual cues at study and test. Furthermore, such contextual effects might be due to differences in modality specific features of the representations at study and test (Pecher et al., 2004, 2007; Pecher, Van Dantzig, Zwaan, & Zeelenberg, 2009). In the present study, we investigated how memory of property words was influenced by context (i.e., a concept word). During the study phase, property words were presented together with one of their two concepts (e.g., “grey elephant”). Subsequently, memory for the property words was tested using a cued recognition test, in which the property was presented together with either the same concept word as in the study phase (“grey elephant”), or with the other concept of the item pair (“grey pebble”). We selected item pairs that had either highly similar modality profiles (e.g., grey ELEPHANT – grey PEBBLE) or very different modality profiles (e.g., mild CURRY – mild WINTER). We expected a main effect of concept switch; recognition of the property would be worse if the concept was switched than if the concept was repeated. Moreover, we expected the switch effect to be influenced by the similarity of the modality profiles; it should be larger for items with different modality profiles than for items with similar modality profiles. After all, for items with different modality profiles switching the concept would result in very different representations of the property during study and test, whereas this difference would be much less pronounced for items with similar modality profiles.

Method

Participants

Twenty-five undergraduate students from the University of California, San Diego, participated in the experiment in return for course credits. None had participated in the norming study. One participant did not perform the memory task as instructed and was removed from the analysis.

Stimuli

Two sets of 40 properties each were selected. Each property was paired with the two concepts that had been presented in the norming study. The similar modality profile properties had very high cosines (small angles) between the modality vectors for the two concepts, indicating that they had very similar ratings on the sensory modalities. The different modality profile properties had very low cosines (large angles) between the modality vectors for the two concepts, indicating that they had very different ratings on the sensory modalities. The stimuli in the two sets were matched on length of the property word, word frequency of the property and both concept words, and the Latent Semantic Analysis (LSA) cosine between the two concept words (Landauer & Dumais, 1997). Word frequency norms were derived from the Celex database (Baayen, Piepenbrock, & van Rijn, 1993). The characteristics of the two sets of stimuli are provided in Table 4.

Table 4 Characteristics of the items of the similar modality profile items and different modality profile items (standard deviations in parentheses) used in the memory experiment
Full size table

We created four lists to counterbalance the items. Each participant saw only one list; six participants were assigned to each list. Across these lists, each property was used twice in the repeated concept condition and twice in the switch concept condition, and for each of the two conditions a concept was used once as study concept and once as test concept. Examples of the conditions and counterbalancing are provided in Table 5. The resulting lists contained 20 properties in each of the four conditions: similar profile-repeat concept, similar profile-switch concept, different profile-repeat concept, and different profile-switch concept. For the memory test, an additional set of 80 different property-concept pairs was created to serve as new items (distractors).

Table 5 Examples of the design and counterbalancing in the memory experiment
Full size table

Procedure

Participants were instructed to study the properties (presented in uppercase) for a later memory test. They were told that they should also read the concepts (presented in lowercase) because they might be provided at test to help them remember the properties. Each property-concept pair was presented for 3,500 ms in the center of the computer screen. The property was always the left hand item of the pair and the concept the right hand item (e.g., MILD curry). Between items, a 500-ms blank screen was presented. The 80 items were presented in a different random order for each participant. The test phase started immediately after the study phase. Participants were instructed that they would see property-concept pairs, and their task was to indicate whether they had studied the property in the previous phase. They were told that the concept might help their memory, but that they should indicate “old” to all studied properties, whether the concept was studied or not. An item was presented at the center of the screen and remained there until the participant indicated “old” by pressing the m key or “new” by pressing the z key. All studied items and distractors were presented in different random order for each participant.

Results

The average hit rates per condition are shown in Table 6. We analyzed the individual hit rates in a 2 (modality profile) by 2 (concept switch) ANOVA. The results showed a main effect of switch, F(1, 23) = 60.61, p  <  .001, η 2  =  .73, indicating that memory for the properties was better when the concept was repeated than when it was switched. There was no main effect of modality profile, F  <  1, indicating that memory for the two sets of properties was not different. Most interesting, there was a significant interaction between modality profile and concept repetition, F(1, 23) = 5.01, p  <  .05, η 2  =  .18. As we predicted, the memory decrease due to concept switch was larger for different modality profile properties than for similar modality profile properties.

Table 6 Hit rates (standard errors in parentheses) in the memory experiment
Full size table

Conclusions

We collected modality-exclusivity norms for 387 properties, each paired with two different concepts. The results show that modality profiles and modality exclusivity vary widely for a substantial proportion of properties. In a following experiment, we showed that such variability affects recognition memory for the properties. These results indicate that the norms reflect something essential about the representation of properties. When applying the property verification task or a different verbal task to study modality-specific effects, such as the modality-switch effect, it is important to select items carefully based on their modality-exclusivity. This may reduce the variability attributed to error, yielding cleaner data and larger effect sizes, as demonstrated by the study of Lynott and Connell (2009). Because our norms incorporate the variance of properties across concepts, they are more specific than the norms collected by Lynott and Connell. In order to select items for an experiment, the two data sets can be used in conjunction. We recommend our norms for selecting items for a property-verification task or other verbal tasks using specific concept-property combinations (such as the memory task described in this paper). The norms of Lynott and Connell might be preferred for tasks that address more general meanings of properties.