Multisensory contributions to the perception of vibrotactile events

Abstract

We argue that audio-tactile interactions during vibrotactile processing provide a promising, albeit largely neglected, benchmark for the systematic study multisensory integration. This article reviews and discusses current evidence for multisensory contributions to the perception of vibratory events, and proposes a framework to address a number of relevant questions. First, we highlight some of the features that characterize the senses of hearing and touch in terms of vibratory information processing, and which allow for potential cross-modal interactions at multiple levels along the functional architecture of the sensory systems. Second, we briefly review empirical evidence for interactions between hearing and touch in the domain of vibroactile perception and related stimulus properties, covering behavioural, electrophysiological and neuroimaging studies in humans and animals. Third, we discuss the vibrotactile discrimination task, which has been successfully applied in the study of perception and decision processes in psychophysical and physiological research. We argue that this approach, complemented with computational modeling using biophysically realistic neural networks, may be a convenient framework to address auditory contributions to vibrotactile processing in the somatosensory system. Finally, we comment on a series of particular issues which are relevant in multisensory research and potentially addressable within the proposed framework.

Introduction

The feeling of skin dryness or moistness that arises when we rub our hands against each other is subjectively referred to the friction forces at the epidermis. Yet, it has been demonstrated that acoustic information also participates in this bodily sensation, because altering the sound arising from the hand rubbing action changes our sensation dryness/moistness at the skin. Above and beyond the mere curiosity of this phenomenon, dubbed the parchment-skin illusion [53], this illustration touches on the very fundamental issue of how the interplay between different sensory systems contributes to our perception of the world and of ourselves. There is growing consensus around the idea that a satisfactory account of multisensory processes will be integral to any comprehensive theory of perception [13], [22], [80]. Indeed, there are multiple examples of how interactions between the senses can have a dramatic impact on sensory and perceptual processes, both measured behaviourally as well as in terms of physiological correlates of neural activity [11], [101], [107]. Although the amount of research effort devoted to understand multisensory interactions has been remarkable over the last decade, the stress has been primarily placed on a few particular modality pairings (i.e., audio-visual being the most prominent one, possibly followed by visuo-tactile) and perceptual domains (i.e., spatial perception, speech perception, object recognition, and more recently, temporal perception). Here we focus on the specific case of auditory–tactile interactions in the perception of vibrotactile events. We think that this case is of particular interest in its own right because of the close relationship that exists between the senses of hearing and touch. In fact, it is surprising that despite the relatively good knowledge that currently exists regarding the physiology and psychophysics of vibrotactile processing at a unisensory level and the close relationship between the senses of hearing and touch (see next section), the multisensory aspects of vibrotactile perception have received little attention in the literature.

In our view, understanding the neuronal and cortical mechanisms underlying multisensory perception does not only require empirical observations of behavioural or neural responses, but also theoretical and computational analyses of the processes putatively involved. Computational models explicitly link neurophysiological and behavioural experimental observations by the construction and simulation of microscopic models based on local networks with large numbers of neurons and synapses that lead to the desired global behaviour of the whole system. Biophysically realistic microscopic models are expressed by a dynamical system due to the fact that processing does not operate in a completely feed-forward fashion, since recurrent feedback is also present and there are good grounds for supposing that these feedback connections have a functional role in most aspects of brain processing [28]. To understand information processing, we need to understand the dynamic contributions from feed-forward and feed-back processes over time, and this involves more than a one-sweep feed-forward computation [43], [67], [86]. Many recent examples in the neuroscience literature (e.g., working memory, attention, decision-making) advocate for a dynamical system approach [8], [18], [86], [113] that allows us to discover the neuronal computation underlying specific brain function by solving the neurodynamical inverse problem of finding the connectivity structure from which the measured neuronal correlates emerges.

In the present article, we will briefly review some relevant aspects about the perception of vibrotactile patterns in the senses of hearing and touch, with a special stress on studies addressing audio-tactile interactions. Then we focus on one particular methodological approach based on vibrotactile discrimination task which we have recently applied to address somatosensory perception using a combination of psychophysical and computational analysis tools. We argue that this type of methodological approach can be adopted to help address several important aspects of audio-tactile interactions and multisensory perception in general. Finally, we will point out several research questions which we think are relevant, and potentially addressable within the proposed context.