Mechatronics applied to auditory localisation for telepresence

Abstract

This paper describes the importance of the sense of hearing for immersive telepresence systems, and a fundamental approach to designing and implementing a binaural sensor system. This approach relies on knowledge of how the human auditory localisation mechanisms work to locate sounds in space, in order that a synthetic system can be derived. The most important of these mechanisms are explained. The Strathclyde auditory system is then described including ideas drawn from the literature to define a binaural approach. This novel system consists of a set of artificial pinnae and microphones mounted on a binocular stereo sensor platform capable of pan, tilt and roll which is slaved to a head tracking system. A set of preliminary experiments is described which test the performance of the system in practice, as a basis for further development.

Introduction

We define ‘transparent telepresence’ as the use of technologically mediated experience to enable a telepresence system operator at a home site to experience being fully present at a remote site. This implies that the intervening technology will be totally transparent to the user. A full system would therefore allow the user to experience the remote site via all five classical senses plus vestibular and kinaesthetic sensations. The creation of such a transparent system lies decades in the future. However at the ‘Transparent Telepresence Research Group’ (TTRG) within the Department of DMEM a telepresence system has already been developed [1] incorporating a mechatronic stereo camera platform, communication links and immersive stereopsis display equipment. To augment this system we are developing a binaural sound capability to greatly enhance the users’ sense of presence. The importance of hearing in contributing to a sense of presence is well documented. For example, in a study of suddenly deafened adults it was noted that they experienced a radically reduced sense of reality and presence in the world around them [2]. Relatively little attention has thus far been given in the literature to the potential of sound for telepresence compared with vision systems. Analogous work is, however, going on in virtual systems to enhance the sense of presence in computer generated environments and in cockpit information systems for military aircraft, to provide directional information to the pilot on approaching aircraft or missiles [3]. By incorporating binaural sound into the system we are able to enhance the level of realism generated by the remote environment and presented to the user. This relies on knowledge of the processes which humans use to localise and externalise sound [4]. This paper describes some of these mechanisms and some preliminary tests used on the mechatronic telepresence head.