research-article

Open Access

Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation

Authors:
Nicole Robson

School of Electronic Engineering and Computer Science, Queen Mary University of London, United Kingdom

School of Electronic Engineering and Computer Science, Queen Mary University of London, United Kingdom

0009-0006-3915-8480
View Profile

,
Andrew McPherson

Dyson School of Design Engineering, Imperial College London, United Kingdom

Dyson School of Design Engineering, Imperial College London, United Kingdom

0000-0001-9323-9069
View Profile

,
Nick Bryan-Kinns

Creative Computing Institute, University of the Arts London, United Kingdom

Creative Computing Institute, University of the Arts London, United Kingdom

0000-0002-1382-2914
View Profile

CHI '24: Proceedings of the CHI Conference on Human Factors in Computing SystemsMay 2024Article No.: 496Pages 1–14https://doi.org/10.1145/3613904.3642616

Published:11 May 2024Publication History

CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems

Pages 1–14

Abstract

Entanglement theories are well established in HCI discourse. These involve a commitment to view human experience in encounters with technology as relational and contingent, and research apparatuses as co-producers rather than passive observers of phenomena. In this paper, we argue that sound is the sensory modality best suited to the investigation of entanglements. Materialist theories of sound and listening guide both the design of a novel interactive sound installation and the methodological approach of a participant study exploring the experience of listening. We present a diffractive analysis whereby micro-phenomenological interview data is read with sonic theories, generating accounts that might otherwise remain mute: the temporal fluctuation and physical feeling of proximity in listener entanglements with sound, somatic intention setting, and plural interpretations of interactivity. Finally, we offer a series of provocations for HCI to embrace qualities of the sonic and consider epistemological positions grounded in other sense modalities.

1 INTRODUCTION

Reader, wherever you find yourself in this moment, we invite you to clap your hands together as emphatically as the social situation will allow. Listen attentively to the sound that is produced, and consider the questions: “what do you hear?” and “where?.”

We propose that this clap is not merely the sound of one hand hitting the other but the “action of two hands clapping sounding with everything else.” The clap that we tend to demarcate as a separate nameable noise is rather the “sound of everything vibrating in its agitation... the room, furniture, bodies, carpets, windows, doors, sounding indivisibly together” [92]. Furthermore, it does not have an audio reality out there in the material world. You might imagine it as a burst of energy that disturbs the air around and outwards from your hands in the moment of striking, but the sound that you hear exists only in your perception.

Examining the clap experience more closely, notice the directional character of your attention in this moment of listening. We imagine that it is not on the inward character of sensations - listening to the body - but stretches outward, beyond the body to meet that burst of sonic energy as it unfolds in time. And so your relationship as a listening subject to the clap is somewhat complex and interrelated. Arguably, listening is an intermediary field that dissolves the notions of distant and separate subjects and objects based on the sensory modality of vision [11, 39, 79].

Central to this paper is the proposal that the relational character of sound allows us to more readily imagine ourselves as entangled with a vibrational world in flux [9, 92]. We put this proposal to work by investigating the experiential character of an entangled subjectivity - which implies a subtle decentring of the human subjectivity - during listening encounters with an ultrasonic art installation, called Being With The Waves. The materiality of sound is harnessed in the installation’s design as an active nonhuman participant, determining interactivity without knowing anything about the behaviour of humans that encounter it.

Being With The Waves is conceived as a research probe, deployed in a lab-based participant study using the micro-phenomenological interview technique [65] to elicit the relational experience of listening. Explicitly adopting an epistemological stance grounded in the materiality of sound, our diffractive analysis of interview data reveals temporal, ephemeral, and mobile qualities of interactive experience, and emphasises the plural nature of listening. We argue that due to an underlying visual bias in HCI methods, these are felt qualities of interactive experience that would otherwise not be articulated. Finally, we offer a series of provocations for HCI to embrace the sonic and consider epistemological positions grounded in other sense modalities.

2 FOUNDATIONS

This paper describes research spanning areas of interaction design and sound studies research, which share a theoretical interest in entanglement theories [28] and the agential realism of feminist science scholar Karen Barad [1]. For HCI, these theories redefine interaction as co-produced by entangled agencies, and entail a paradigm shift from “user-centred” to “more-than-human” design [13, 36, 96]. Now well established in the field, entanglement theories challenge the positivist epistemological basis of traditional HCI methods. This paper represents a practical step to address these frictions, joining recent discussions at CHI in this direction [13, 96].

2.1 Agential Realism and the Co-Production of Knowledge

The radically relational ontology of feminist science scholar Karen Barad has been widely adopted by HCI practitioners and is central to this paper. Agential realism is a total rejection of representationalism. For Barad, bounded entities do not exist independently of their relations and so the primary ontological units of reality are not subjects and objects but phenomena that come to exist via intra-actions across agential cuts within human and material configurations. Experiences with technology may be described as the result of specific agential cuts; co-produced in the moment of intra-action according to the meeting of a singular human with a particular material configuration. Equally, the research methods and apparatuses that we employ to articulate those experiences such as survey, body mapping, talk-aloud, experience sampling, or phenomenological interview make particular cuts. It would be flawed to subscribe to Barad’s relational ontology as part of a design approach without also acknowledging these epistemological consequences.

For Barad, discursive practices (i.e. concepts, meanings, and the way that they are acted on and circulated within cultures) are also “ongoing material [re]configurings of the world. Discursive practices and material phenomena do not stand in a relationship of externality to each other; rather... matter and meaning are mutually articulated.” [1, p.151-152]. Therefore, data or knowledge about particular phenomena is co-produced by the research apparatus, which we can broadly define as a “material-discursive” assemblage. This agential realist stance runs counter to the positivist assumption “that experiments reveal the preexisting determinate nature of the entity being measured”: the so-called “commodity-fiction” of data [33].

2.1.1 Diffractive Analysis.

Barad adopts Haraway’s articulation of diffraction [35], as the basis for an analysis method that seeks to understand the world from within and is attuned to “differences that our knowledge-making practices make and the effects they have on the world” [1, p.72]. Diffraction refers to the way that waves interfere and combine as they overlap, and the bending and spreading out of waves upon meeting an obstruction [1, p.74]. Adopting diffraction as a metaphor for analysis entails an examination of interference patterns, i.e. how data is co-produced by the research apparatus. Mazzei frames this as a shift in qualitative methods that spreads thought in “unpredictable patterns producing different knowledge” [51].

Diffractive analysis is well documented in social science domains [43, 51, 85] and has been implemented by HCI researchers to examine difference by reading data through theoretical insights [8], different disciplinary perspectives [57, 97] and subjectivities [46]. Diffraction has also inspired slow and open-ended engagements with “lived data” in design [73].

2.2 Thinking with Sound

In response to the underlying visuocentrism of Western thought [39], sound studies scholars promote alternative ways of knowing through sound; not to replace the visual with a “counter-monopoly of the ear” [23, 83] but to disrupt the status quo and promote the senses as an “integrated network in ones relating to the world” [11, 23]. This is an extensive field of study and we focus here on work that extends Barad’s relational ontologies while seeking to account for sound’s materiality in its specificity ¹.

2.2.1 Sonic Materialism.

According to Christoph Cox, sound art is profoundly under-theorised. Contemporary cultural theory and criticism entrench the notion that art experience is mediated by the symbolic field. Such an approach is suited to account for the textual and the visual but fails to capture the nature of the sonic. Sound artists and installation artists in particular (such as those described in section 2.3) explore the materiality of sound in their work, which Cox claims, is fundamentally nonrepresentational: “How does one talk about such work, which has no fixed duration, is site-specific and sculptural, and is fundamentally concerned with sound as a physical, intensive force?” [15]. This might account for the lack of phenomenological studies focused on listening and the public’s experience of sound art, a shortcoming that is lamented by Keylin [44] and Semmerling [75].

Explicitly building on the relational ontologies and new materialism, Cox’s “sonic materialism” [14, 15] is intended as an alternative theoretical framework for the sonic arts, grounded in the materiality of sound². Sound is immersive, surrounding and passing through the body, which implies rethinking the “chasm” between subject and object, and the spatial, static, and parallel existence of “actants” imposed by a visual paradigm. Instead, Cox proposes an emphasis on “events instead of objects, flux instead of immobility, becomings instead of beings” [9].

2.2.2 Listening as Punctuation.

The philosopher Peter Szendy describes listening as an intermediary field. Like vision, listening implies distance and critical consideration but also implies a contact that has more to do with the sense of touch, a similarity also noted by Davis [17]. It is easy to conceive of a sound hitting the listener’s eardrum and penetrating inside them, but that listener also acts, they transform sound in their perception. For Szendy, listening is a kind of punctuation or articulation of sonic phenomena. It is “an action, a decision in every moment” [39, 84].

2.2.3 The In-Between Space of Listening.

The clap example that opened this paper demonstrated the relational character of sounding and listening. For Salomé Voegelin, listening is a “knowing of the moment... that involves the listener and the sound in a reciprocal inventive production” [90]. As an analogy, she draws on Merleau-Ponty’s account of “being-honeyed,” the sensation of grasping at honey, “which comes apart as soon as it has been given a particular shape” and simultaneously clutches the hands. Voegelin continues: “the honey can only be felt through my stickiness. It cannot be grasped as a remote object but comes to being in my honeyed-hands as a complex phenomenon of no certain shape but a demanding nature” [90]. The relational experience of listening, therefore, entails a shift from thinking about separate sounds and listeners to thinking in terms of “stickiness” and orienting to an in-between space in which the connection with sound is felt.

2.2.4 Plural Possibilites in Listening.

Sympathetic with materialist notions of sound, Voegelin associates listening with an openness, an inhabiting or being-with sound rather than attention to “solid structures and being able to recognise what can be heard” [9, 91]. Like the ephemeral materiality of sound, listenings are fragile and uncertain. For Voegelin, sound’s “invisible mobility” engages the listener in Merleau-Ponty’s “non-sense,” his “sensorial sense that comes from sensation rather than knowledge” [91]. According to Merleau-Ponty, perception involves a succession of “dis-illusions” i.e. apparitions that lead us to the real and demonstrate its fragility. Listening allows us to explore dis-illusions without resolution and instead remain within the plural experience of the heard as equal possibilities from the same world [91].

2.3 Entangled Agencies in Sonic Interaction

Visual bias is a critique that has already been levelled at HCI by the sub-field of Sonic Interaction Design (SID) [26]. SID researchers argued that an overemphasis on visual displays in computing was a constraining factor in the development of interactive systems [72]. Consider “auditory icons” [30] for example, whereby sounds simply step in for visual signifiers to convey symbolic information. Like proponents of sonic materialism, SID researchers are concerned with sound as an active medium, the specific characteristics of which may be harnessed to “enable novel phenomenological and social experiences with and through interactive technology” [26]. In this context, the sound installation format has been employed in HCI research to explore a variety of research goals relating to, for example, intimacy and embodiment [24], kinaesthetic awareness [25] and the effect of urban settings on listener interactions [27].

The research detailed in this paper is inspired by installation practices in the sound art tradition, whereby sounds are composed in space, allowing the listener to place sounds in their own time through their movement and behaviour in the installation [74]. Key figures in this paradigm are the artists Maryanne Amacher, Christina Kubisch, Max Neuhaus, Alvin Lucier, and Francisco López. All are artists who explore the materiality of sound, “its texture and temporal flow, its palpable effect on and affection by the materials through and against which it is transmitted” [14]. The site of installation is, therefore, an active determinant in the installation’s composition, and artists often work to include its acoustic response and material properties in the work [70].

Barad’s notion of intra-action entails an entangled and distributed view of agencies that is useful for demarcating the interactivity afforded by sound art installations. Intra-action avoids situating agency in either the listener or the installation as an artefact and emphasises that it is negotiated and co-produced in their encounter [1]. In the paradigm just described, sound art installations are “static” [12] in the sense that sound sources are fixed and do not objectively change as a result of listener actions. There are no sensors, mappings, or feedback loops, yet they may be experienced as interactive due to the close entanglement of listener and sonic phenomena [71].

In a 2003 alt.chi paper, Sha and Freed highlight sound design practices that blend computational techniques with the materiality of acoustic space [76]. Nicolas Collins’ installation piece Pea Soup, exemplifies this approach. Here, “a self-stabilizing network of analog circuitry nudges the pitch of audio feedback to a different resonant frequency every time the feedback starts to build,” producing “architectural melodies” that are influenced by any disturbances to air in the space [10]. Further examples of technological assemblies that harness the complex and nonlinear properties of audio may be found in Di Scipio’s ecosystemic interfaces [18] and the extensive field of feedback musicianship and instrument design [22, 50, 52, 53]. These practices produce instances of what Sha and Freed term “human matter” interaction and afford rich interactive experiences in which human action is entangled with the sonic. There is a temporal immediacy compared with digital tools and no “non-grammatical” action [76], with systems responsive to all human movement, on all scales.

3 BEING WITH THE WAVES: AN ULTRASONIC ART INSTALLATION

Please refer to the video figure for excerpts of the installation. Headphone listening is recommended to hear the same audio that was fed to the listener during their interaction.

Figure 1: Being With The Waves system design. On the emitter side, a multi-channel composition is shifted to the ultrasonic range before sounding in the space. On the receiver side, audio signals taken from microphones housed on the outside of headphone earcups are routed to a microcontroller, de-modulated back into audibility, and output to the headphones.

Being With The Waves is a sound installation intended to foreground the materiality of sound as a physical presence in the space and encourage nuanced and exploratory listening-based interaction. Employing ultrasound, the installation sounds acoustically but mostly inaudibly in the exhibition space, and a hidden multi-channel musical composition may only be heard via specially designed sets of headphones. The process of shifting sounds between the audible and ultrasonic range using amplitude modulation alters the spatial appearance of sound and exaggerates the Doppler effect, such that listeners hear sounds bend and shift, in a manner intimately connected with their bodily position, orientation, and movement in the installation space.

The installation is conceived as an artwork and research probe [40]; an instrument to be deployed in participant studies exploring the experience of listening to sound art. In Being With The Waves the presentation of sound is out of the ordinary. It is intended as a provocation to listen - to attend to the dimensions and attributes of sound, and notice the perceptual activity involved in doing so. Positioned as an artwork rather than a design or research product [42, 63], Being With The Waves is what the philosopher Alva Noë describes as a “strange tool”; a non-functional and alien artefact that disrupts our ordinary mode of being in the world [56]. Listeners, encountering such an installation for the first time, work to make sense of its strangeness, asking “what is it?” [56]. The activities of listening and sense-making are thereby brought into the foreground of experience to be questioned and explored in post-listening interviews.

3.1 Design Intentions

The design of Being With The Waves is motivated by the observation that, through the assembly of sound materials, playback technologies, and space, installations in the sound art tradition yield richly engaging and nuanced interactions due to the listener’s close entanglement with the phenomena of sound. Informed by an interview study probing the practices of sound artists [70], the design of Being With The Waves was guided by the following intentions:

(1)	To preserve the acoustic entanglement of body, sound, and space as the origin of interactivity, rather than programmed event-loops and mappings.
(2)	To augment this paradigm using digital technology as a mediating layer that transforms the appearance of sound.
(3)	To emphasise the spatiality of sound and its physical presence in the installation space.
(4)	To encourage close and attentive listening together with an active awareness of the body’s relationship to sound and its physicality.

Early in the design process, the idea of hiding sound such that listeners need to actively engage with the work to hear it at all, felt like a productive direction. This led to a consideration of ultrasound and experiments with amplitude modulation. It was also decided that speakers should not be visible, placed in a darkened space, or concealed behind other objects so that the spatial placement of sounds in the space must be discovered through the act of listening, rather than the visible presence of sound sources.

3.2 Implementation

Being With The Waves may be broadly conceived in terms of emitter and receiver side technologies. The installation system is described in Figure 1.

Figure 2: The modified headphone sets designed for Being With The Waves. Small electret microphones are mounted on the outside of headphone earcups. Ultrasonic signals are routed to an amplification circuit and Teensy microcontroller housed in the small black box. This is worn around the neck of listeners and includes an on/off switch and volume control knob.

3.2.1 Emitter.

The installation consists of audio playback into the space via a series of tweeters suited to the transmission of ultrasound. Eight individual audio tracks are composed in the audible range and arranged in time (rather than procedurally generated) using the digital audio workstation Ableton Live³. Each track is shifted into the ultrasonic range using the technique of amplitude modulation. This is implemented by a Max for Live device⁴ on each audio channel that multiplies the audio signal by a sine wave with a frequency of 20.5kHz, shifting the signal by this amount. Each modulated signal is output via an audio interface to an amplification circuit and emitted into the installation space by an individual tweeter.

3.2.2 Receiver.

The multi-channel composition sounds physically but inaudibly in the space. Listeners hear the installation by wearing specially designed sets of headphones, seen in Figure 2. Sound phenomena are captured by 10mm omnidirectional electret microphone capsules, housed on the outside of headphone earcups, approximating a binaural arrangement. Signals are amplified via a custom electrical circuit and routed to a Teensy 3.6 microcontroller where a piece of code applying the same amplitude modulation process reverses the effect, shifting sounds back into audibility before being output to the headphones.

3.3 Modulation Effects

Following de-modulation, the audio fidelity of the original multi-channel composition remains remarkably intact but is coloured by room acoustics, bodily presences, and the materiality of electronic components. The spatial appearance of sounds, however, is transformed. A feature of ultrasound is that it has a tighter directivity in its transmission than lower-frequency sounds. This means that the stereo image becomes unnaturally sensitive to subtle movements. Sounds are not apprehended directly by the naked ear but mediated by microphones housed on the outside of the headphone earcups. Listening through the microphones entails some flattening of the spatial dimension. Sounds appear closer, with a lack of depth.

In Being With The Waves, ultrasonic modulation has the unusual property of exaggerating the Doppler effect. The Doppler effect is a familiar process where the perceived frequency of a sound changes when the sound source moves toward or away from the listener, owing to the finite speed of sound in air [69]. In everyday life, it is often most noticeable with moving vehicles, where an approaching vehicle’s sound will be shifted upward in pitch, and a receding vehicle’s sound will be shifted downward. Doppler shift functions as a ratio in frequency, where each frequency component is multiplied by a factor that depends on the relative velocity of a source and listener. For technical details please refer to [71]. Due to both the high-frequency content of signals that are emitted into the space, and the process of amplitude de-modulation, the Doppler effect is exaggerated such that bodily movements are heard as pitch shifts and timbral distortions to the spatial composition. These effects are inextricably tied to the body and incredibly sensitive to movement. For emphasis, a listener’s movement does not alter the sonic composition emitted into the space. Effects are heard by individual listeners, determined by their physical movement and spatial orientation to sound sources.

3.4 Interaction Without Symbolic Representation

Being With The Waves is experienced as being richly interactive due to normally imperceptible acoustic effects becoming exaggerated through amplitude modulation. The materiality of sound is thus harnessed in the design as an active nonhuman participant, determining the installation’s interactivity without knowing anything about the behaviour of the humans that encounter it. The system design may be contrasted with interactive musical systems in which sensor data acquires symbolic meaning in the computer program and a system of mappings determine the relationship between human action and auditory feedback. Magnusson describes such tools as having such a high degree of symbolic pertinence that they become systems of knowledge and thinking in their own terms [49].

In Being With The Waves the technology involves no symbolic representation of human behaviour. Emitter and receiver side modulation occur separately and concurrently, i.e. one does not depend on the other in the way that inputs and outputs are dependent in the event-loop paradigm. Arguably, the microphones operate as sensors in the system. They do not take “measurements” corresponding to a Cartesian system of coordinates or sense listener movements from an outside perspective as a motion capture system might. Nordmoen notes the sensor’s entanglement in the observation of phenomena that it observes [58]. The microphones in the installation observe the meeting point of a listening body and the sounding space in its entire material configuration, a rich encoding of spatial and sonic relationships.

3.5 Sound Design

The process of composing for the installation entailed listening to different sound materials through the technology to understand their spatial appearance from a first-person perspective. It was found that simple sustained sounds such as sine tones or harmonic instrumental tones appear to fill the space as an ambient presence and are very sensitive to the Doppler effect; tiny sonic fluctuations may be appreciated via micro-movements and more prominent distortions to the sound are heard as a result of quicker movements over a larger area. By contrast, inharmonic, articulated, and transient-rich sounds have a well-defined spatial profile. Turning on the spot, the listener becomes aware of their changing orientation to sounds, but the system distorts their perception of distance. Speech, for example, sounds broadcast as though on the radio, but with the signal coming from a very particular, narrow location and perceived close to the ear rather than originating from a faraway source.

Compositional ideas that were not grounded in the experience of listening to the installation system, but of composing for audible multi-channel speaker setups simply did not translate. The process of composing, therefore, necessitated a materially-guided approach, whereby the sonic content is guided by the affordances and constraints of the installation, rather than an imposed conceptual or aesthetic idea [71]. The sensation of sound as a kind of fluid matter that moves intimately with listener movement due to the Doppler effect and suitability of the installation to speech and environmental sound recordings inspired the sound design and artistic concept for Being With The Waves. Excerpts from micro-phenomenological interviews exploring sensory experiences in and with the sea are interwoven with an immersive musical composition composed of drones, instrumental textures and field recordings. Different layers of the composition are emitted from different spatial locations in the installation.

4 EXPLORING THE RELATIONAL EXPERIENCE OF LISTENING

The following participant study was conceived to explore the questions: “How do listeners experience their entangled relationship with sound in the installation?” and “What do they do to listen?”. The focus of the study was deliberately broad and interview questions content-free to allow interviews to be guided by participants. The study received ethical approval and followed participant data handling and ethical guidelines outlined by Queen Mary University of London.

4.1 Participants

A call for participants was made from the first author’s social media accounts and interested parties were invited to complete an online form, which included basic demographic information and simple screening questions designed to limit participation to people who visit art galleries regularly and would visit an interactive or sound art exhibition. 10 participants (6 female, 4 male) took part in the study. All but P4 had experienced a sound installation before. All participants regularly engage with the relevant practices of sound art/ experimental music (P1, P2, P7, P8), musical performance (P5, P6, P9), or audio technologies (P3, P4). The selected participants therefore had the relevant expertise “to actively and knowingly speculate with us in our inquiry” in ways that we could not do alone [93].

4.2 Physical Set-Up

The study took place in a university lab, transformed to become a dark gallery space, with black curtains lining the perimeter. Eight ultrasonic speakers were spread throughout the space and hidden from view, either fixed to the ceiling truss or the curtains. The space was dimly lit by coloured lights, as shown in Figure 3. Every effort was taken to present the research lab as an exhibition space, or “hybrid” study location [38, 54], with minimal visual stimuli to suggest listening.

Figure 3: Being With The Waves installed at a university lab space

4.3 Procedure

4.3.1 Briefing.

Before the study, the first author met with each participant for a short online meeting. Participants were briefed about the general procedure of the study and engaged in a short micro-phenomenological practice interview that served to introduce the kinds of questions used and the sensation of coming into contact and describing pre-reflective aspects of subjective experience.

4.3.2 Listening Encounter.

Participants had individual encounters with the installation at scheduled times. Upon arrival, the participants were told quite simply that the object of the study is to listen to a sound installation artwork called Being With The Waves, that the installation is not audible to the naked ear, and that they need to wear special sets of headphones to listen to it. The listening apparatus was then introduced, and participants were shown how to adjust the volume. They were then invited to walk into the installation space and place the headphones over their ears when ready. The researcher asked them to read a curatorial text (see Appendix A) on the wall before proceeding into the space. They were invited simply to listen to the installation and stop whenever they felt ready, removing their headphones before leaving the space. In practice, participants listened for between 5 and 20 minutes. After 20 minutes the researcher asked the listener to return to allow enough time for the interview.

4.3.3 Interview.

Each participant was then taken to a separate space for an interview lasting around one hour. These interviews fell into two parts: a semi-structured interview (15-20 minutes) and micro-phenomenological interview (40-45 minutes).

4.3.4 Semi-Structured Interview.

The semi-structured interview was designed to capture meta-descriptions of the experience as a whole: the trajectory of the interaction over time; broad descriptions of what happened, listener intentions, and sensations. Each participant was asked the questions in Appendix B.1, diverging as necessary to probe the participant for more detail or follow a trail of thought that might lead to interesting new insights. Throughout this part of the interview, the first author noted experiential descriptions that were of particular interest in relation to the research questions.

4.3.5 Micro-Phenomenological Interview.

Micro-phenomenology (M-P) is an interview method for eliciting finely-grained accounts of short slices (5-60 seconds) of lived experience, including pre-reflective aspects that the interviewee may not have been consciously aware of at the time. M-P has been deployed in HCI research since 2001 [68], for example, to study sensory modalities [16, 25, 60, 61, 66] and relational characteristics of interactive experience such as interconnection [82] and intercorporeality [45].

We adhered to the M-P procedure as described in [65], following guidance provided during the training course that the first author completed in 2020. To articulate the moment(s) to focus the interview on, participants were asked: “Are there any moments in your experience that stand out as particularly salient or memorable?”. When interviewees had described an experience of the Doppler Effect or moments of close bodily relationship to sound, the interviewer revealed an interest in those moments. We did not wish to lead participants, so for participants who did not mention these aspects (P6, P8, P9), the M-P interview focused on aspects already articulated in the semi-structured interview, such as attentive listening and/ or the relationship of the listener to the spatial placement of sounds. To probe auditory sensations and bodily feelings we used questions suggested in the M-P training guide (Appendix B.2) and sought to deepen experiential descriptions by reformulating the participant’s description and then framing “how” questions. For example: “You describe the sound as ‘pouring in’, how does it pour in?”, or “How do you know that the sound is sensitive to your moment?”.

4.4 Data Preparation

All interview data was transcribed, according to the conventions described by Poland [67]. M-P interview data was prepared for analysis in the manner described in [89]. Interviewer questions and reformulations were removed and interviewee descriptions broken down into numbered utterances [89]), then re-organised to reflect the diachronic unfolding of the experience over time. In this way, the M-P interview transcript reads as a chronological description of the moment under study, with synchronic detail nested within.

4.5 Diffractive Analysis

The open and exploratory nature of the study led to a breadth of experiential descriptions including auditory perceptions, intentions, bodily behaviour, mental imagery, emotional states, thought processes, and semantic interpretations. We conducted a much narrower diffractive analysis to specifically examine listener experiences of entanglement with sound in the installation. Given the material-discursive co-production of phenomena by research apparatuses (section 2.1), we sought an alignment of analysis method and the phenomena under study: sonic experience.

Materialist theories of sound (section 2.2) were “plugged-in” to the interview data [43] producing an emergent series of readings, which we conceive as patterns of positive interference, where theory and experiential descriptions resonate. This led to a clustering of phenomenological descriptions around particular theories. Rather than abstracting away from the data itself via qualitative coding, we sought to harness the fine-grained detail produced by M-P interviews. Furthermore, we explicitly sought to set the disciplines of sound studies and interaction design into conversation with one another. Throughout the analysis, we were attentive to the “production of boundaries”: how as some experiential descriptions move into the foreground, others are excluded [1, p. 92-93]. Thinking with sound, we became aware of emergent differences concerning other accounts of entangled experience in HCI and taken-for-granted, invisible dynamics [46] of research methods. These wider reverberations are presented as provocations in section 5.2.

4.6 Findings

The diffractive analysis led to the following four clusters of experiential descriptions:

(1)	Feeling Connection: Reading the data with Voegelin/ Merleau-Ponty’s notion of “being-honeyed” surfaces data describing the felt quality of connection with sound as material.
(2)	Relational Flux: The mobile and temporal quality of sonic material leads to the identification of temporal instability of embodiment sensations.
(3)	Somatic Alignment: Szendy’s proposal that listeners “punctuate” sound with their listening surfaces descriptions of intention setting with bodily poses and movement qualities.
(4)	Expecting Sensors: Plugging the doubtfulness and ambiguity of listening (Voegelin) into the data surfaces ways in which listener expectations regarding the system design lead listeners to dwell in technological “dis-illusions.”

4.6.1 Feeling Connection.

As described in section 3.4, Being With The Waves harnesses the acoustic entanglement of a listener with sonic phenomena as the root of interactive effects. The computer has no symbolic representation for human input and no mapping space separates behaviour from sound generation. Instead, the bodily presence and movement of the listener and sound as heard through the system are co-dependent, generated simultaneously, and as such there is no perceptible latency in the system⁵. As a result, sound is perceived as being intimately related to bodily behaviour. P7 describes the experience of sound as “tactile in a sort of keen aesthetic sense, that the feeling in your body as you move then kind of maps experientially onto the sound.”

Plugging Merleau-Ponty’s notion of “being-honeyed” (section 2.2.3) into the interview data amplifies phenomenological descriptions that express felt qualities of listeners’ close connection with sound as a material. One such quality is that of resistance, which ordinarily describes a physical feeling at the meeting point of a body and matter of some kind. P3 describes the feeling of a force pulling them back: it “feels like I’m kind of walking against something, like there’s a force that’s pulling me back or trying to slow me down.” But, of course, there is no force physically acting on the body to resist its movement, there is just the experience of sound, a nuance that P3 is keen to express: “it’s just a feeling.. that’s how I felt it’s not like it is actually doing anything.” P2 describes the drone sound as “like a cloud a few hours from raining, like it’s building up with humidity inside... There’s something fluid about it but it’s still airy.” In moments of movement interaction, they describe this sound as having a viscosity “because when I turned my head at other moments I felt the sound pour in at the same time as me sort of turning and opening up to it.” For P2, “viscosity” describes a felt quality of interaction, related (we infer) to a rate of sonic change, and felt sense of materiality, as a texture with density.

Describing the sensation of interaction with the Doppler effect, P1 and P2 describe variation in the sound according to movement properties associated with physical objects, that exhibit temporal variation as a result of their form and material. P2 describes the instability of sound as a jelly-like quivering, while P1 makes an analogy with toys that can’t be knocked over and instead oscillate around a central position, gradually reaching a standstill. For them, the sound “felt as if it was spinning within itself, and then when I stopped moving it sort of went back to its original tone.” Importantly, they also describe a close connection to the movement of the sound. P1 feels it as located within the top of their head, “as if I was dizzy, sort of being spun with the sound. It doesn’t make me feel dizzy but I think it just feels like it’s there,” while P2 describes feeling closely bound to the jelly sound-movement sensation, stating “I am that jelly.”

4.6.2 Relational Flux.

Reading the data with Cox’s sonic materialism (section 2.2.1) foregrounds the temporal dynamics of relational experience. We tend to think of technology relationships such as Ihde’s [41] as fixed and immobile. In the experience of interaction with Being With The Waves, listeners describe close bodily connections with sound that have underlying temporal fluctuation, varying in intensity and location relative to the body.

P7 describes whirling and spinning through the installation space. They also describe having a close connection with sound, in that it is experienced as both inside and outside the body, or rather “there’s not really such a big distinction.” This intimate relational quality is not fixed or static, it moves and is in flux, in a way that is tied to the speed and shape of their movement. The sound is “like a figure of eight, it kind of whips through you and then around you.” As they whirl through the installation, P7 says that “the sound is just undulating and modulating its pitch with every whirl and there’s a kind of consistency in my body sort of creating a rhythmic intensity within that drone sound.” Probed further about the quality of this rhythm, P7 describes it as “internal and it’s like something that is generated from within,” rather than something imposed on the body from outside. Rhythm, here, appears to capture a felt quality of the interactive relationship rather than a musical quality. Unintended, it is an embodied and emergent sensation at the meeting point of (moving) listener and external field of vibration.

The Doppler effect was felt most strongly during periods when the installation was dominated by a drone; at times more pure in sound and at others more textural, composed of sustained violin notes. A drone is sustained over long periods, it has a stable presence in time. However, several participants (P1, P3, P9, P10) invoke the word “infinite” to describe a spatial presence, as immense, filling the space: “I felt like it’s an infinite field of these two tones” (P3). In stillness, the drone is heard as continuous, but bodily movement leads to pitch bends and timbral distortions as a result of the Doppler Effect. In movement, the sound is no longer infinite for P1: “It doesn’t sound infinite because... when I felt as if I was triggering the spinning I was able to stop it so it almost felt as if it did have an end.” P1 moves from describing the sound as heavy and solid to having a spinning sound associated with the movement quality of toys (described earlier), with perceived size in the range of handheld objects. Through their movement interaction with the installation, sound comes closer into the body and is experienced as mobile and light, rather than static and heavy. Sound appears to shape-shift and the embodied relationship of listener to sound installation is in flux.

4.6.3 Body-Sound Alignment.

Reading the data with Szendy’s notion of listening as an articulation or punctuation of the sound (section 2.2.2) highlights accounts by participants (P2, P5, P7, P8) that describe the adoption of particular bodily postures that are simultaneously inspired by the sound of the installation and adopted to express a particular listening intention or sense. These dispositions cannot neatly be described as either passive or active and instead, are suggestive of an intention to align the body with sound.

Reflecting on their experience as a whole, P2 describes a journey of figuring out how they can work in harmony with sound. They describe holding their body in a controlled way: “I really wanted to be strong and grounded, so that I could make those deliberate footsteps and movements. I didn’t want to flail about in there.” By calming their movements they can give more of their body to the act of listening. This somatic disposition toward sound is echoed by P5, who describes a moment of bodily poise in which the music was supporting their physicality, rather than the experience being about their body in relation to the music, “so it was like a switch of emphasis or switch of relationship.” The sounds of the installation are continuous at this point and they express a sense of “something being sustained through it and the pose had that potential for stillness and yet movement... It had a constant alertness and smoothness around it like the music did.”

P8 describes sitting in a meditative pose, breathing deeply, and attempting to let go “of being a separate entity.” They describe feeling their body as part of a wider vibrational plane: “it felt like the space around me was also inside with the resonance... physically it’s like tiny particles moving in my lung when I breathe in.” The body posture is adopted to focus on listening, but it is the sound that causes the sensation. P5 and P7 both describe the intention to become fluid with their bodily movements, to match the perceived fluidity of sound. P5 notes how this bodily disposition changes their listening: “when I was imagining my body being light or fluid whatever then you start listening in a different way.. so what I was doing with my body did affect my... listening sense.” In their attempts to make their body fluid like water - “I turned my body into an eddy of water” - P7 notices how their body is solid, how their joints can’t move fluidly: “there’s this whole feeling of resistance in my joints- that I can’t be a perfect fluid- because I’ve got these joints and my sort of irritating physical form that’s inside.”

4.6.4 Expecting Sensors.

The speaker array in Being With The Waves is hidden from view, the microphones on the outside of headphones are so tiny as to go unnoticed and the Teensy microcontroller along with amplification circuitry are concealed inside a box. Participants have no prior knowledge of the system design with respect to the configuration of technologies that produce interactive effects. As described in section 4.1, participants were recruited as co-speculators and experienced listeners to sound and interactive installations, many artists and technicians themselves. Participants describe attempts to make sense of sound’s organisation in the space and how the installation works, making hypotheses regarding the technologies involved. Their descriptions reveal an assumption that interactivity is configured via the mapping of sensor data to a sound engine, with changing auditory feedback output to the listener’s headphones. In short, listeners assume that there is a computational process at work that has a symbolic representation of listener movement in the space.

Like many listeners, P4 describes repeatedly moving and stopping to “test” the interaction effects: “testing the length of pauses to check it wasn’t a coincidence and feeling actually it was almost too sensitive. So almost as if I was imagining starting to go left the juddering would start. So I thought ‘that’s too sensitive, so it can’t be that’ and tried it again and it happened half a dozen times at least.” There is an expectation that interactivity effects must be coupled with some latency for computation to occur.

P2, P3, and P10 speculate that the installation involves movement sensing; either pressure sensors in the floor, an accelerometer in the box worn around the neck, infra-red distance sensors, or a camera-based motion capture system mounted on the ceiling. Reading the interview data with Merleau-Ponty’s notion of dis-illusions as the root of experience, and Voegelin’s proposal that in listening we might more readily dwell in these alternative possible worlds (section 2.2.4) highlights ways in which assumptions about technology influence listening behaviour and experience. P2 makes a cognitive connection between the rate of change to the sound and the movement quality of their footsteps. This leads them to believe there are pressure sensors in the floor. They describe listening carefully with this connection in mind and “trying to put that sound that was in my ears in my foot... I was trying to imagine them... not like that’s my foot and that’s my ear, that’s coming through here and that’s doing that, it’s more like, okay this is one unit and how does this whole thing work together’.” Similarly, P10 believes there to be a matrix of sensors above them detecting their position and movement in the space. This leads them to associate their body movement and the bending sound of the Doppler effect with the position of imaginary sensors. These associations of movement and sound with particular spatial locations surely transform the listening experience. Both are phenomenological dis-illusions and demonstrate the possibility for plural interpretations of the installation.

5 DISCUSSION

The experience of listening is ephemeral and elusive. Human experience is complex, layered, and diffuse. We find this sonic framework helping for grounding the analysis of phenomenological descriptions in dimensions of experience that are most salient to the materiality of sound: temporality, spatiality, mobility, and plurality.

5.1 Micro-Phenomenology and Sonic Experience

Research is often generative, new aspects and ideas come to light as a result of renewed engagement with the subject at hand. In our case, the participant study was designed and carried out long before the analysis approach and central ideas in this paper came to be. Careful consideration was given to the method for articulating listening experiences and we stand by our selection of micro-phenomenology (M-P) as the optimal fit. When applied rigorously, the method is unparalleled in its capacity to reveal rich experiential descriptions, the significance of which is yet to be fully explored within the field of HCI. However, via our diffractive engagement with theory we noted some frictions between M-P and our orientation to the sonic, a slight epistemological mis-match. As such, the temporality of sonic experience does not flow out of the data.

O’Callaghan draws attention to a more or less implicit assumption in the history of philosophical work on perception, “that what we learn about perception by studying vision generalizes to the other sense modalities” [62]. M-P tends to treat sensations as objects rather than events, which in the case of sound is problematic. To probe the synchronic details of an experience, the M-P interviewer works to stabilise the interviewees evocation on a past moment, thereby freezing time. While visual sensations and mental images are readily stabilised in this way, sounds, which are inherently temporal - are not. It may be possible to repeatedly iterate through a short period, but not to freeze a sound in time. The M-P interview questions suggested in training material (see Appendix B.2) elicit the details of a sonic sensation by asking participants to describe its object qualities, such as direction, location, size, loudness, tone, persistence. We found these questions productive in that they provided a shared language within which to talk about sound in phenomenological terms, without the need to use technical or musical jargon. However, these questions do tend to treat sonic sensations as objects, not events. Therefore descriptions do not readily account for the temporal fluctuation of sounds, or indeed any other sensations, on a micro gestural level. Detail, which in the case of our study would be beneficial to capture more readily.

Of the M-P questions to probe sonic sensation, we found the question “does this sound come from somewhere?” productive as it situated the sonic sensation in relation to the body and revealed, quite organically, the perceived closeness of sound, along with descriptions of size and other felt qualities including the temporal fluctuation of sonic sensation. As a result, we could approach the question of what it feels like to be entangled, with a level of descriptive detail that would not have been obtained by using traditional qualitative interview methods. The identification of possible frictions is a constructive finding as it allows us to consider how we might tailor and improve the M-P framework to account for more fluid, rather than discrete events and interview questions to most effectively study a variety of phenomena including an experience of sound.

How we might capture temporal fluctuation with the M-P framework requires close consideration, because the stabilisation of attention on a singular moment of past experience is foundational to the method. We plan to seek advice from expert practitioners to examine this issue and also introduce sound art practitioners to the M-P technique; engaging them in ongoing discussion and experimentation with the method to frame questions that target the sonic, such as “how tightly bound do you feel to that sound?,” “does that sensation fluctuate?,” “how stable is it?.”

5.2 Reverberations and Provocations

“The case of sounds and audition demonstrates that attention to modalities other than vision enriches our understanding of perception and its role in situating oneself, forming beliefs, and acting upon the environment” - O’Callaghan [62].

Throughout this paper, we have been promoting the sonic as an alternative orientation to the visual, one that might reveal alternative kinds of knowledge about our experiences with technologies. By considering an alternative approach, the (often tacit and unchallenged) ontological assumptions and methodological infrastructures that govern practices in HCI-related fields begin to surface, and become perceptible. We champion the sonic here as a provocation to open up new ways of thinking about the experience of interaction and offer the following provocations for HCI:

Design for an Entangled Subjectivity

Being With The Waves is designed to encourage close, attentive listening together with a keen awareness of bodily entanglement with the installation. We describe this disposition as an entangled subjectivity. It is marked by an attentional disposition that is directed outward to meet the nonhuman world and an ongoing awareness of relationality. Derived from theories of listening (section 2.2), this subjectivity subtly decentres the human in the sense that it is neither an introspective mode (examining one’s own thoughts, feelings, bodily sensations), nor an extrospective mode (examining what is outside oneself). Instead, it is a hybrid perspective. Still rooted in lived experience, an entangled subjectivity examines how the ongoing phenomena of experience are co-constituted by our bodily engagement with a material assemblage. For example, participants in our study attended to feelings of close connection with sound such as viscosity, which relies equally on qualities of the material and the character of human behaviour (section 4.6.1).

In HCI and design, many different approaches to decentring human subjectivity have been described such as conducting ethnography with nonhumans [32], noticing differently [47] and designing for nonhuman users [31]. Nicenboim et al. raise the important question of “how do we cope with the impossibility of accessing nonhuman perspectives really, given our inherent human assumptions and ways of knowing?” [55]. Our approach is still interested in designing for humans while also sharing the stage with other things. We propose the concept of entangled subjectivity as a slight deviation from modes of bodily knowing [59] and a useful design sensitivity in the context of somatic [37] and experience-centred design [94, 95] and their intersection with entanglement theory (e.g. [45, 88]).

Sense Less, Know Less, Quantify Less

As the art material in Being With The Waves - i.e. that which is experienced by listeners via the headphones - sound cannot be reduced to qualities of the musical composition nor its physical sounding in the installation space above the range of human hearing. Instead, the material of sound is intra-acted: co-produced in the entanglement of a listener with the installation’s technologies, and it is due to this co-constitution of sound that listeners perceive interactive qualities and sensations of intimacy and connection. The installation, therefore, foregrounds a relational notion of sound as a kind of “interstitial matter”, that which is in-between different media and bodies involved in its production [19].

Our findings regarding the expectation of sensors, dialogic relationships and computational latency (section 4.6.4), highlight the assumption that an interactive device or artwork has a symbolic representation for human behaviour. The prevalence of the event-loop paradigm may be constraining both the design and experience of new technologies. The expectation of sensors foregrounds a cognitive mode of engagement whereby a user works to make sense of a system through their interaction with it. By harnessing the materiality of sound as an active determinant in the configuring of interaction, Being With The Waves demonstrates an alternative design ideology that Sha termed “human matter interaction” in 2013 [76], the full potential of which is far from being realised. The installation’s technologies co-construct sound in a nonrepresentational way: without inspecting and processing data about human behaviour. It is our conviction that this approach promotes interactive experience that is richly multi-dimensional, nuanced, and visceral, addressing ongoing embodied thought [77] and reflection [4]. Therefore, moving against the tide of computational complexity and advancement we offer the provocation to sense less, understand less, quantify less, and move away from the notion that the machine needs to encode a neat symbolic representation that humans understand on similar terms.

Assume Dynamism

For Cobussen, sonic materialism entails “a true paradigm shift in how Western philosophy and thinking in general should consider the world: not as stable, filled with independently perceivable objects, separated from the perceiving subject, but as a world in flux, an eternal becoming in which subjects (also eternally becoming) are immersed” [9]. Interactions with technology are rarely static and yet temporality is often overlooked in the articulation of user experience. We encourage HCI researchers to assume dynamism and flux rather than fixed properties and to consider the temporality of interaction on all scales from the macro trajectory or narrative arc of an interaction [3, 86] to micro details. As we found in the phenomenological descriptions grouped under Relational Flux (section 4.6.2), felt sensations might have rhythm or local micro fluctuation, shape-shift, drift in and out of awareness, and vary in intensity or stability. A key challenge lies in the articulation and development of research methods that allow these aspects to surface.

Attend to Sensory Difference

Study participants encountered the same assemblage of sound and technological mediations but enacted wildly diverse listenings (section 4.6.4). Perceptions of the installation do not neatly mirror the stimulus, but diffract it, leading to marked differences in the felt experience of listeners. Spiel has highlighted the prevalence of normative body conceptions in design research [80] and the need for embodied computing to account for diverse bodyminds, differences in “felt experience” [80] and neural difference [81]. These ideas are echoed by Drever in their writing on auraldiversity [20, 21] and are calls that we wish to amplify here along with a provocation to consider sensory impairment in design practices. By sensory impairment, we mean a continuum of difference, from the fluctuation to sensory acuity that we experience on a day-to-day basis, to sensory loss and more debilitating conditions. During early public presentations of Being With The Waves we witnessed anxiety around noise levels in the case of listeners with hyperacusis (which involves a reduced tolerance to sound) and the tentative wearing of headphones by hearing aid users. Our observations drew attention to the fact that we hadn’t placed diverse listening needs front and centre in the design process. They highlighted the need for accessible control of volume and muting, and the careful mixing of audio elements in the composition to inspire confidence in diverse listeners.

Adopt Multi-Modal Epistemology

Interactive technologies rarely address the modality of vision alone, and more often address multi-modal perception. We are not suggesting that a sonic orientation would be appropriate for examining interaction broadly. Rather, that methods produce data [33] and should not be uncritically adopted. In fields such as Soma Design whereby the data of experience explicitly shapes design practice [37], this issue is perhaps even more acute. In this paper, we have demonstrated that epistemologies derived from other sensory modalities may reveal alternative characteristics of interactive experience. Besides the sonic, it may be productive for HCI researchers to explore epistemologies grounded in haptic touch [29] or the kinetic intelligence of movement [78]. Ultimately, however, working towards an integration of all sensory knowledge.

5.3 Comments on the Research Apparatus

In our installation, study design, and analysis method we chose to make particular agential cuts, which determine the scope and content of our findings. For example, paratextual elements of the installation such as its title, curatorial text and musical symbolism inevitably shape the experience of listeners in some manner. It is possible that when describing sensations of resistance or viscosity - qualities ordinarily associated with liquids - listeners are influenced by aquatic references in the work. By making this point we are not suggesting that the data is distorted or its validity undermined, but rather that these verbal descriptions are inseparable from the singular installation assemblage. We do not expect our findings to describe the experience of sound installation art in general, but we are interested in how design elements come to shape experience. Therefore, the question arises: where broadly similar interactivity is encountered without those aquatic cues, is the feel of interaction articulated by listeners with similar language and metaphor use? A future study will remove explicit references to water to examine the extent to which they inform experiential descriptions and understand how the installation as an apparatus shapes the experiences of listeners.

5.4 Conclusion

Contributing to the fields of posthuman and more-than-human design [13], this paper offers a practical implementation of entanglement theories and the agential realism of Karen Barad in particular [1], seeking consistency between epistemological commitments, design intentions, and qualitative research methods. With the ultrasonic installation Being With The Waves we worked to create a richly engaging listening experience while remaining sensitive to the nonhuman influence of sound. We decentred human agency in the design process by working with and harnessing sound’s materiality, and in the articulation of listening experiences by targeting an entangled subjectivity. Our findings highlight relational, temporal, and mobile qualities of interactive experience, and emphasise the plural nature of listening. Broader questions such as “how does the inaudible decentre human subjectivity?” and “who or what can listen to the installation?” fall outside the scope of this paper, but point to the wider implication that sound as a material can lead to new ways of thinking about interaction design and entanglements with technology [6, 7]. Further inspiration might be found in the activities of deep listening, which involves “expanding to the whole field of sound while finding focus” as a way to connect and attune with the environment [48, 64], and field recording through electromagnetic, seismic, and underwater microphones to enact more-than-human engagements [2].

ACKNOWLEDGMENTS

This research was supported by the EPSRC grant EP/L01632X/1 (Centre for Doctoral Training in Media and Arts Technology) and by the Royal Academy of Engineering under the Research Chairs and Senior Research Fellowships scheme. We wish to thank our study participants for their time and thoughtful contributions.

A CURATORIAL TEXT

Explore sound as a fluid presence that fills the space around you and discover stories of seaborne experience woven into an immersive composition that bends and shifts with your movement. To the naked ear, the installation appears silent, but a hidden world of voices, instrumental tones, and maritime sounds and is revealed through wearing special headphones.

B INTERVIEW QUESTIONS

B.1 Semi-Structured Interview

(1)	In broad strokes, could you tell me the story of your experience, listening to the installation? If it makes sense for you to do so, then you could start at the beginning when you first entered the space, and then describe was happened over time.
(2)	What did you do to listen to the installation? Probe - physical actions, listening strategies, physical and perceptual behaviours that happened over time.
(3)	How did you feel while listening to the installation? Probe - emotions, bodily sensations, mental images.
(4)	Examining your own experience, are there ways in which the experience of listening is similar/ different to everyday listening situations? Or, an alternate framing of this question: How would you describe the sensation of listening to the installation to somebody unfamiliar with it?
(5)	Did you notice the listening technology?
(6)	Are there any moments in your experience that stand out as particularly salient or memorable?

B.2 Micro-Phenomenological Interview

Example interview questions for describing an auditory sensation:

•	And when you hear x, what do you hear?
•	Does this sound come from somewhere? From how far away? From which direction?
•	How loud is it?
•	What is its tone?
•	How persistent is it?

Example interview questions for describing a bodily feeling:

•	And when you feel this, what do you feel?
•	Where is this sensation? Is it located somewhere?
•	Does it have a size?
•	How intense is it?
•	If I had this feeling, what would I feel?

Footnotes

¹ We expect these ideas may be unfamiliar to an HCI audience, and recommend the podcast episode Thinking (Through) The Ear [39] as an entry-point.
Footnote
² It should be noted that Cox’s ideas are a topic of debate [5, 34, 87], criticisms that have been refuted by Cox in [15]. The nuances of discourse are beyond the scope of this review but important to acknowledge.
Footnote
³ Ableton Live is a software sequencer for producing, arranging, mixing and performing with audio: https://www.ableton.com/
Footnote
⁴ Max for Live devices integrate the functionality of the visual programming language Max, within Ableton Live, allowing users to create custom audio effects and instruments: https://www.ableton.com/en/live/max-for-live/
Footnote
⁵ Ring (de-)modulation by 20.5kHz shouldn’t produce any more than a few tens of microseconds of latency, but there will be some minimal latency in the audio codec and buffering of the Teensy.
Footnote

Supplemental Material

Video Presentation

mp4

61.7 MB

Download

Video Figure

A listener is seen experiencing the ultrasonic art installation 'Being With The Waves'. Headphone listening is recommended to hear the same audio that was heard during their interaction.

mp4

10.4 MB

Download

Available for Download

vtt

3613904.3642616-talk-video.vtt (16.4 KB)

References

Karen Barad. 2007. Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning. Duke University Press, Durham & London.Google ScholarCross Ref
Navigate to
Reference 1
Reference 2
Reference 3
Reference 4
Reference 5
Reference 6
Reference 7
Leah Barclay. 2019. Acoustic ecology and ecological sound art: Listening to changing ecosystems. Sound, Media, Ecology (2019), 153–177. https://doi.org/10.1007/978-3-030-16569-7_8Google ScholarCross Ref
Reference
Steve Benford, Gabriella Giannachi, Boriana Koleva, and Tom Rodden. 2009. From interaction to trajectories: designing coherent journeys through user experiences. In Proceedings of the SIGCHI conference on human factors in computing systems (CHI’09). 709–718. https://doi.org/10.1145/1518701.1518812Google ScholarDigital Library
Reference
Camille Buttingsrud. 2021. Bodies in skilled performance: how dancers reflect through the living body. Synthese 199, 3-4 (2021), 7535–7554. https://doi.org/10.1007/s11229-021-03127-2Google ScholarCross Ref
Reference
Iain Campbell. 2020. Sound’s Matter:‘Deleuzian Sound Studies’ and the Problems of Sonic Materialism. Contemporary Music Review 39, 5 (2020), 618–637. https://doi.org/10.1080/07494467.2020.1852804Google ScholarCross Ref
David Cecchetto. 2013. Humanesis: sound and technological posthumanism. University of Minnesota Press.Google Scholar
Reference
Amy M Cimini. 2015. Posthuman sounds: Humanesis: sound and technological posthumanism, by David Cecchetto. Sound Studies1:1 (2015), 180–183. https://doi.org/10.1080/20551940.2015.1079977Google ScholarCross Ref
Reference
Marianne I Clark and Holly Thorpe. 2020. Towards diffractive ways of knowing women’s moving bodies: A Baradian experiment with the fitbit–motherhood entanglement. Sociology of Sport Journal 37, 1 (2020), 12–26. https://doi.org/10.1123/ssj.2018-0173Google ScholarCross Ref
Reference
Marcel Cobussen. 2022. The sonic turn: Toward a sounding sonic materialism. New Sound International Journal of Music 60, II (2022), 11–24. https://doi.org/10.5937/newso2260011CGoogle ScholarCross Ref
Navigate to
Reference 1
Reference 2
Reference 3
Reference 4
Nicolas Collins. 2021. Improvising with architecture: Pea soup and related work with audio feedback. Resonance: The Journal of Sound and Culture 2, 2 (2021), 168–181. https://doi.org/10.1525/res.2021.2.2.168Google ScholarCross Ref
Reference
Marcel Corbussen, Holger Schulze, and Vincent Meelberg. 2013. Editorial: Towards New Sonic Epistemologies. Journal of Sonic Studies 4, 1 (2013). https://www.researchcatalogue.net/view/266013/266014Google Scholar
Reference 1Reference 2
Stroud Cornock and Ernest Edmonds. 1973. The creative process where the artist is amplified or superseded by the computer. Leonardo 6, 1 (1973), 11–16. https://doi.org/10.2307/1572419Google ScholarCross Ref
Reference
Aykut Coskun, Nazli Cila, Iohanna Nicenboim, Christopher Frauenberger, Ron Wakkary, Marc Hassenzahl, Clara Mancini, Elisa Giaccardi, and Laura Forlano. 2022. More-than-human Concepts, Methodologies, and Practices in HCI. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(CHI EA ’22). 1–5. https://doi.org/10.1145/3491101.3516503Google ScholarDigital Library
Reference 1Reference 2Reference 3
Christoph Cox. 2011. Beyond representation and signification: Toward a sonic materialism. Journal of visual culture 10, 2 (2011), 145–161. https://doi.org/10.1177/1470412911402880Google ScholarCross Ref
Reference 1Reference 2
Christoph Cox. 2018. Sonic flux: sound, art, and metaphysics. University of Chicago Press.Google Scholar
Reference 1Reference 2
Tor-Salve Dalsgaard, Joanna Bergström, Marianna Obrist, and Kasper Hornbæk. 2022. A user-derived mapping for mid-air haptic experiences. International Journal of Human-Computer Studies 168 (2022), 102920. https://doi.org/10.1016/j.ijhcs.2022.102920Google ScholarCross Ref
Reference
Tom Davis. 2010. On Listening to Installation. Performance Research 15, 3 (2010), 66–71. https://doi.org/10.1080/13528165.2010.527207Google ScholarCross Ref
Reference
Agostino Di Scipio. 2003. ‘Sound is the interface’: from interactive to ecosystemic signal processing. Organised Sound 8, 3 (2003), 269–277. https://doi.org/10.1017/S1355771803000244Google ScholarDigital Library
Reference
Luc Döbereiner. 2020. Materiality, Contingency and Emergence of Compositional Material. Contemporary Music Review 39, 5 (2020), 602–617. https://doi.org/10.1080/07494467.2020.1852803Google ScholarCross Ref
Reference
John Levack Drever. 2019. ‘Primacy of the Ear’–But Whose Ear?: The case for auraldiversity in sonic arts practice and discourse. Organised Sound 24, 1 (2019), 85–95. https://doi.org/10.1017/S1355771819000086Google ScholarCross Ref
Reference
John L Drever and Andrew Hugill. 2022. Aural Diversity. Taylor & Francis.Google Scholar
Reference
Alice Eldridge, Chris Kiefer, Dan Overholt, and Halldor Ulfarsson. 2021. Self-resonating Vibrotactile Feedback Instruments ||: Making, Playing, Conceptualising :||. In Proceedings of the 2021 New Interfaces for Musical Expression conference(NIME’21). https://doi.org/10.21428/92fbeb44.1f29a09eGoogle ScholarCross Ref
Reference
Veit Erlmann. 2020. But what of the ethnographic ear? Anthropology, sound, and the senses. In Hearing Cultures. Routledge, 1–20. https://doi.org/10.4324/9781003103189-1/Google ScholarCross Ref
Reference 1Reference 2
Sidney Fels. 2000. Intimacy and embodiment: implications for art and technology. In Proceedings of the 2000 ACM workshops on Multimedia(MULTIMEDIA’00). 13–16. https://doi.org/10.1145/357744.357749Google ScholarDigital Library
Reference
Jules Françoise, Yves Candau, Sarah Fdili Alaoui, and Thecla Schiphorst. 2017. Designing for kinesthetic awareness: Revealing user experiences through second-person inquiry. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI’17). 5171–5183. https://doi.org/10.1145/3025453.3025714Google ScholarDigital Library
Reference 1Reference 2
Karmen Franinovic and Stefania Serafin. 2013. Sonic interaction design. MIT Press.Google ScholarDigital Library
Reference 1Reference 2
Karmen Franinovic and Yon Visell. 2007. New musical interfaces in context: sonic interaction design in the urban setting. In Proceedings of the 7th international conference on New Interfaces for Musical Expression(NIME’07). 191–196. https://doi.org/10.1145/1279740.1279776Google ScholarDigital Library
Reference
Christopher Frauenberger. 2019. Entanglement HCI the next wave?ACM Transactions on Computer-Human Interaction (TOCHI) 27, 1 (2019), 1–27. https://doi.org/10.1145/3364998Google ScholarDigital Library
Reference
Matthew Fulkerson. 2020. Sensory Interactions and the Epistemology of Haptic Touch. In The Epistemology of Non-Visual Perception. Oxford University Press. https://doi.org/10.1093/oso/9780190648916.003.0003Google ScholarCross Ref
Reference
William W Gaver. 1993. Synthesizing auditory icons. In Proceedings of the INTERACT’93 and CHI’93 conference on Human factors in computing systems. 228–235. https://doi.org/10.1145/169059.169184Google ScholarDigital Library
Reference
Maliheh Ghajargar and Jeffrey Bardzell. 2022. Learning About Plant Intelligence from a Flying Plum Tree: Music Recommenders and Posthuman User Experiences. In Proceedings of the 25th International Academic Mindtrek Conference. 343–346. https://doi.org/10.1145/3569219.3569388Google ScholarDigital Library
Reference
Elisa Giaccardi, Nazli Cila, Chris Speed, and Melissa Caldwell. 2016. Thing ethnography: Doing design research with non-humans. In Proceedings of the 2016 ACM conference on designing interactive systems(DIS’16). 377–387. https://doi.org/10.1145/2901790.2901905Google ScholarDigital Library
Reference
Lisa Gitelman. 2013. "Raw Data" Is an Oxymoron. The MIT Press.Google Scholar
Reference 1Reference 2
Annie Goh. 2017. Sounding situated knowledges: echo in archaeoacoustics. Parallax 23, 3 (2017), 283–304. https://doi.org/10.1080/13534645.2017.1339968Google ScholarCross Ref
Donna Haraway. 1991. The promises of monsters: A regenerative politics for inappropriate/d others. In Cultural studies. Routledge, 295–337.Google Scholar
Reference
Sarah Homewood, Marika Hedemyr, Maja Fagerberg Ranten, and Susan Kozel. 2021. Tracing conceptions of the body in HCI: From user to more-than-human. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI’21). 1–12. https://doi.org/10.1145/3411764.3445656Google ScholarDigital Library
Reference
Kristina Höök, Caroline Hummels, Katherine Isbister, Patrizia Marti, Elena Márquez Segura, Martin Jonsson, Florian’Floyd’ Mueller, Pedro AN Sanches, Thecla Schiphorst, Anna Ståhl, 2017. Soma-based design theory. In Proceedings of the 2017 CHI conference Extended abstracts on human factors in computing systems(CHI EA ’17). 550–557. https://doi.org/10.1145/3027063.3027082Google ScholarDigital Library
Reference 1Reference 2
Kristina Höök, Phoebe Sengers, and Gerd Andersson. 2003. Sense and sensibility: evaluation and interactive art. In Proceedings of the SIGCHI conference on Human factors in computing systems(CHI’03). 241–248. https://doi.org/10.1145/642611.642654Google ScholarDigital Library
Reference
Arnau Horta, Salome Voegelin, Peter Szendy, Christoph Cox, Casey O’Callaghan, Seth Kim-Cohen, and Julian Henriques. 2016. On Listening 1: Thinking (Through) The Ear. https://rwm.macba.cat/en/research/listening-1-thinking-through-earGoogle Scholar
Reference 1Reference 2Reference 3
Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, 2003. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI conference on Human factors in computing systems(CHI’03). 17–24. https://doi.org/10.1145/642611.642616Google ScholarDigital Library
Reference
Don Ihde. 2009. Postphenomenology and technoscience: The Peking university lectures. Suny Press.Google ScholarDigital Library
Reference
Robert Jack, Jacob Harrison, and Andrew McPherson. 2020. Digital musical instruments as research products. In Proceedings of the International Conference on New Interfaces for Musical Expression(NIME’20). 446–451. https://doi.org/10.5281/zenodo.4813465Google ScholarCross Ref
Reference
Alecia Y Jackson and Lisa A Mazzei. 2013. Plugging one text into another: Thinking with theory in qualitative research. Qualitative inquiry 19, 4 (2013), 261–271. https://doi.org/10.1177/1077800412471510Google ScholarCross Ref
Reference 1Reference 2
Vadim Keylin. 2020. Crash, boom, bang: Affordances for participation in sound art. SoundEffects-An Interdisciplinary Journal of Sound and Sound Experience 9, 1 (2020), 98–115. https://doi.org/10.7146/se.v9i1.118243Google ScholarCross Ref
Reference
Ozgun Kilic Afsar, Yoav Luft, Kelsey Cotton, Ekaterina R Stepanova, Claudia Núñez-Pacheco, Rebecca Kleinberger, Fehmi Ben Abdesslem, Hiroshi Ishii, and Kristina Höök. 2023. Corsetto: A Kinesthetic Garment for Designing, Composing for, and Experiencing an Intersubjective Haptic Voice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI’23). 1–23. https://doi.org/10.1145/3544548.3581294Google ScholarDigital Library
Reference 1Reference 2
Amanda Lazar, Ben Jelen, Alisha Pradhan, and Katie A Siek. 2021. Adopting diffractive reading to advance hci research: A case study on technology for aging. ACM Transactions on Computer-Human Interaction (TOCHI) 28, 5 (2021), 1–29. https://doi.org/10.1145/3462326Google ScholarDigital Library
Reference 1Reference 2
Jen Liu, Daragh Byrne, and Laura Devendorf. 2018. Design for collaborative survival: An inquiry into human-fungi relationships. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI’18). 1–13. https://doi.org/10.1145/3173574.3173614Google ScholarDigital Library
Reference
Stephanie Loveless and Freya Zinovieff. [n. d.]. Situated Listening: Partial Perspectives and Critical Listening Positionality. In Proceedings of the ’Listening Pasts, Listening Futures’ 2023 World Forum for Acoustic Ecology Conference, volume=1, number=1, year=2023.Google Scholar
Reference
Thor Magnusson. 2009. Of epistemic tools: Musical instruments as cognitive extensions. Organised Sound 14, 2 (2009), 168–176. https://doi.org/10.1017/S1355771809000272Google ScholarDigital Library
Reference
Thor Magnusson, Chris Kiefer, and Halldor Ulfarsson. 2022. Reflexions upon Feedback. In Proceedings of the International Conference on New Interfaces for Musical Expression(NIME’22). https://doi.org/10.21428%2F92fbeb44.aa7de712Google ScholarCross Ref
Reference
Lisa A Mazzei. 2014. Beyond an easy sense: A diffractive analysis. Qualitative inquiry 20, 6 (2014), 742–746. https://doi.org/10.1177/1077800414530257Google ScholarCross Ref
Reference 1Reference 2
Tom Mudd. 2019. Material-oriented musical interactions. New Directions in Music and Human-Computer Interaction (2019), 123–133. https://doi.org/10.1007/978-3-319-92069-6_8Google ScholarCross Ref
Reference
Tom Mudd. 2023. Playing with Feedback: Unpredictability, Immediacy, and Entangled Agency in the No-input Mixing Desk. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI’23). 1–11. https://doi.org/10.1145/3544548.3580662Google ScholarDigital Library
Reference
Lizzie Muller, Greg Turner, George Khut, and Ernest Edmonds. 2006. Creating affective visualisations for a physiologically interactive artwork. In Tenth International Conference on Information Visualisation (IV’06). IEEE, 651–657. https://doi.org/10.1109/IV.2006.36Google ScholarDigital Library
Reference
Iohanna Nicenboim, Doenja Oogjes, Heidi Biggs, and Seowoo Nam. 2023. Decentering Through Design: Bridging Posthuman Theory with More-than-Human Design Practices. Human–Computer Interaction (2023), 1–27. https://doi.org/10.1080/07370024.2023.2283535Google ScholarCross Ref
Reference
Alva Noë. 2017. Strange Tools: Art and Human Nature: A Précis. Philosophy and Phenomenological Research 94, 1 (2017), 211–213. https://doi.org/10.1111/phpr.12361Google ScholarCross Ref
Reference 1Reference 2
Charlotte Nordmoen and Andrew P McPherson. 2022. Making space for material entanglements: A diffractive analysis of woodwork and the practice of making an interactive system. In Proceedings of the 2022 ACM Designing Interactive Systems Conference(DIS’22). 415–423. https://doi.org/10.1145/3532106.3533572Google ScholarDigital Library
Reference
Charlotte Nordmoen and Andrew P McPherson. 2023. What the Sensor Knows: More-Than-Human Knowledge Co-Production in Wood Carving. In Proceedings of the 2023 ACM Designing Interactive Systems Conference(DIS’23). 779–789. https://doi.org/10.1145/3563657.3596075Google ScholarDigital Library
Reference
Claudia Núñez-Pacheco and Lian Loke. 2022. Focusing for Interaction Design: An Introspective Somatic Method. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI’22). 1–18. https://doi.org/10.1145/3491102.3501978Google ScholarDigital Library
Reference
Marianna Obrist, Rob Comber, Sriram Subramanian, Betina Piqueras-Fiszman, Carlos Velasco, and Charles Spence. 2014. Temporal, affective, and embodied characteristics of taste experiences: A framework for design. In Proceedings of the SIGCHI conference on human factors in computing systems(CHI’14). 2853–2862. https://doi.org/10.1145/2556288.2557007Google ScholarDigital Library
Reference
Marianna Obrist, Sue Ann Seah, and Sriram Subramanian. 2013. Talking about tactile experiences. In Proceedings of the SIGCHI conference on human factors in computing systems(CHI’13). 1659–1668. https://doi.org/10.1145/2470654.2466220Google ScholarDigital Library
Reference
Casey O’Callaghan. 2007. Sounds: a philosophical theory. OUP Oxford.Google Scholar
Reference 1Reference 2
William Odom, Ron Wakkary, Youn-kyung Lim, Audrey Desjardins, Bart Hengeveld, and Richard Banks. 2016. From research prototype to research product. In Proceedings of the 2016 CHI conference on human factors in computing systems(CHI’16). 2549–2561. https://doi.org/10.1145/2858036.2858447Google ScholarDigital Library
Reference
Pauline Oliveros. 2022. Quantum Listening. Terra Ignota.Google Scholar
Reference
Claire Petitmengin. 2006. Describing one’s subjective experience in the second person: An interview method for the science of consciousness. Phenomenology and the Cognitive sciences 5, 3-4 (2006), 229–269. https://doi.org/10.1007/s11097-006-9022-2Google ScholarCross Ref
Reference 1Reference 2
Bruna Petreca, Sharon Baurley, and Nadia Bianchi-Berthouze. 2015. How do designers feel textiles?. In 2015 International conference on affective computing and intelligent interaction (ACII). IEEE, 982–987. https://doi.org/10.1109/ACII.2015.7344695Google ScholarDigital Library
Reference
Blake D. Poland. 2011. Trascription Quality. In Handbook of Interview Research, Jaber F. Gubriam and James A. Holstein (Eds.). Sage Publications.Google Scholar
Reference
Mirjana Prpa, Sarah Fdili-Alaoui, Thecla Schiphorst, and Philippe Pasquier. 2020. Articulating experience: Reflections from experts applying micro-phenomenology to design research in HCI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI’20). 1–14. https://doi.org/10.1145/3313831.3376664Google ScholarDigital Library
Reference
Joshua D Reiss and Andrew McPherson. 2014. Audio effects: theory, implementation and application. CRC Press.Google Scholar
Reference
Nicole Robson, Nick Bryan-Kinns, and Andrew McPherson. 2023. On Mediating Space, Sound and Experience: Interviews with situated sound art practitioners. Organised Sound 28, 1 (2023), 25–34. https://doi.org/10.1017/S1355771822000103Google ScholarCross Ref
Reference 1Reference 2
Nicole Robson, Andrew McPherson, and Nick Bryan-Kinns. 2022. Being With The Waves: An Ultrasonic Art Installation Enabling Rich Interaction Without Sensors. In Proceedings of the International Conference on New Interfaces for Musical Expression(NIME’22). https://doi.org/10.21428/92fbeb44.376bc758Google ScholarCross Ref
Reference 1Reference 2Reference 3
Davide Rocchesso, Stefania Serafin, Frauke Behrendt, Nicola Bernardini, Roberto Bresin, Gerhard Eckel, Karmen Franinovic, Thomas Hermann, Sandra Pauletto, Patrick Susini, 2008. Sonic interaction design: sound, information and experience. In CHI’08 Extended Abstracts on Human Factors in Computing Systems. 3969–3972. https://doi.org/10.1145/1358628.1358969Google ScholarDigital Library
Reference
Pedro Sanches, Noura Howell, Vasiliki Tsaknaki, Tom Jenkins, and Karey Helms. 2022. Diffraction-in-action: designerly explorations of agential realism through lived data. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI’22). 1–18. https://doi.org/10.1145/3491102.3502029Google ScholarDigital Library
Reference
Carsten Seiffarth. 2012. About Sound Installation Art. Kunstjournalen B-post 1, 12 (2012). https://b-post.no/en/12/seiffarth.htmlGoogle Scholar
Reference
Linnea Semmerling. 2018. Explorative Listening: A phenomenological approach to sounding artworks at museums. Leonardo Electronic Almanac 22, 3 (2018).Google Scholar
Reference
Xin Wei Sha, Adrian Freed, and Navid Navab. 2013. Sound design as human matter interaction. In CHI’13 Extended Abstracts on Human Factors in Computing Systems. 2009–2018. https://doi.org/10.1145/2468356.2468718Google ScholarDigital Library
Reference 1Reference 2Reference 3
Maxine Sheets-Johnstone. 1981. Thinking in Movement. The Journal of Aesthetics and Art Criticism 39, 4 (1981), 399–407. https://doi.org/10.2307/430239Google ScholarCross Ref
Reference
Maxine Sheets-Johnstone. 2002. Introduction to the special topic: Epistemology and movement. Journal of the Philosophy of Sport 29, 2 (2002), 103–105. https://doi.org/10.1080/00948705.2002.9714627Google ScholarCross Ref
Reference
Peter Sloterdijk. 2015. Der ästhetische Imperativ: Schriften zur Kunst. Suhrkamp Verlag.Google Scholar
Reference
Katta Spiel. 2021. The bodies of TEI–investigating norms and assumptions in the design of embodied interaction. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction(TEI’15). 1–19. https://doi.org/10.1145/3430524.3440651Google ScholarDigital Library
Reference 1Reference 2
Katta Spiel, Eva Hornecker, Rua Mae Williams, and Judith Good. 2022. ADHD and technology research–investigated by neurodivergent readers. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI’22). 1–21. https://doi.org/10.1145/3491102.3517592Google ScholarDigital Library
Reference
Ekaterina R Stepanova, Denise Quesnel, and Bernhard E Riecke. 2019. Understanding AWE: Can a virtual journey, inspired by the overview effect, lead to an increased sense of interconnectedness?Frontiers in Digital Humanities 6 (2019), 9. https://doi.org/10.3389/fdigh.2019.00009Google ScholarCross Ref
Reference
Jonathan Sterne. 2003. The audible past: Cultural origins of sound reproduction. Duke University Press.Google Scholar
Reference
Peter Szendy. 2020. Of Stigmatology: Punctuation as Experience. Fordham University Press.Google ScholarCross Ref
Reference
Hillevi Lenz Taguchi. 2012. A diffractive and Deleuzian approach to analysing interview data. Feminist theory 13, 3 (2012), 265–281. https://doi.org/10.1177/14647001124560Google ScholarCross Ref
Reference
Paul Tennent, Kristina Höök, Steve Benford, Vasiliki Tsaknaki, Anna Ståhl, Claudia Dauden Roquet, Charles Windlin, Pedro Sanches, Joe Marshall, Christine Li, 2021. Articulating soma experiences using trajectories. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI’21). 1–16. https://doi.org/10.1145/3411764.3445482Google ScholarDigital Library
Reference
Marie Thompson. 2017. Whiteness and the ontological turn in sound studies. Parallax 23, 3 (2017), 266–282. https://doi.org/10.1080/13534645.2017.1339967Google ScholarCross Ref
Vasiliki Tsaknaki, Kelsey Cotton, Pavel Karpashevich, and Pedro Sanches. 2021. “Feeling the Sensor Feeling you”: A Soma Design Exploration on Sensing Non-habitual Breathing. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI’21). 1–16. https://doi.org/10.1145/3411764.3445628Google ScholarDigital Library
Reference
Camila Valenzuela-Moguillansky and Alejandra Vásquez-Rosati. 2019. An analysis procedure for the micro-phenomenological interview. Constructivist Foundations 14, 2 (2019), 123–145. https://constructivist.info/14/2/123Google Scholar
Reference 1Reference 2
Salomé Voegelin. 2010. Listening to noise and silence. Listening to Noise and Silence (2010), 1–256.Google ScholarCross Ref
Reference 1Reference 2
Salomé Voegelin. 2021. Sonic sense: The meaning of the invisible. In The Oxford Handbook of Sound Art. Oxford University Press, Chapter 24, 349–366.Google Scholar
Reference 1Reference 2Reference 3
Salomé Voegelin. 2023. Who is Afraid of Sound: Listening as Transgression and Questioning of Cultural Visuality. springerin1 (2023). https://www.springerin.at/en/2023/1/wer-hat-angst-vor-klang/Google Scholar
Reference 1Reference 2
Ron Wakkary, Doenja Oogjes, Henry WJ Lin, and Sabrina Hauser. 2018. Philosophers living with the tilting bowl. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI’18). 1–12. https://doi.org/10.1145/3173574.3173668Google ScholarDigital Library
Reference
Peter Wright and John McCarthy. 2022. Experience-centered design: designers, users, and communities in dialogue. Springer Nature. https://doi.org/10.1007/978-3-031-02192-3Google ScholarCross Ref
Reference
Peter Wright, Jayne Wallace, and John McCarthy. 2008. Aesthetics and experience-centered design. ACM Transactions on Computer-Human Interaction (TOCHI) 15, 4 (2008), 1–21. https://doi.org/10.1145/1460355.1460360Google ScholarDigital Library
Reference
Daisy Yoo, Tilde Bekker, Peter Dalsgaard, Eva Eriksson, Simon Skov Fougt, Christopher Frauenberger, Batya Friedman, Elisa Giaccardi, Anne-Marie Hansen, Ann Light, 2023. More-Than-Human Perspectives and Values in Human-Computer Interaction. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems(CHI EA ’23). 1–3. https://doi.org/10.1145/3544549.3583174Google ScholarDigital Library
Reference 1Reference 2
Eevee Zayas-Garin, Charlotte Nordmoen, Andrew McPherson, 2023. Transmitting Digital Lutherie Knowledge: The Rashomon Effect for DMI Designers. In Proceedings of the International Conference on New Interfaces for Musical Expression(NIME’23).Google Scholar
Reference

Index Terms

Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation
1. Applied computing
  1. Arts and humanities
    1. Media arts
    2. Sound and music computing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction techniques
      1. Auditory feedback
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

On becoming a sociomaterial researcher: Exploring epistemological practices grounded in a relational, performative ontology
Abstract
What is the role of the researcher in a world that is continuously enacted and reconfigured in sociomaterial practices, a world in which subject and object, structure and agency, body and mind, knower and known, are assumed to be ...
Highlights
- The paper contributes to the development of epistemological practices available to draw upon for researchers assuming a sociomaterial approach.
Read More
In Pursuit of Measuring Pre-reflective Music Listening Experiences
AM '22: Proceedings of the 17th International Audio Mostly Conference

While the diverse effects and uses of music and sound have been extensively documented within music psychology, relatively little attention has been paid to the process and experience of listening itself. Previous literature have, however, considered ...
Read More
The Listening Artist : on Listening as an Artistic Practice Beyond Sound Art
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Published in
CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems
May 2024
18961 pages
ISBN:9798400703300
DOI:10.1145/3613904
Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University
,
Max L. Wilson
University of Nottingham
,
Phoebe Toups Dugas
Monash University/New Mexico State University
,
Irina Shklovski
University of Copenhagen
Copyright © 2024 Owner/Author
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 11 May 2024
Check for updates
Badges
- Artifacts Available / v1.1
- Honorable Mention
Author Tags
Diffraction
Entanglement
Epistemology
Experience-Centred Design
Listening
Micro-Phenomenology
Sonic Interaction
Sound Installation
Sound Studies
Ultrasound
Qualifiers
- research-article
- Research
- Refereed limited
Conference

Acceptance Rates
Overall Acceptance Rate6,199of26,314submissions,24%
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 0
  Total Citations
  View Citations
- 170
  Total Downloads
- Downloads (Last 12 months)170
- Downloads (Last 6 weeks)170
Other Metrics
View Author Metrics
Cited By
This publication has not been cited yet

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format .

View HTML Format

Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation

CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems

Abstract

1 INTRODUCTION

2 FOUNDATIONS

2.1 Agential Realism and the Co-Production of Knowledge

2.1.1 Diffractive Analysis.

2.2 Thinking with Sound

2.2.1 Sonic Materialism.

2.2.2 Listening as Punctuation.

2.2.3 The In-Between Space of Listening.

2.2.4 Plural Possibilites in Listening.

2.3 Entangled Agencies in Sonic Interaction

3 BEING WITH THE WAVES: AN ULTRASONIC ART INSTALLATION

3.1 Design Intentions

3.2 Implementation

3.2.1 Emitter.

3.2.2 Receiver.

3.3 Modulation Effects

3.4 Interaction Without Symbolic Representation

3.5 Sound Design

4 EXPLORING THE RELATIONAL EXPERIENCE OF LISTENING

4.1 Participants

4.2 Physical Set-Up

4.3 Procedure

4.3.1 Briefing.

4.3.2 Listening Encounter.

4.3.3 Interview.

4.3.4 Semi-Structured Interview.

4.3.5 Micro-Phenomenological Interview.

4.4 Data Preparation

4.5 Diffractive Analysis

4.6 Findings

4.6.1 Feeling Connection.

4.6.2 Relational Flux.

4.6.3 Body-Sound Alignment.

4.6.4 Expecting Sensors.

5 DISCUSSION

5.1 Micro-Phenomenology and Sonic Experience

5.2 Reverberations and Provocations

5.3 Comments on the Research Apparatus

5.4 Conclusion

ACKNOWLEDGMENTS

A CURATORIAL TEXT

B INTERVIEW QUESTIONS

B.1 Semi-Structured Interview

B.2 Micro-Phenomenological Interview

Footnotes

Supplemental Material

Available for Download

References

Cited By

Index Terms

Recommendations

On becoming a sociomaterial researcher: Exploring epistemological practices grounded in a relational, performative ontology

In Pursuit of Measuring Pre-reflective Music Listening Experiences

The Listening Artist : on Listening as an Artistic Practice Beyond Sound Art

Comments

Login options

Full Access

Published in

Sponsors

In-Cooperation

Publisher

Publication History

Check for updates

Badges

Author Tags

Qualifiers

Conference

Acceptance Rates

Funding Sources

Article Metrics

Other Metrics

PDF Format

eReader

Digital Edition

HTML Format

Share this Publication link

Share on Social Media