A Perceptually Motivated Visualisation Paradigm for Musical Timbre

Background is given on existing research into the perception and description of timbre. This is used to contextualise the development of visual mappings for acoustic timbre features. Two example systems are explained and discussed, which demonstrate the use of perceptually motivated mappings from acoustic features to visual properties, in specific contexts. Possible applications and extensions of such systems are proposed.

Content

Author and article information

Contributors

Sean Soraghan

Alain Renaud

Ben Supper

Conference

Publication date: July 2016

Publication date (Print): July 2016

Pages: 53-60

Affiliations

[1 ] ROLI / Center for Digital Entertainment

2 Glebe Road, London E8 4BD

United Kingdom

[2 ] Center for Digital Entertainment

Bournemouth University, BH12 5BB

United Kingdom arenaud@bournemouth.com

[3 ] ROLI

2 Glebe Road, London E8 4BD

United Kingdom

Article

DOI: 10.14236/ewic/EVA2016.11

SO-VID: a2e022dd-9a1a-459b-9de5-b1b3441a7560

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 12 - 14 July 2016

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2016.11

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

V. AlluriP. Toiviainen 2010 Exploring perceptual and acoustical correlates of polyphonic timbre Music Perception 27 3 223 241
American Standards Association and Acoustical Society 1960 American standard acoustical terminology (including mechanical shock and vibration). American Standards Association
A. L. Beal 1985 The skill of recognizing musical structures Memory & Cognition 13 5 405 412
J. W. Beauchamp 1982 Synthesis by spectral amplitude and "brightness" matching of analyzed musical instrument tones Journal of the Audio Engineering Society 30 6 396 406
K. W. Berger 1964 Some factors in the recognition of timbre Journal of the Acoustical Society of America 36 10 1888 1891
F. BerthautM. Desainte-CatherineM. Hachet 2010 Combining audiovisual mappings for 3d musical interaction International Computer Music Conference, 100
J. F. Blinn 1977 Models of light reflection for computer synthesized pictures SIGGRAPH 11 192 198
V. C. CarusoE. Balaban 2014 Pitch and timbre interfere when both are parametrically varied PLOS ONE 9 1
R. B. Dannenberg 1993 Music Representation Issues, Techniques, and Systems Computer Music Journal 17 3 20 30
A. C. DisleyD. M. HowardA. D. Hunt 2006 Timbral description of musical instruments International Conference on Music Perception and Cognition, 61 68
J. M. Grey 1975 An exploration of musical timbre. Dept. of Music, Stanford University
J. M. GreyJ. W. Gordon 1978 Perceptual effects of spectral modifications on musical timbres Journal of the Acoustical Society of America 63 5 1493 1500
K. Giannakis 2001 Sound mosaics: a graphical user interface for sound synthesis based on audiovisual associations PhD thesis, Middlesex University UK
R. A. KendallE. C. CarteretteJ. M. Hajda 1999 Perceptual and acoustical features of natural and synthetic orchestral instrument tones Music Perception, 327 363
J. KrimphoffS. McAdamsS. WinsbergH. PetitS. BakchineB. DuboisB. LaurentB. MontagneJ. TouchonP. Robert 1994 Characterization of the timbre of complex sounds. 2. Acoustic analysis and psychophysical quantification J. de Physique 4 625 628
C. L. KrumhanslP. Iverson 1992 Perceptual interactions between musical pitch and timbre Journal of Experimental Psychology: Human Perception and Performance 18 3 739
J. MarozeauA. de Cheveign’e 2007 The effect of fundamental frequency on the brightness dimension of timbre Journal of the Acoustical Society of America 121 1 383 387
S. McAdamsS. WinsbergS. DonnadieuG. De SoeteJ. Krimphoff 1995 Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes Psychological research 58 3 177 192
R. D. MelaraL. E. Marks 1990 Interaction among auditory dimensions: Timbre, pitch, and loudness Perception & Psychophysics 48 2 169 178
G. PeetersB. L. GiordanoP. SusiniN. MisdariisS. McAdams 2011 The timbre toolbox: Extracting audio descriptors from musical signals Journal of the Acoustical Society of America 130 5 2902 2916
J. R. PlattR. J. Racine 1985 Effect of frequency, timbre, experience, and feedback on musical tuning skills Perception & Psychophysics 38 6 543 553
R. PrattP. Doak 1976 A subjective rating scale for timbre Journal of Sound and Vibration 45 3 317 328
E. SchubertJ. Wolfe 2006 Does timbral brightness scale with frequency and spectral centroid? Acta acustica united with acustica 92 5 820 825
E. SchubertJ. WolfeA. Tarnopolsky 2004 Spectral centroid and timbre in complex, multiple instrumental textures Proc. 8th Int. Conf. on Music Perception & Cognition (ICMPC)
A. Seago 2013 A new interaction strategy for musical timbre design Music and humancomputer interaction, 153 169 London Springer
A. SeagoS. HollandP. Mulholland 2004 A critical analysis of synthesizer user interfaces for timbre Proceedings of the XVIIIrd British HCI Group Annual Conference Retrieved from Citeseer
A. Tsiros 2013 The Dimensions and Complexities of Audio-Visual Association Proceedings of Electronic Visualisation and the Arts, London
A. TsirosG. Leplatre 2015 AniMorph: Animation Driven Audio Mosaicing Proceedings of Electronic Visualisation and the Arts. London
G. von Bismarck 1974 Timbre of steady sounds: A factorial investigation of its verbal attributes Acta Acustica united with Acustica 30 3 146 159
J. WapnickP. Freeman 1980 Effect of dark-bright timbral variation on the perception of flatness and sharpness Journal of Research in Music Education 28 3 176 184
C. M. WarrierR. J. Zatorre 2002 Influence of tonal context and timbral variation on perception of pitch Perception & Psychophysics 64 2 198 207
L. WedinG. Goude 1972 Dimension analysis of the perception of instrumental timbre Scandinavian Journal of Psychology 13 1 228 240
D. L. Wessel 1978 Low dimensional control of musical timbre Audio Engineering Society 59^th Convention, 13 37
R. S. Wolpert 1990 Recognition of melody, harmonic accompaniment, and instrumentation: Musicians vs. nonmusicians Music Perception, 95 105
A. ZacharakisK. Pastiadis 2015 A confirmatory approach of the luminance-texture-mass model for musical timbre semantics Proceedings of the Audio Mostly 2015 on Interaction With Sound, 39 ACM
A. ZacharakisK. PastiadisG. PapadelisJ. D. Reiss 2011 An investigation of musical timbre: Uncovering salient semantic descriptors and perceptual dimensions ISMIR, 807 812
A. ZacharakisK. PastiadisJ. D. Reiss 2014 An interlanguage study of musical timbre semantic dimensions and their acoustic correlates Music Perception: An Interdisciplinary Journal 31 4 339 358
A. ZacharakisK. PastiadisJ. D. ReissG. Papadelis 2012 Analysis of musical timbre semantics through metric and non-metric data reduction techniques Proceedings of the 12th International Conference on Music Perception and Cognition (ICMPC12) and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music (EsCoM 08), 1177 1182

Comments

Comment on this article

[1] V. AlluriP. Toiviainen 2010 Exploring perceptual and acoustical correlates of polyphonic timbre Music Perception 27 3 223 241

[2] American Standards Association and Acoustical Society 1960 American standard acoustical terminology (including mechanical shock and vibration). American Standards Association

[3] A. L. Beal 1985 The skill of recognizing musical structures Memory & Cognition 13 5 405 412

[4] J. W. Beauchamp 1982 Synthesis by spectral amplitude and "brightness" matching of analyzed musical instrument tones Journal of the Audio Engineering Society 30 6 396 406

[5] K. W. Berger 1964 Some factors in the recognition of timbre Journal of the Acoustical Society of America 36 10 1888 1891

[6] F. BerthautM. Desainte-CatherineM. Hachet 2010 Combining audiovisual mappings for 3d musical interaction International Computer Music Conference, 100

[7] J. F. Blinn 1977 Models of light reflection for computer synthesized pictures SIGGRAPH 11 192 198

[8] V. C. CarusoE. Balaban 2014 Pitch and timbre interfere when both are parametrically varied PLOS ONE 9 1

[9] R. B. Dannenberg 1993 Music Representation Issues, Techniques, and Systems Computer Music Journal 17 3 20 30

[10] A. C. DisleyD. M. HowardA. D. Hunt 2006 Timbral description of musical instruments International Conference on Music Perception and Cognition, 61 68

[11] J. M. Grey 1975 An exploration of musical timbre. Dept. of Music, Stanford University

[12] J. M. GreyJ. W. Gordon 1978 Perceptual effects of spectral modifications on musical timbres Journal of the Acoustical Society of America 63 5 1493 1500

[13] K. Giannakis 2001 Sound mosaics: a graphical user interface for sound synthesis based on audiovisual associations PhD thesis, Middlesex University UK

[14] R. A. KendallE. C. CarteretteJ. M. Hajda 1999 Perceptual and acoustical features of natural and synthetic orchestral instrument tones Music Perception, 327 363

[15] J. KrimphoffS. McAdamsS. WinsbergH. PetitS. BakchineB. DuboisB. LaurentB. MontagneJ. TouchonP. Robert 1994 Characterization of the timbre of complex sounds. 2. Acoustic analysis and psychophysical quantification J. de Physique 4 625 628

[16] C. L. KrumhanslP. Iverson 1992 Perceptual interactions between musical pitch and timbre Journal of Experimental Psychology: Human Perception and Performance 18 3 739

[17] J. MarozeauA. de Cheveign’e 2007 The effect of fundamental frequency on the brightness dimension of timbre Journal of the Acoustical Society of America 121 1 383 387

[18] S. McAdamsS. WinsbergS. DonnadieuG. De SoeteJ. Krimphoff 1995 Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes Psychological research 58 3 177 192

[19] R. D. MelaraL. E. Marks 1990 Interaction among auditory dimensions: Timbre, pitch, and loudness Perception & Psychophysics 48 2 169 178

[20] G. PeetersB. L. GiordanoP. SusiniN. MisdariisS. McAdams 2011 The timbre toolbox: Extracting audio descriptors from musical signals Journal of the Acoustical Society of America 130 5 2902 2916

[21] J. R. PlattR. J. Racine 1985 Effect of frequency, timbre, experience, and feedback on musical tuning skills Perception & Psychophysics 38 6 543 553

[22] R. PrattP. Doak 1976 A subjective rating scale for timbre Journal of Sound and Vibration 45 3 317 328

[23] E. SchubertJ. Wolfe 2006 Does timbral brightness scale with frequency and spectral centroid? Acta acustica united with acustica 92 5 820 825

[24] E. SchubertJ. WolfeA. Tarnopolsky 2004 Spectral centroid and timbre in complex, multiple instrumental textures Proc. 8th Int. Conf. on Music Perception & Cognition (ICMPC)

[25] A. Seago 2013 A new interaction strategy for musical timbre design Music and humancomputer interaction, 153 169 London Springer

[26] A. SeagoS. HollandP. Mulholland 2004 A critical analysis of synthesizer user interfaces for timbre Proceedings of the XVIIIrd British HCI Group Annual Conference Retrieved from Citeseer

[27] A. Tsiros 2013 The Dimensions and Complexities of Audio-Visual Association Proceedings of Electronic Visualisation and the Arts, London

[28] A. TsirosG. Leplatre 2015 AniMorph: Animation Driven Audio Mosaicing Proceedings of Electronic Visualisation and the Arts. London

[29] G. von Bismarck 1974 Timbre of steady sounds: A factorial investigation of its verbal attributes Acta Acustica united with Acustica 30 3 146 159

[30] J. WapnickP. Freeman 1980 Effect of dark-bright timbral variation on the perception of flatness and sharpness Journal of Research in Music Education 28 3 176 184

[31] C. M. WarrierR. J. Zatorre 2002 Influence of tonal context and timbral variation on perception of pitch Perception & Psychophysics 64 2 198 207

[32] L. WedinG. Goude 1972 Dimension analysis of the perception of instrumental timbre Scandinavian Journal of Psychology 13 1 228 240

[33] D. L. Wessel 1978 Low dimensional control of musical timbre Audio Engineering Society 59^th Convention, 13 37

[34] R. S. Wolpert 1990 Recognition of melody, harmonic accompaniment, and instrumentation: Musicians vs. nonmusicians Music Perception, 95 105

[35] A. ZacharakisK. Pastiadis 2015 A confirmatory approach of the luminance-texture-mass model for musical timbre semantics Proceedings of the Audio Mostly 2015 on Interaction With Sound, 39 ACM

[36] A. ZacharakisK. PastiadisG. PapadelisJ. D. Reiss 2011 An investigation of musical timbre: Uncovering salient semantic descriptors and perceptual dimensions ISMIR, 807 812

[37] A. ZacharakisK. PastiadisJ. D. Reiss 2014 An interlanguage study of musical timbre semantic dimensions and their acoustic correlates Music Perception: An Interdisciplinary Journal 31 4 339 358

[38] A. ZacharakisK. PastiadisJ. D. ReissG. Papadelis 2012 Analysis of musical timbre semantics through metric and non-metric data reduction techniques Proceedings of the 12th International Conference on Music Perception and Cognition (ICMPC12) and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music (EsCoM 08), 1177 1182

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65