Abstract
Music players for personal computers are often featured with music visualization by generating animated patterns according to the music’s low-level features such as loudness and spectrum. This paper proposes an emotion-based music player which synchronizes visualization (photos) with music based on the emotions evoked by auditory stimulus of music and visual content of visualization. For emotion detection from photos, we collected 398 photos with their emotions annotated by 496 users through the web. With these annotations, a Bayesian classification method is proposed for automatic photo emotion detection. For emotion detection from music, we adopt an existing method. Finally, for composition of music and photos, in addition to matching high-level emotions, we also consider low-level feature harmony and temporal visual coherence. It is formulated as an optimization problem and solved by a greedy algorithm. Subjective evaluation shows emotion-based music visualization enriches users’ listening experiences.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Lu, L., Liu, D., Zhang, H.J.: Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio, Speech & Language Processing 14(1), 5–18 (2006)
Wu, T.L., Jeng, S.K.: Probabilistic estimation of a novel music emotion model. In: The 14th International Multimedia Modeling Conference, Kyoto, Japan (2008)
Chen, J.C., Chu, W.T., Kuo, J.H., Weng, C.Y., Wu, J.L.: Tiling slideshow. In: ACM Multimedia, Santa Barbara, CA, USA (2006)
Hua, X.S., Lu, L., Zhang, H.J.: Automatic music video generation based on temporal pattern analysis. In: ACM Multimedia, New York, NY, USA (2004)
Mulhem, P., Kankanhalli, M.S., Yi, J., Hassan, H.: Pivot vector space approach for audio-video mixing. IEEE MultiMedia 10(2), 28–40 (2003)
Hevner, K.: Expression in music: a discussion of experimental studies and theories. Psychol. Rev. 42, 186–204 (1935)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (iaps): technical manual and affective ratings. NIMH Center for the Study of Emotion and Attention (1997)
Wu, T.L., Jeng, S.K.: Regrouping expressive terms for musical qualia. In: WOCMAT on Computer Music and Audio Technology, Taiwan (2007)
Zhang, L., Lin, F., Zhang, B.: A cbir method based on color-spatial feature. In: IEEE Region 10 Annual International Conference (1999)
Grubinger, M., Clough, P., Muller, H., Deselears, T.: The iapr tc-12 benchmark – a new evaluation resource for visual information systems. In: International Workshop OntoImage’2006 Language Resources for Content-Based Image Retrieval (2006)
Frey, B.J.J., Dueck, D.: Clustering by passing messages between data points. Science (2007)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. (2001) Software, available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Dixon, S.: Mirex 2006 audio beat tracking evaluation: Beatroot. MIREX (2006)
Ox, J.: Two performances in the 21st century virtual color organ: GridJam and Im Januar am Nil. In: Proceedings of the Seventh International Conference on Virtual Systems and Multimedia, p. 580 (2001)
Grey, J.: An exploration of musical timbre. Ph.D. Dissertation, Stanford University (1975)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, CH., Weng, MF., Jeng, SK., Chuang, YY. (2008). Emotion-Based Music Visualization Using Photos. In: Satoh, S., Nack, F., Etoh, M. (eds) Advances in Multimedia Modeling. MMM 2008. Lecture Notes in Computer Science, vol 4903. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77409-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-77409-9_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77407-5
Online ISBN: 978-3-540-77409-9
eBook Packages: Computer ScienceComputer Science (R0)