Abstract
This paper presents useful techniques and considerations in implementing underlying mandarin singing voice synthesis system using the RSLI unit. The system can receive the continuous speech of the lyrics of a song, and can synthesize the intended song based on the MIDI-based music database. This system is designed based on 3 units.. The first one is the input unit which allows the user specifies a musical score and phonetically-spelled lyrics to system. The second one is the modified unit and it is employed to implement the pitch-shifting function using the PSOLA method. The third one is the mixed unit which has some undesirable artificial-sounding buzzy-effects, including echo and vibrato effects. Moreover, the energy, duration, and spectrum modifications are also implemented in the mixed unit. The synthesized singing voice sounds reasonably good. From the subjective listening test, the MOS (mean opinion score) of 3.3 and 3.2 are obtained for the synthesized singing voices and the similarity of singer’s voice, respectively.
This research conducts in according to Services Oriented Machine Open Platform Software Development Project executed by Institute for information industry subsidized by Economic departments of Taiwan.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bennett, G., Rodet, X.: Synthesis of the singing voice. In: Mathews, M.V., Pierce, J.R. (eds.) Current Directions in Computer Music Research, pp. 19–44. MIT Press, Cambridge (1989)
Jang, J.-S.R., Jang, Y.-S.: Micro-controller Implementation of Melody Recognition. In: The 13th ACM Multimedia Conference (Demo paper), Berkeley, CA, USA (November 2003)
Lin, C.-Y., Jang, J.-S.R., Hsu, M.-Y.: An Automatic Singing Voice Rectifier. In: The 13th ACM Multimedia Conference (Poster paper, acceptance rate 17%), Berkeley, CA, USA (November 2003)
Jang, J.-S.R., Lee, H.-R., Yeh, C.-H.: Query by Tapping: A New Paradigm for Content-based Music Retrieval from Acoustic Input. In: The Second IEEE Pacific-Rim Conference on Multimedia, Beijing, China (October 2001)
Lin, C.-Y., Roger Jang, J.-S.: Automatic Phonetic Segmentation by Score Predictive Model for the Corpora of Mandarin Singing Voices. IEEE Transactions on Speech, Audio, and Language Processing (2007)
Chou, F.C., Tseng, C.Y., Lee, L.S.: A Set of Corpus-based Text-to-speech Synthesis Technologies for Mandarin Chinese. IEEE Transactions on Speech and Audio Processing 10(7), 481–494 (2002)
Lin, C.-Y., Jao, P.-C., Roger Jang, J.-S.: Effective Initial/Final Duration Prediction Method for Corpus-based Singing Voice Synthesis of Mandarin Chinese. In: Europen Conference on Speech Communication and Technology, Eurospeech (2007)
Jang, J.S., Gao, M.Y.: A Query-by-singing System Based on Dynamic Programming. In: Proc. International Workshop on Intelligent Systems Resolutions, pp. 85–89 (2000)
Jang, J.S., Lee, H.R.: Hierarchical Filtering Method for Content-based MusicRetrieval via Acoustic Input. In: Proc. ACM Multimedia, pp. 401–410 (2001)
Lin, C.-Y., Roger Jang, J.-S.: Automatic Phonetic Segmentation by Using a SPM-based Approach for a Mandarin Singing Voice Corpus. In: Proc. ICSLP, pp. 1489–1492 (2006)
Macon, M.W., Clements, M.A.: Speech Concatenation and Synthesis Using an Overlap-add Sinusoidal Model. In: Proc. of International Conference on Acoustics, Speech, and Signal Processing, pp. 361–364 (1996)
Proakis, J.R.J.G., Hansen, J.H.L.: Discrete-time processing of speech signals. Macmillan Pub. Co., New York (1993)
Jang, J.-S.R., Jang, Y.-S.: Micro-controller Implementation of Melody Recognition. In: The 13th ACM Multimedia Conference (Demo paper), Berkeley, CA, USA (November 2003)
Lin, C.-Y., Roger Jang, J.-S., Hwang, S.-H.: An On-the-fly Mandarin Singing Voice Synthesis System. In: The Third IEEE Pacific-Rim Conference on Multimedia, Hsinchu, Taiwan, D
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, HR., Huang, CF., Hsu, CH., Wang, WN. (2009). Rhythm Speech Lyrics Input for MIDI-Based Singing Voice Synthesis. In: Muneesawang, P., Wu, F., Kumazawa, I., Roeksabutr, A., Liao, M., Tang, X. (eds) Advances in Multimedia Information Processing - PCM 2009. PCM 2009. Lecture Notes in Computer Science, vol 5879. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10467-1_40
Download citation
DOI: https://doi.org/10.1007/978-3-642-10467-1_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10466-4
Online ISBN: 978-3-642-10467-1
eBook Packages: Computer ScienceComputer Science (R0)