Abstract
In this research, we propose a system to extract the object which is pointed by the instructor with a demonstrative speech phrase. The system extracts audio features from a speech signals of the instructor and calculates the likelihood of the demonstrative speech phrase. The system also extracts video features from the video camera and calculates the likelihood of the pointing gesture. Then, the system combines the likelihood of audio and video features to decide whether the instructor points the object with demonstrative speech phrase. We conducted an experiment using a real lecture, which has shown the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Kato, N., et al.: A Basic Study on Effects of Keyword in Remote Captioning Services for a Symposium. IEICE technical report Welfare Information technology 106(57), 47–52 (2006) (in Japanese)
Kimura, T., Umemura, Y., Kanda, K., Hara, D.: An automated caption insertion system for the PowerPoint presentation. IEICE technical report Welfare Information technology 107(61), 75–78 (2007) (in Japanese)
Murata, K.T., Kimura, E., Kurita, S.: Impact of Network Delay to Remote Captioning Service. IEICE technical report Welfare Information technology 106(489), 37–42 (2007)
Mukhopadhyay, S., Smith, B.: Passive capture and structuring of lectures. In: Proc. 7th ACM Int. Conf. on MM, pp. 477–487 (1999)
Ishizuka, K., Kamera, Y., Minoh, M.: Speech and Video Indexing on Automatic Lecture Recording System. Technical report of IEICE PRMU 99(709), 91–98 (2006) (in Japanese)
Takeuchi, Y., Sakashita, Y., Wakatsuki, D., Minagawa, H., Ohnishi, N.: Communication Supporting System in a Classroom Environment for the Hearing Impaired. In: Proc. Int. Conf. Computer Helping People with Special Needs, pp. 627–634 (2006)
Takeuchi, Y., Saito, K., Ohnishi, N., Iizuka, S., Nakajima, S.: Camera Control for Remote Transcription System. In: Proc. IASTED Int. Conf. Computer Graphics and Imaging, pp. 88–93 (2008)
Onishi, M., Fukunaga, K.: Shooting the lecture scene using computer-controlled cameras based on situation understanding and evaluation of video images. In: Proc. of Int. Conf. on Pattern Recognition, vol. 1, pp. 781–784 (2004)
Kamada, Y., Ishizuka, K., Minoh, M.: A Live Video Imaging Method for Capturing Presentation Information In Distance Learning. In: Int. Conf. on Multimedia and Expo., vol. 3, pp. 1237–1240 (2000)
Nishigori, S., Suganuma, A.: Automatic Camera Control System for a Distant Lecture with Videoing a Normal Classroom. In: Proc. of World Conf. on Educational Multimedia, Hypermedia and Telecommunications, pp. 1892–1897 (2002)
Ashikawa, T., Suganuma, A., Taniguchi, R.-i.: A Judgment of Beginning and Ending of Teacher’s Chalking with Images and Sounds Processing. IEICE technical report Education technology 102(509), 43–48 (2002) (in Japanese)
Pozzer-Ardenghi, L., Roth, W.M.: Gestures: Helping students to understand photographs in Lectures. In: Proc. Connections 2003, pp. 31–58 (2003)
Sakiyama, T., Mukunoki, M., Katsuo, I.: Detection of the Indicated Area with an Indication Srick. In: Int. Conf. on Multimodal Interfaces (200), pp. 480–487.
Marutani, T., Nishiguchi, S., Kakusho, K., Minoh, M.: Making a lecture content with deictic information about indicated objects in lecture materials. In: AEARU Workshop on Network Education, pp. 70–75 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Takeuchi, Y., Saito, K., Ito, A., Ohnishi, N., Iizuka, S., Nakajima, S. (2008). Extracting Pointing Object with Demonstrative Speech Phrase for Remote Transcription in Lecture. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds) Computers Helping People with Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70540-6_89
Download citation
DOI: https://doi.org/10.1007/978-3-540-70540-6_89
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70539-0
Online ISBN: 978-3-540-70540-6
eBook Packages: Computer ScienceComputer Science (R0)