Geza Kovacs
Robert C. Miller
ABSTRACT
Language learners often use subtitled videos to help them learn. However, standard subtitles are geared more towards comprehension than vocabulary learning, as translations are nonliteral and are provided only for phrases, not vocabulary. This paper presents Smart Subtitles, which are interactive subtitles tailored towards vocabulary learning. Smart Subtitles can be automatically generated from common video sources such as subtitled DVDs. They provide features such as vocabulary definitions on hover, and dialog-based video navigation. In our pilot study with intermediate learners studying Chinese, participants correctly defined over twice as many new words in a post-viewing vocabulary test when they used Smart Subtitles, compared to dual Chinese-English subtitles. Learners spent the same amount of time watching clips with each tool, and enjoyed viewing videos with Smart Subtitles as much as with dual subtitles. Learners understood videos equally well using either tool, as indicated by self-assessments and independent evaluations of their summaries.
- Bansal, M., DeNero, J., and Lin, D. Unsupervised translation sense clustering. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics (2012), 773--782. Google Scholar
Digital Library
- Bianchi, F., and Ciabattoni, T. Captions and Subtitles in EFL Learning: an investigative study in a comprehensive computer environment. EUT-Edizioni Università di Trieste (2008).Google Scholar
- Bird, S., Klein, E., and Loper, E. Natural Language Processing with Python. O'Reilly, 2009. Google ScholarDigital Library
- Breen, J. WWWJDIC - A Feature-Rich WWW-Based Japanese Dictionary. eLEX2009 (2009), 31.Google Scholar
- Danan, M. Reversed subtitling and dual coding theory: New directions for foreign language instruction. Language Learning 42, 4 (1992), 497--527.Google ScholarCross Ref
- Danan, M. Captioning and subtitling: Undervalued language learning strategies. Meta: Translators? Journal 49, 1 (2004), 67--77.Google Scholar
- Dummitt, N. Chinese Through Tone and Color. Hippocrene Books, 2008.Google Scholar
- d'Ydewalle, G. Foreign-language acquisition by watching subtitled television programs. Journal of Foreign Language Education and Research 12 (2002), 59--77.Google Scholar
- Fukunaga, N. "Those anime students": Foreign language literacy development through Japanese popular culture. Journal of Adolescent & Adult Literacy 50, 3 (2006), 206--222.Google ScholarCross Ref
- Harris, C., and Stephens, M. A combined corner and edge detector. In Alvey vision conference, vol. 15, Manchester, UK (1988), 50.Google Scholar
- Herron, C. An investigation of the effectiveness of using an advance organizer to introduce video in the foreign language classroom. The Modern Language Journal 78, 2 (1994), 190--198.Google ScholarCross Ref
- Katsamanis, A., Black, M., Georgiou, P. G., Goldstein, L., and Narayanan, S. SailAlign: Robust long speech-text alignment. In Proc. of Workshop on New Tools and Methods for Very-Large Scale Phonetics Research (2011).Google Scholar
- Koolstra, C. M., Peeters, A. L., and Spinhof, H. The pros and cons of dubbing and subtitling. European Journal of Communication 17, 3 (2002), 325--354.Google ScholarCross Ref
- Krippendorff, K. Computing Krippendorff's alpha reliability. Departmental Papers (ASC) (2007), 43.Google Scholar
- Kurohashi, S., Nakamura, T., Matsumoto, Y., and Nagao, M. Improvements of Japanese morphological analyzer JUMAN. In Proceedings of The International Workshop on Sharable Natural Language (1994), 22--28.Google Scholar
- Lee, D.-S. Effective gaussian mixture learning for video background subtraction. Pattern Analysis and Machine Intelligence, IEEE Transactions on 27, 5 (2005), 827--832. Google ScholarDigital Library
- MDBG. CC-CEDICT Chinese-English Dictionary. MDBG (2013).Google Scholar
- Microsoft. Microsoft Office OneNote 2010. Microsoft (2010).Google Scholar
- Mitterer, H., and McQueen, J. M. Foreign subtitles help but native-language subtitles harm foreign speech perception. PloS one 4, 11 (2009), e7785.Google Scholar
- Raine, P. Incidental Learning of Vocabulary through Authentic Subtitled Videos. JALT - The Japan Association for Language Teaching (2012).Google Scholar
- Sakunkoo, N., and Sakunkoo, P. GliFlix: Using Movie Subtitles for Language Learning. In UIST 2013 Adjunct, ACM (2009).Google Scholar
- Secules, T., Herron, C., and Tomasello, M. The effect of video context on foreign language learning. The Modern Language Journal 76, 4 (1992), 480--490.Google ScholarCross Ref
- Shea, P. Leveling the playing field: A study of captioned interactive video for second language learning. Journal of Educational Computing Research 22, 3 (2000), 243--264.Google ScholarCross Ref
- Smith, R. An overview of the Tesseract OCR engine. In Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on, vol. 2, IEEE (2007), 629--633. Google ScholarDigital Library
- Speed, E. Rikaikun. Google Chrome Web Store (2013).Google Scholar
- Tseng, H., Chang, P., Andrew, G., Jurafsky, D., and Manning, C. A Conditional Random Field Word Segmenter for SIGHAN bakeoff 2005. In Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, vol. 171, Jeju Island, Korea (2005).Google Scholar
- W3C. WebVTT: The Web Video Text Tracks Format. W3C (2013).Google Scholar
- Wesche, M., and Paribakht, T. S. Assessing Second Language Vocabulary Knowledge: Depth Versus Breadth. Canadian Modern Language Review 53, 1 (1996), 13--40.Google Scholar
- Zesch, T., Müller, C., and Gurevych, I. Extracting Lexical Semantic Knowledge from Wikipedia and Wiktionary. In LREC, vol. 8 (2008), 1646--1652.Google Scholar
- Zuggy, B. SubRip (2011).Google Scholar
Index Terms
- Smart subtitles for vocabulary learning
Recommendations
Smart subtitles for language learning
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsLanguage learners often use subtitled videos to help them learn the language. However, standard subtitles are suboptimal for vocabulary acquisition, as translations are nonliteral and made at the phrase level, making it hard to find connections between ...
Dynamic Subtitles: The User Experience
TVX '15: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online VideoSubtitles (closed captions) on television are typically placed at the bottom-centre of the screen. However, placing subtitles in varying positions, according to the underlying video content (`dynamic subtitles'), has the potential to make the overall ...
Chinese EFL Teachers' Perceptions of Subtitles' Effects on English Comprehension and Vocabulary Acquisition: Subtitles' Effects on English Capability
This article reports on a qualitative study that evaluated Chinese EFL (English as a Foreign Language) teachers' perceptions of the effects of four different subtitling modes on English comprehension and vocabulary acquisition of learners in secondary ...
Comments