Abstract
Remote participants in hybrid meetings often have problems to follow what is going on in the (physical) meeting room they are connected with. This paper describes a videoconferencing system for participation in hybrid meetings. The system has been developed as a research vehicle to see how technology based on automatic real-time recognition of conversational behavior in meetings can be used to improve engagement and floor control by remote participants. The system uses modules for online speech recognition, real-time visual focus of attention as well as a module that signals who is being addressed by the speaker. A built-in keyword spotter allows an automatic meeting assistant to call the remote participant’s attention when a topic of interest is raised, pointing at the transcription of the fragment to help him catch-up.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
McCowan, I., Carletta, J., Kraaij, W., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kronenthal, M., Lathoud, G., Lincoln, M., Lisowska, A., Post, W., Reidsma, D., Wellner, P.: The AMI Meeting Corpus. In: Proceedings of Measuring Behavior 2005, the 5th International Conference on Methods and Techniques in Behavioral Research, Wageningen, Netherlands (2005)
Carletta, J.C.: Unleashing the killer corpus: experiences in creating the multi-everything AMI meeting corpus. Resources and Evaluation 41(2), 181–190 (2007)
Post, W., Cremers, A., Henkemans, O.: A research environment for meeting behavior. In: Nijholt, A., Nishida, T., Fruchter, R., Rosenberg, D. (eds.) Social Intelligence Design, Enschede, The Netherlands (2004)
Wilson, T.: Annotating subjective content in meetings. In: Proceedings of LREC (2008)
Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., Voormann, H.: The nite xml toolkit: Flexible annotation for multi-modal language data. Behavior Research Methods, Instruments, and Computers 35(3), 353–363 (2003)
Reidsma, D., Hofs, D., Jovanovic, N.: A presentation of a set of new annotation tools based on the nxt api. In: Proceedings of Measuring Behavior, Wageningen, The Netherlands (2005)
Nijholt, A., Rienks, R., Zwiers, J., Reidsma, D.: Online and off-line visualization of meeting information and meeting support. Visual Comput. 22, 965–976 (2006)
Matena, L., Jaimes, A., Popescu-Belis, A.: Graphical representation of meetings on mobile devices. In: MobileHCI 2008 Demonstrations (10th International Conference on Human-Computer Interaction with Mobile Devices and Services), Amsterdam (2008)
Edelsky, C.: Who’s got the floor? Language in Society 10, 383–421 (1981)
Hayashi, R.: Floor structure of english and japanese conversation. Journal of Pragmatics 16, 1–30 (1991)
Aoki, P., Romaine, M., Szymanski, M., Thornton, J., Wilson, D., Woodruff, A.: The mad hatter’s cocktail party: A social mobile audio space supporting multiple conversations. In: Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems (CHI 2003), pp. 425–432 (2003)
Bettinghaus, E.P., Cody, M.J.: Persuasive Communication. In: Wadsworth Thomson Learning, 5th edn. (1994)
Clark, H.H., Schaefer, F.E.: Dealing with overhearers. In: Arenas of language use. University of Chicago Press, Chicago (1992)
Monk, A., Watts, L.: Peripheral participation in video-mediated communication. Int. J. Human-Computer Studies 52, 933–958 (2000)
Fogg, B., Tseng, H.: The elements of computer credibility. In: Proceeding of CHI 1999, pp. 80–87 (1999)
Yankelovich, N., Kaplan, J., Simpson, N., Provino, J.: Porta-person: telepresence for the connected meeting room. In: Proceedings of CHI 2007, pp. 2789–2794 (2007)
Sacks, H., Schegloff, E., Jefferson, G.: A simplest systematics for the organization of turn-taking for conversation. Language 50, 696–735 (1974)
Donath, J.S.: Mediated faces. In: Beynon, M., Nehaniv, C.L., Dautenhahn, K. (eds.) CT 2001. LNCS, vol. 2117, pp. 373–390. Springer, Heidelberg (2001)
Hain, T., El Hannani, A., Wrigley, S.N., Wan, V.: Automatic speech recognition for scientific purposes - webasr. In: Proceedings of the international conference on spoken language processing (Interspeech 2008) (2008)
Op den Akker, H., Schulz, C.: Exploring features and classifiers for dialogue act segmentation. In: Popescu-Belis, A., Stiefelhagen, R. (eds.) MLMI 2008. LNCS, vol. 5237, pp. 196–207. Springer, Heidelberg (2008)
Germesin, S., Becker, T., Poller, P.: Determining latency for on-line dialog act classification. In: Poster Session for the 5th International Workshop on Machine Learning for Multimodal Interaction, vol. 5237 (2008)
Szöke, I., Schwarz, P., Burget, L., Fapšo, M., Karafiát, M., Černocký, J., Matějka, P.: Comparison of keyword spotting approaches for informal continuous speech. In: Interspeech 2005 - Eurospeech - 9th European Conference on Speech Communication and Technology, pp. 633–636 (2005)
Ba, S., Odobez, J.M.: Recognizing human visual focus of attention from head pose in meetings. IEEE Transaction on Systems, Man, and Cybernetics, Part B (Trans. SMC-B) 39, 16–33 (2009)
Op den Akker, H.: On addressee prediction for remote hybrid meeting settings. Master’s thesis, University of Twente (2009)
Hauber, J., Regenbrecht, H., Billinghurst, M., Cockburn, A.: Spatiality in videoconferencing: Trade-offs between efficiency and social presence. In: Proceedings ACM Conference on computer-supported cooperative work, CSCW 2006 (November 2004)
Baltes, B.B., Dickson, M.W., Sherman, M.P., Bauer, C.C., LaGanke, J.S.: Computer-mediated communication and group decision making: A meta-analysis. Organizational Behavior and Human Decision Processes 87(1), 156–179 (2002)
Popescu-Belis, A., Stiefelhagen, R. (eds.): MLMI 2008. LNCS, vol. 5237. Springer, Heidelberg (2008)
Takemae, Y., Otsuka, K., Yamato, J., Ozawa, S.: The subjective evaluation experiments on an automatic video editing system using vision-based head tracking for multiparty conversations. IEEJ Transactions on Electronics, Information and Systems 126(4), 435–442 (2006)
Takemae, Y., Otsuka, K., Yamato, J.: Effects of automatic video editing system using stereo-based head tracking for archiving meetings. In: IEEE International Conference on Multimedia and Expo., ICME 2005, July 2005, pp. 185–188 (2005)
Otsuka, K., Araki, S., Ishizuka, K., Fujimoto, M., Heinrich, M., Yamato, J.: A realtime multimodal system for analyzing group meetings by combining face pose tracking and speaker diarization. In: Proceedings of ICMI 2008 - International Conference on Multimodal Interfaces (2008)
Rienks, R., Nijholt, A., Barthelmess, P.: Pro-active meeting assistants: Attention please? In: Proceedings of the 5th workshop on Social Intelligence Design (2006)
Ehlen, P., Fernandez, R., Frampton, M.: Designing and evaluating meeting assistants, keeping humans in mind. In: Popescu-Belis, A., Stiefelhagen, R. (eds.) MLMI 2008. LNCS, vol. 5237, pp. 309–314. Springer, Heidelberg (2008)
Popescu-Belis, A., Boertjes, E., Kilgour, J., Poller, P., Castronovo, S., Wilson, T., Jaimes, A., Carletta, J.: The amida content linking device: Just-in-time document retrieval in meetings. In: Popescu-Belis, A., Stiefelhagen, R. (eds.) MLMI 2008. LNCS, vol. 5237, pp. 272–283. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
op den Akker, R., Hofs, D., Hondorp, H., op den Akker, H., Zwiers, J., Nijholt, A. (2009). Supporting Engagement and Floor Control in Hybrid Meetings. In: Esposito, A., VÃch, R. (eds) Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions. Lecture Notes in Computer Science(), vol 5641. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03320-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-03320-9_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03319-3
Online ISBN: 978-3-642-03320-9
eBook Packages: Computer ScienceComputer Science (R0)