Shunichi Kasahara
Jun Rekimoto
ABSTRACT
JackIn is a new human-human communication framework for connecting two or more people. With first-person view video streaming from a person (called Body) wearing a transparent head-mounted display and a head-mounted camera, the other person (called Ghost) participates in the shared first-person view. With JackIn, people's activities can be shared and assistance or guidance can be given through other peoples expertise. This can be applied to daily activities such as cooking lessons, shopping navigation, education in craft-work or electrical work, and sharing experiences of sporting and live events. For a better viewing experience with frist-person view, we developed the out-of-body view in which first-person images are integrated to construct a scene around a Body, and a Ghost can virtually control the viewpoint to look around the space surrounding the Body. We also developed a tele-pointing gesture interface. We conducted an experiment to evaluate how effective this framework is and found that Ghosts can understand the spatial situation of the Body.
- Tachi, S., Arai, H., Maeda, T., Oyama, E., Tsunemoto, N. and Inoue, Y., "Tele-existence in real world and virtual world," Advanced Robotics, 1991.193--198Google Scholar
- Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proc, CHI 2013. ACM Press (2013), pp.179--188. Google ScholarDigital Library
- Tecchia, F., Alem, L. and Huang, W. 2012. 3D helping hands: a gesture based MR system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI '12). pp.323--328. Google ScholarDigital Library
- Kuzuoka, H., Spatial workspace collaboration: a SharedView video support system for remote collaboration capability, CHI 1992, pp.533--540, 1992. Google ScholarDigital Library
- Fussell, S. R., Setlock, L. D. and Kraut, R. E. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks, CHI 2003, pp. 513--520. 2003. Google ScholarDigital Library
- Kirk, D., Rodden, T. and Fraser, D. S. 2007. Turn it this way: grounding collaborative action with remote gestures. CHI 2007, pp.1039--1048, 2007. Google ScholarDigital Library
- Steve M., Telepointer: Hands-Free Completely Self Contained Wearable Visual Augmented Reality without Headwear and without any Infrastructural Reliance. In Proceedings of ISWC '00, 177--178. 2000. Google ScholarDigital Library
- Bauer, M., Kortuem, G. Segall, Z. "Where Are You Pointing At?" A Study of Remote Collaboration in a Wearable Videoconference System. ISWC 1999, pp.151--.158, 1999. Google ScholarDigital Library
- Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H. and Billinghurst, M. 2004. Remote Collaboration using a Shoulder-Worn Active Camera/Laser. In Proc. ISWC 2004. 62--69. Google ScholarDigital Library
- Gauglitz, S., Lee, C., Turk M. and Hllerer, T. Integrating the physical environment into mobile remote collaboration. In Proc. MobileHCI '12. ACM Press (2012) 241--250. Google ScholarDigital Library
- Ou, J., Fussell, S. R., Chen, X., Setlock, L. D. and Yang, J. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proc. ICMI '03. ACM Press (2003), 242--249. Google ScholarDigital Library
- Snavely, N., Seitz, S. and Szeliski, R. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (TOG) 2006. 83--846 Google ScholarDigital Library
- SmartAR (http://www.sony.net/SonyInfo/News/Press/201105/11-058E/index.html)Google Scholar
- LUMUS (http://www.lumus-optical.com/)Google Scholar
- Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A. and Fitzgibbon, A. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. UIST '11. ACM Press (2011) 559--568. Google ScholarDigital Library
- L. Cheng and J. Robinson, Dealing with speed and robustness issues for video-based registration on a wearable computing platform. In Proc. ISWC98, pages 84.91, 1998. Google ScholarDigital Library
- Baudisch, P. and Rosenholtz, R. Halo: a technique for visualizing off-screen objects. In Proc. CHI '03. ACM Press (2013) 481--488. Google ScholarDigital Library
- Benson, Alan J. (2002). "Motion Sickness". In Kent B. Pandoff and Robert E. Burr. Medical Aspects of Harsh Environments 2. Washington, D.C.: Borden Institute. pp. 1048--1083. Retrieved 4 Dec 2012.Google Scholar
- Goldberg, K., Song, D., Khor, Y., Pescovitz, D., Levandowski, A., Himmelstein, J., Shih, J., Ho, A., Paulos, E. and Donath, J. Collaborative Online Teleoperation with Spatial Dynamic Voting and a Human "Tele-Actor" In Proc. ICRA '02. 1179--1184.Google Scholar
- Tamaki, E., Miyaki, T. and Rekimoto, J. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proc. CHI2011, ACM Press (2011) 543--552. Google ScholarDigital Library
- Leap Motion (https://www.leapmotion.com/)Google Scholar
- William Gibson, Neuromancer, Ace Science Fiction, 1984.Google Scholar
- GoPro Official Website, http://gopro.com/Google Scholar
- Leonard, J. J. and Durrant-whyte, H. F., Simultaneous map building and localization for an autonomous mobile robot. Proceedings IROS'91. IEEE/RSJ International Workshop, pp.1442--1447, 1991.Google ScholarCross Ref
- Rekimoto, J. "Traxion: A Tactile Interaction Device with Virtual Force Sensation, ACM UIST 2013, pp.427--432, 2013. Google ScholarDigital Library
- Gerhard, R., Ethan, E., and Tom, W., D. Semi-automatic Annotations in Unknown Environments. In Proceeding of ISMAR '2007, pp. 67--70, 2007.Google Scholar
Index Terms
- JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation
Recommendations
JackIn space: designing a seamless transition between first and third person view for effective telepresence collaborations
AH '17: Proceedings of the 8th Augmented Human International ConferenceTraditional telepresence systems only supported first person view and users had difficulty in recognizing the surrounding situation of their remote workspace. JackIn Space is a telepresence system that solves this problem by seamlessly integrating first ...
JackIn head: an immersive human-human telepresence system
SA '15: SIGGRAPH Asia 2015 Emerging TechnologiesSharing full immersive experience in real-time has been the one of ultimate goals of telecommunication. Possible application can include various applications such as entertainment, sports viewing, education, social network and professional assistance. ...
Augmented Human: Augmented Reality and Beyond
AltMM'18: Proceedings of the 3rd International Workshop on Multimedia Alternate RealitiesWill Augmented Reality (AR) allow us to access digital information, experience others' stories, and thus explore alternate realities? AR has recently attracted attention again due to the rapid advances in related multimedia technologies as well as ...
Comments