Skip to main content
SpringerLink
Log in

Collectively Sharing People’s Visual and Auditory Capabilities: Exploring Opportunities and Pitfalls

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

The sharing economy, virtualizing the peer-to-peer-based sharing of physical or logical resources such as cars, houses or people’s spare time with digital technologies, has recently attracted a great deal of attention. Investigating the feasibility of sharing other types of physical resources, particularly human bodies, offers promising opportunities with which to expand the current scope of the sharing economy. This study explores the opportunities and pitfalls of collectively sharing parts of the body, particularly human eyes and ears, to build and explore novel services based on wearable cameras and microphones. The paper proposes a novel concept named CollectiveEyes to realize a digital platform for collectively sharing human seeing and hearing, then presents a prototype platform of CollectiveEyes to demonstrate key features of CollectiveEyes. The essential contribution of the paper is that it reveals potential opportunities and pitfalls of collectively sharing physical capabilities of the human eyes and ears for building a novel digital platform and diverse services on the platform. We mainly found the following two promising research directions extracted from the current study: (1) CollectiveEyes enhances human thinking abilities through multiple visual perspectives; (2) nonhuman’s visual perspectives offer potential opportunities to access more diverse perspectives for augmenting human thinking abilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.youtube.com/user/googleglass.

  2. https://www.youtube.com/watch?v=4EvNxWhskf8.

  3. We will explain the expertise of a design fiction expert is described in Sect. 5.

  4. https://www.airbnb.com/.

  5. https://www.uber.com/.

  6. https://www.taskrabbit.com/.

  7. https://www.getfove.com/.

  8. https://www.tobii.com/.

  9. https://unity3d.com/.

  10. https://www.google.co.jp/earth/.

  11. R0: Description: Revisiting, R1: Reflective Description: Revisiting with Explanation, R2: Dialogic Reflection: Exploring Relationships.

  12. The participants are different from the experiment described in Sect. 4.

  13. The participants are different from those in the experiments shown in Sects. 4 and 6.1.

  14. We used the sound of a bell in the experiment.

References

  1. Botsman R, Rogers R. What’s mine is yours: the rise of collaborative consumption. New York: Harper Business; 2010.

    Google Scholar 

  2. Haidt L. Nonprofits imagine a better sharing economy. https://www.forums.techsoup.org/cs/community/b/tsblog/archive/2014/05/14/nonprofits-imagine-a-better-sharing-economy.aspx. Accessed 1 July 2020.

  3. Hamari J, Sjöklint M, Ukkonen A. The sharing economy: why people participate in collaborative consumption. J Assoc Inf Sci Technol. 2015;67(9):2047–59. https://doi.org/10.1002/asi.23552.

    Article  Google Scholar 

  4. Nakajima T, Lehdonvirta V. Designing motivation using persuasive ambient mirrors. Pers Ubiquitous Comput. 2013;17(1):107–26. https://doi.org/10.1007/s00779-011-0469-y.

    Article  Google Scholar 

  5. Sakamoto M, Nakajima T, Alexandrova T. Enhancing values through virtuality for intelligent artifacts that influence human attitude and behavior. Multimed Tools Appl. 2015;74(24):11537–68. https://doi.org/10.1007/s11042-014-2250-5.

    Article  Google Scholar 

  6. Tharp BM, Tharp SM. Discursive design: critical, speculative, and alternative things. Cambridge: The MIT Press; 2019.

    Book  Google Scholar 

  7. Kawsar F, Min C, Mathur A, Montanari A. Earables for personal-scale behavior analytics. IEEE Pervasive Comput. 2018;17(3):83–9. https://doi.org/10.1109/MPRV.2018.03367740.

    Article  Google Scholar 

  8. Ren X. Rethinking the relationship between humans and computers. Computer. 2016;49(8):104–8. https://doi.org/10.1109/MC.2016.253.

    Article  Google Scholar 

  9. Niksirat KS, Sarcar S, Sun H, Law EL-C, Clemmensen T, Bardzell J, Oulasvirta A, Silpasuwanchai C, Light A, Ren X. Approaching engagement towards human-engaged computing. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems (CHI EA ‘18) , Paper SIG14. ACM, New York, NY, USA. 2018. https://doi.org/10.1145/3170427.3185364.

  10. Boluk S, Lemieux P. Metagaming: playing, competing, spectating, cheating, trading, making, and breaking videogames. Minneapolis: University of Minnesota Press; 2017.

    Book  Google Scholar 

  11. Dunne A, Raby F. Speculative everything: design, fiction, and social dreaming. Cambridge: The MIT Press; 2013.

    Google Scholar 

  12. Ishizawa F, Sakamoto M, Nakajima T. Extracting intermediate-level design knowledge for speculating digital-physical hybrid alternate reality experiences. Multimed Tools Appl. 2018;77(16):21329–70. https://doi.org/10.1007/s11042-017-5595-8.

    Article  Google Scholar 

  13. Kimura R, Nakajima T. Sharing collective human’s eyesights towards reflective thinking, In: Proceedings of the 17th international conference on mobile and ubiquitous multimedia; 2018. p. 341–9. https://doi.org/10.1145/3282894.3289724.

  14. Kilteni C, Maselli A, Kording KP, Slater M. Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception. Front Hum Neurosci. 2015;9:141. https://doi.org/10.3389/fnhum.2015.00141.

    Article  Google Scholar 

  15. Malpass M. Critical design in context: history, theory, and practices. New York: Bloomsbury USA Academic; 2017.

    Book  Google Scholar 

  16. Odom W, Zimmerman J, Davidoff S, Forlizzi J, Dey AK, Lee MK. A fieldwork of the future with user enactments. In: Proceedings of the designing interactive systems conference (DIS ‘12). ACM, New York, NY, USA; 2012. p. 338–47. https://doi.org/10.1145/2317956.2318008.

  17. Evans DS, Schmalensee R. Matchmakers: the new economics of multisided platforms. Boston: Harvard Business Review Press; 2016.

    Google Scholar 

  18. Kittur A, Chi EH, Suh B. Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA; 2008. p. 453–6. https://doi.org/10.1145/1357054.1357127.

  19. Bigham JP, Jayant C, Ji H, Little G, Miller A, Miller RC, Miller R, Tatarowicz A, White B, White S, Yeh T. VizWiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST ‘10). ACM, New York, NY, USA; 2010. p. 333–42. https://doi.org/10.1145/1866029.1866080.

  20. Lasecki W, Miller C, Sadilek A, Abumoussa A, Borrello D, Kushalnagar R, Bigham J. Real-time captioning by groups of non-experts. In: Proceedings of the 25th annual ACM symposium on user interface software and technology (UIST '12). ACM, New York, NY, USA; 2012. p. 23–34. https://doi.org/10.1145/2380116.2380122.

  21. Sakamoto M, Nakajima T, Akioka S. Gamifying collective human behavior with gameful digital rhetoric. Multimedia Tools Appl. 2017;76(10):12539–81. https://doi.org/10.1007/s11042-016-3665-y.

    Article  Google Scholar 

  22. Liu Y, Alexandrova T, Nakajima T. Using stranger as sensors: temporal and geo-sensitive question answering via social media. In: Proceedings of the 22nd international conference on World Wide Web (WWW '13). ACM, New York, NY, USA; 2013. p. 803–14. https://doi.org/10.1145/2488388.2488458.

  23. Schön D. The reflective practitioner: how professionals think in action. New York: Basic Books; 1983.

    Google Scholar 

  24. Carayannis EG. Encyclopedia of creativity, invention, innovation and entrepreneurship. New York: Springer; 2013.

    Book  Google Scholar 

  25. Tigelaar DEH, Dolmans D, de Grave WS, Wolfhagen IHAP, van der Vleuten CPM. Portfolio as a tool to stimulate teachers’ reflections. Med Teach. 2006;28(3):277–82. https://doi.org/10.1080/01421590600607013.

    Article  Google Scholar 

  26. Ghajargar M, Wilberg M, Stolterman E. Designing IoT systems that support reflective thinking: a relational approach. Int J Des [Online] 12:1. 2018. https://www.ijdesign.org/index.php/IJDesign/article/view/2972/801

  27. Kasahara S, Rekimoto J. JackIn: integrating first-person view with out-of-body vision generation for human–human augmentation. In: Proceedings of the 5th augmented human international conference (AH '14). ACM, New York, NY, USA; 2014. (article 46 , 8 pages).https://doi.org/10.1145/2582051.2582097

  28. Kasahara S, Ando M, Suganuma K, Rekimoto J. Parallel eyes: exploring human capability and behaviors with paralleled first person view sharing. In: ACM SIGGRAPH 2016 VR Village (SIGGRAPH ‘16). ACM, New York, NY, USA; 2016. https://doi.org/10.1145/2929490.2929495(article 16, 2 pages).

  29. Procyk J, Neustaedter C, Pang C, Tang A, Judge TK. Exploring video streaming in public settings: shared geocaching over distance using mobile video chat. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘14). ACM, New York, NY, USA; 2014. p. 2163–2172. https://doi.org/10.1145/2556288.2557198.

  30. Ikeuchi K, Otsuka T, Yoshii A, Sakamoto M, Nakajima T. KinecDrone: enhancing somatic sensation to fly in the sky with Kinect and AR. Drone. In: Proceedings of the 5th augmented human international conference (AH ‘14). ACM, New York, NY, USA; 2014. https://doi.org/10.1145/2582051.2582104(article 53, 2 pages)

  31. Cain W, Bell J, Cheng C. Implementing robotic telepresence in a synchronous hybrid course. In: Proceedings of the IEEE 16th international conference on advanced learning technologies; 2016. p. 171–5. https://doi.org/10.1109/ICALT.2016.79.

  32. Tsui KM, Desai M, Yanco HA, Uhlik C. Exploring use cases for telepresence robots. In: Proceedings of the 6th international conference on human–robot interaction (HRI ‘11). ACM, New York, NY, USA; 2011. p. 11–8. https://doi.org/10.1145/1957656.1957664.

  33. https://www.techcrunch.com/2017/12/13/china-cctv-bbc-reporter/. Accessed 1 July 2020

  34. Kimura R, Nakajima T. A ubiquitous computing platform for virtualizing collective human eyesight and hearing capabilities. In: Proceedings of the 10th international symposium on ambient intelligence; 2019.

  35. Heikkilä H, Räihä K-J. Speed and accuracy of gaze gestures. J Eye Mov Res. 2010. https://doi.org/10.16910/jemr.3.2.1.

    Article  Google Scholar 

  36. Singh H, Singh J. Real-time eye blink and wink detection for object selection in HCI systems. J Multimodal User Interfaces. 2018;12(1):55–65. https://doi.org/10.1007/s12193-018-0261-7.

    Article  Google Scholar 

  37. England D, Schiphorst T, Bryan-Kinns N. Curating the digital: space for art and interaction. Berlin: Springer; 2016.

    Book  Google Scholar 

  38. Dewey J. How we think: a restatement of the relation of reflective thinking to the educative process. New York: D.C. Heath and Company; 1933.

    Google Scholar 

  39. Åkerström A. Comparison between gaze and moving objects in videos for smooth pursuit eye movement evaluation. Lund: Department of Electrical and Information Technology, Lund University; 2015.

    Google Scholar 

  40. Ghosh S, Winston L, Panchal N, Kimura-Thollander P, Hotnog J, Cheong D, Reyes G, Abowd GD. NotifiVR: exploring interruptions and notifications in virtual reality. IEEE Trans Vis Comput Gr. 2018;24(4):1447–566. https://doi.org/10.1109/TVCG.2018.2793698.

    Article  Google Scholar 

  41. Weinschenk S. 100 things every designer needs to know about people. Berkeley: New Riders Press; 2011.

    Google Scholar 

  42. Wang J, Shen HT, Song J, Ji J. Hashing for similarity search: a survey, CoRR. 2014. arXiv:1408.2927.

  43. Mahajan D, Girshick R, Ramanathan V, Paluri M, Van Der Maaten L. Advancing state-of-the-art image recognition with deep learning on hashtags. 2018. https://www.code.fb.com/applied-machine-learning/advancing-state-of-the-art-image-recognition-with-deep-learning-on-hashtags/. Accessed 1 July 2020.

  44. von Ahn L, Dabbish L. Designing games with a purpose. Commun ACM. 2008;51(8):58–67. https://doi.org/10.1145/1378704.1378719.

    Article  Google Scholar 

  45. Mitake H, Yamada H, Nakajima T. Looking into the Peak memory consumption of epoch-based reclamation in scalable in-memory database systems. In: Proceedings of the 30th international conference on databases and expert systems applications (DEXA2019); 2019.

  46. Ousterhout J, Gopalan A, Gupta A, Kejriwal A, Lee C, Montazeri B, Ongaro D, Park SJ, Qin H, Rosenblum M, Rumble S, Stutsman R, Yang S. The RAMCloud storage system. ACM Trans Comput Syst. 2015. https://doi.org/10.1145/2806887.

    Article  Google Scholar 

  47. Toyama S, Al Sada M, Nakajima T. VRowser: a virtual reality parallel web browser. In: Proceedings of the 10th international conference on virtual, augmented and mixed reality; 2018.

  48. Giaccardi E, Stappers PJ. Research through design. Chapter 43: the encyclopedia of human–computer interaction. 2nd ed. Aarhus: Interaction Design Foundation; 2017.

    Google Scholar 

  49. Braun V, Clarke V, Hayfield N, Terry G. Thematic analysis. In: Liamputtong P, editor. Handbook of research methods in health social sciences. Singapore: Springer; 2018.

    Google Scholar 

  50. Nowell LS, Norris JM, White DE, Modules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods. 2017;16:1–13.

    Article  Google Scholar 

  51. Lucero A. Using affinity diagrams to evaluate interactive prototypes. In: Abascal J, Barbosa S, Fetter M, Gross T, Palanque P, Winckler M, editors. Human–computer interaction—INTERACT 2015. INTERACT 2015. Lecture notes in computer science, vol. 9297. Cham: Springer; 2015.

    Google Scholar 

  52. Bateson G. A Theory of play and fantasy. In: Bateson G, editor. Steps to a ecology of mind. New York: Ballantine Books; 1972. p. 177–193.

    Google Scholar 

  53. Tokyo cute, strange, weird travel tips! Suspiria horror bar, Pokemon cafe, Line Friends, Gundam. https://www.youtube.com/watch?time_continue=30&v=VuTPA3uk0h0. Accessed 1 July 2020.

  54. Gonzatto RF, van Amstel FMC, Merkle LE, Hartmann T. The ideology of the future in design fiction. Digit Creat. 2013;24(1):36–45.

    Article  Google Scholar 

  55. Blythe M, Encinas E. Research fiction and thought experiments in design, foundations and trends®. Hum Comput Interact. 2018;12(1):1–105. https://doi.org/10.1561/1100000070.

    Article  Google Scholar 

  56. Kirby D. The future is now: diegetic prototypes and the role of popular films in generating real-world technological development. Soc Stud Sci. 2010;40(1):41–70.

    Article  Google Scholar 

  57. Maeda K. Mirrorworlds. https://www.blog.leapmotion.com/mirrorworlds/. Accessed 1 July 2020.

  58. Lindlbauer D, Wilson AD. 2018. Remixed reality: manipulating space and time in augmented reality. In: Proceedings of the 2018 CHI conference on human factors in computing systems (CHI ‘18). ACM, New York, NY, USA, paper 129; 2018. https://doi.org/10.1145/3173574.3173703.

  59. Saraiji MHDY, Sugimoto S, Fernando CL, Minamizawa K, Tachi S. 2016. Layered telepresence: simultaneous multi presence experience using eye gaze based perceptual awareness blending. In: ACM SIGGRAPH 2016 emerging technologies (SIGGRAPH '16). ACM, New York, NY, USA, Article 14, 2 pages; 2016. https://doi.org/10.1145/2929464.2929467.

  60. Al-Sada M, Höglund T, Khamis M, Urbani J, Nakajima T. Orochi: investigating requirements and expectations for multipurpose daily used supernumerary robotic limbs. In: Proceedings of the 10th augmented human international conference 2019 (AH2019). ACM, New York, NY, USA; 2019. https://doi.org/10.1145/3311823.3311850(article 37, 9 pages)

  61. Hase S. Beatless. Tokyo: Kadokawa Publishing; 2012.

    Google Scholar 

  62. Yamabe T, Lehdonvirta V, Ito H, Soma H, Kimura H, Nakajima T. Applying pervasive technologies to create economic incentives that alter consumer behavior. In: Proceedings of the 11th international conference on Ubiquitous computing (UbiComp ‘09). ACM, New York, NY, USA; 2009. p. 175–84. https://doi.org/10.1145/1620545.1620573.

  63. Li I, Dey A, Forlizzi J. A stage-based model of personal informatics systems. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘10). ACM, New York, NY, USA; 2010. p. 557–66. https://doi.org/10.1145/1753326.1753409.

  64. Fleck R, Fitzpatrick G. Reflecting on reflection: framing a design landscape. In: Proceedings of the 22nd conference of the computer–human interaction special interest group of Australia on computer–human interaction (OZCHI ‘10). ACM, New York, NY, USA; 2010. p. 216–23. https://doi.org/10.1145/1952222.1952269.

  65. Bell G, Blythe M, Sengers P. Making by making strange: Defamiliarization and the design of domestic technologies. ACM Trans Comput Hum Interact. 2005;12(2):149–73. https://doi.org/10.1145/1067860.1067862.

    Article  Google Scholar 

  66. Shklovsky V. Art as technique. In: Davis RC, editor. Contemporary literary criticism. Modernism through poststructuralism. New York: Longman Press; 1917.

    Google Scholar 

  67. Chang W-W, Giaccardi E, Chen L-L, Liang R-H. “Interview with Things”: a first-thing perspective to understand the scooter’s everyday socio-material network in Taiwan. In: Proceedings of the 2017 conference on designing interactive systems (DIS ’17). Association for Computing Machinery, New York, NY, USA; 2017. p. 1001–12. https://doi.org/10.1145/3064663.3064717.

  68. Coulton P, Lindley JG. More-than human centered design: considering other things. Des J Int J Asp Des. 2019. https://doi.org/10.1080/14606925.2019.1614320.

    Article  Google Scholar 

  69. Cruickshank L, Trivedi NL. Beyond human-centered design: supporting a new materiality in the internet of things, or how to design when a toaster is one of your users. Des J Int J Asp Des. 2017. https://doi.org/10.1080/14606925.2017.1349381.

    Article  Google Scholar 

  70. Harman G. Object-oriented ontology: a new theory of everything. London: Pelican; 2018.

    Google Scholar 

  71. Latour B. Reassembling the social: an introduction to actor-network-theory. Oxford: Oxford University Press; 2007.

    Google Scholar 

  72. Morton T. Hyperobjects: philosophy and ecology after the end of the world. Minneapolis: University of Minnesota Press; 2013.

    Google Scholar 

  73. Yaco L, Haber K. The science of the X-men. New York: I Books/Marvel; 2004.

    Google Scholar 

  74. Sunstein CR. Simpler: the future of government. New York: Simon & Schuster; 2013.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Waseda University, Tokyo, Japan

    Risa Kimura & Tatsuo Nakajima

Authors
  1. Risa Kimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tatsuo Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Risa Kimura.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kimura, R., Nakajima, T. Collectively Sharing People’s Visual and Auditory Capabilities: Exploring Opportunities and Pitfalls. SN COMPUT. SCI. 1, 298 (2020). https://doi.org/10.1007/s42979-020-00313-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-020-00313-w

Keywords

Search

Navigation