Abstract
This paper describes how we recognize activities of daily living (ADLs) with our designed sensor device, which is equipped with heterogeneous sensors such as a camera, a microphone, and an accelerometer and attached to a user’s wrist. Specifically, capturing a space around the user’s hand by employing the camera on the wrist mounted device enables us to recognize ADLs that involve the manual use of objects such as making tea or coffee and watering plant. Existing wearable sensor devices equipped only with a microphone and an accelerometer cannot recognize these ADLs without object embedded sensors. We also propose an ADL recognition method that takes privacy issues into account because the camera and microphone can capture aspects of a user’s private life. We confirmed experimentally that the incorporation of a camera could significantly improve the accuracy of ADL recognition.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ahmad, F., Musilek, P.: A keystroke and pointer control input interface for wearable computers. In: Proc. PerCom 2006, pp. 2–11 (2006)
Bao, L., Intille, S.S.: Activity recognition from user-annotated acceleration data. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 1–17. Springer, Heidelberg (2004)
Blum, M., Pentland, A.S., Troster, G.: Insense: Interest-based life logging. IEEE Multimedia 13(4), 40–48 (2006)
Bouten, C.V., et al.: A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity. IEEE Trans. on Bio-Medical Engineering 44(3), 136–147 (1997)
Chen, J., Kam, A.H., Zhang, J., Liu, N., Shue, L.: Bathroom activity monitoring based on sound. In: Gellersen, H.-W., Want, R., Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol. 3468, pp. 47–61. Springer, Heidelberg (2005)
Clarkson, B., Mase, K., Pentland, A.: Recognizing user context via wearable sensors. In: Proc. ISWC 2000, pp. 69–75 (2000)
Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. on Pattern Analysis Machine Intelligence 25(5), 564–577 (2003)
Cowling, M.: Non-speech environmental sound recognition system for autonomous surveillance. Ph.D. Thesis, Griffith University, Gold Coast Campus (2004)
Fitzpatrick, P., Kemp, C.C.: Shoes as a platform for vision. In: Proc. ISWC 2003, pp. 231–234 (2003)
Froehlich, J.E., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., Patel, S.N.: HydroSense: Infrastructure-mediated single-point sensing of whole-home water activity. In: Proc. Ubicomp 2009, pp. 235–244 (2009)
Huynh, T., Schiele, B.: Towards less supervision in activity recognition from wearable sensors. In: Proc. ISWC 2006, pp. 3–10 (2006)
Intille, S.S., Tapia, E.M., Rondoni, J., Beaudin, J., Kukla, C., Agarwal, S., Bao, L., Larson, K.: Tools for studying behavior and technology in natural settings. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 157–174. Springer, Heidelberg (2003)
Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Proc. Advances in Neural Information Processing Systems, vol. 11, pp. 487–493 (1999)
Kasteren, T.V., Noulas, A., Englebienne, G., Krose, B.: Accurate activity recognition in a home setting. In: Proc. UbiComp 2008, pp. 1–9 (2008)
Lester, J., Choudhury, T., Kern, N., Borriello, G., Hannaford, B.: A hybrid discriminative/generative approach for modeling human activities. In: Proc. IJCAI 2005, pp. 766–772 (2005)
Lester, J., Choudhury, T., Borriello, G.: A practical approach to recognizing physical activities. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 1–16. Springer, Heidelberg (2006)
Logan, B., Healey, J., Philipose, M., Tapia, E.M., Intille, S.S.: A long-term evaluation of sensing modalities for activity recognition. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 483–500. Springer, Heidelberg (2007)
Lowe, D.G.: Distinctive image features from scale–invariant keypoints. Int’l Journal on Computer Vision 60(2), 91–110 (2004)
Lukowicz, P., Junker, H., et al.: WearNET: a distributed multi-sensor system for context aware wearables. In: Borriello, G., Holmquist, L.E. (eds.) UbiComp 2002. LNCS, vol. 2498, pp. 361–370. Springer, Heidelberg (2002)
Lukowicz, P., Ward, J., Junker, H., Stager, M., Troster, G., Atrash, A., Starner, T.: Recognizing workshop activity using body worn microphones and accelerometers. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 18–32. Springer, Heidelberg (2004)
Maekawa, T., Yanagisawa, Y., Kishino, Y., Kamei, K., Sakurai, Y., Okadome, T.: Object-blog system for environment-generated content. IEEE Pervasive Computing 7(4), 20–27 (2008)
Mayol, W.W., Murray, D.W.: Wearable hand activity recognition for event summarization. In: Proc. ISWC 2005, pp. 122–129 (2005)
Mihailidis, A., Carmichael, B., Boger, J.: The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home. IEEE Trans. on Info. Tech. in BioMedicine 8(3), 238–247 (2004)
Morikawa, S., Ito, K., Shibata, T.: A k-means VLSI processor and its application to autonomous area segmentation in images. IEIC Technical Report 106(342), 19–24 (2006)
Philipose, M., Fishkin, K.P., Perkowitz, M.: Inferring activities from interactions with objects. IEEE Pervasive Computing 3(4), 50–57 (2004)
Raina, R., Shen, Y., Ng, A.Y., McCallum, A.: Classification with hybrid generative/discriminative models. In: Proc. Advances in Neural Information Processing Systems, vol. 16 (2003)
Schiele, B., James, L.C.: Object recognition using multidimensional receptive field histograms. In: Proc. European Conference on Computer Vision, pp. 610–619 (1996)
Shi, Y., Huang, Y., Minnen, D., Bobick, A., Essa, I.: Propagation networks for recognition of partially ordered sequential action. In: Proc. CVPR 2004, vol. 2, pp. 862–869 (2004)
Starner, T., Schiele, B., Pentland, A.: Visual contextual awareness in wearable computing. In: Proc. ISWC 1998, pp. 50–57 (1998)
Swain, M.J., Ballard, D.H.: Color indexing. Int’l Journal of Computer Vision 7, 11–32 (1991)
Tapia, E.M., Intille, S.S., Larson, K.: Activity recognition in the home using simple and ubiquitous sensors. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 158–175. Springer, Heidelberg (2004)
Tapia, E.M., Intille, S.S., Larson, K.: Portable wireless sensors for object usage sensing in the home: challenges and practicalities. In: Schiele, B., Dey, A.K., Gellersen, H., de Ruyter, B., Tscheligi, M., Wichert, R., Aarts, E., Buchmann, A. (eds.) AmI 2007. LNCS, vol. 4794, pp. 19–37. Springer, Heidelberg (2007)
Welk, G., Differding, J.: The utility of the Digi-Walker step counter to assess daily physical activity patterns. Medicine & Science in Sports & Exercise 32(9), S481–S488 (2000)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Wu, J., Osuntogun, A., et al.: A scalable approach to activity recognition based on object use. In: Proc. ICCV 2007, pp. 1–8 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Maekawa, T. et al. (2010). Object-Based Activity Recognition with Heterogeneous Sensors on Wrist. In: Floréen, P., Krüger, A., Spasojevic, M. (eds) Pervasive Computing. Pervasive 2010. Lecture Notes in Computer Science, vol 6030. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12654-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-12654-3_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12653-6
Online ISBN: 978-3-642-12654-3
eBook Packages: Computer ScienceComputer Science (R0)