Skip to main content
SpringerLink
Log in

Recognition of audible disruptive behavior from people with dementia

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Frequently, people with dementia exhibit abnormal behaviors that may cause self-injury or burden their caregivers. Some audible manifestations of these problematic behaviors are of vocal nature (e.g., shouting, mumbling, or cursing), others are environmental sounds (e.g., tapping or slamming). The timely detection of these behaviors could enact non-pharmacological interventions which in turn can assist caregivers or prevent escalation of the disruption with other fellow residents in nursing homes. We conducted a field study in a geriatric residence to gather naturalistic data. With the participation of five residents for 203 h of observation and of the 242 incidents of problematic behaviors were registered, 85% of them had a distinctive auditory manifestation. We used a combination of standard speech detection techniques, along with a novel environmental sound recognition methodology based on the entropy of the signal. We conducted experiments using realistic data, i.e., audio immersed in natural background noise. Based on classification results with F1 score above 87%, we conclude that audible cues can be used to enact non-pharmacological interventions aimed at reducing problematic behaviors, or mitigating their negative impact.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Alberdi A, Aztiria A, Basarab A (2016) On the early diagnosis of alzheimer’s disease from multimodal signals: a survey. Artif Intell Med 71:1–29. https://doi.org/10.1016/j.artmed.2016.06.003. http://www.sciencedirect.com/science/article/pii/S0933365716300732

    Article  Google Scholar 

  2. Beck C, Richards K, Lambert C, Doan R, Landes RD, Whall A, Algase D, Kolanowski A, Feldman Z (2011) Factors associated with problematic vocalizations in nursing home residents with dementia. Gerontologist 51(3):389–405

    Article  Google Scholar 

  3. Beltrán J, Chávez E, Favela J (2015) Scalable identification of mixed environmental sounds, recorded from heterogeneous sources. Pattern Recogn Lett 68:153–160

    Article  Google Scholar 

  4. Beltrán J., Navarro R, Chávez E., Favela J, Soto-Mendoza V, Ibarra C (2014) Detecting disruptive vocalizations for ambient assisted interventions for dementia. In: Ambient assisted living and daily activities, Springer, pp 356–363

  5. Bronkhorst AW (2000) The cocktail party phenomenon: a review of research on speech intelligibility in multiple-talker conditions. Acta Acustica united with Acustica 86(1):117–128 . (2000-01-01T00:00:00). http://www.ingentaconnect.com/content/dav/aaua/2000/00000086/00000001/art00016

    Google Scholar 

  6. Burns K, Jayasinha R, Tsang R, Brodaty H (2012) Behaviour management: a guide to good practice dementia collaborative research centre - assessment and better care

  7. Camarena-Ibarrola A, Chávez E, Tellez ES (2009) Progress in pattern recognition, image analysis, computer vision, and applications: 14th Iberoamerican conference on pattern recognition, CIARP 2009, Guadalajara, Jalisco, Mexico, November 15-18, 2009. Proceedings, chap. Robust Radio Broadcast Monitoring Using a Multi-Band Spectral Entropy Signature. Springer, Berlin, pp 587–594

  8. Chen F, Adcock J, Krishnagiri S (2008) Audio privacy: reducing speech intelligibility while preserving environmental sounds. In: Proceedings of the 16th ACM international conference on multimedia, MM ’08. https://doi.org/10.1145/1459359.1459472. ACM, New York, pp 733–736

  9. Cohen-Mansfield J (1997) Conceptualization of agitation: results based on the cohen-mansfield agitation inventory and the agitation behavior mapping instrument. Int Psychogeriatr 8(S3):309–315

    Article  Google Scholar 

  10. Cohen-Mansfield J (1999) Assessment of agitation in older adults. Wiley, Hoboken

    Google Scholar 

  11. Cummings J, Mega M, Gray K, Rosenberg-Thompson S, Carusi D, Gornbein J (2014) The neuropsychiatric inventory: comprehensive assessment of psychopathology in dementia. neurology 1994; 44: 2308–2314. Dement Geriatr Cogn Disord Extra 4:131–139

    Article  Google Scholar 

  12. Cummings JL, Arciniegas DB, Brooks BR, Herndon RM, Lauterbach EC, Pioro EP, Robinson RG, Scharre DW, Schiffer RB, Weintraub D (2006) Defining and diagnosing involuntary emotional expression disorder. CNS Spectr 11(S6):1–11

    Article  Google Scholar 

  13. Dennis J, Tran H, Chng ES (2013) Image feature representation of the subband power distribution for robust sound event classification. IEEE Trans Audio Speech Lang Process 21(2):367–377

    Article  Google Scholar 

  14. Dennis J, Tran HD, Chng ES (2013) Overlapping sound event recognition using local spectrogram features and the generalised hough transform. Pattern Recogn Lett 34(9):1085–1093

    Article  Google Scholar 

  15. Desai AK, Desai FG (2014) Management of behavioral and psychological symptoms of dementia. Current Geriatrics Reports 3(4):259–272

    Article  Google Scholar 

  16. Fick W, van der Borgh J, Jansen S, Koopmans R (2014) The effect of a lollipop on vocally disruptive behavior in a patient with frontotemporal dementia: a case-study. Int Psychogeriatr 26(12):2023–2026

    Article  Google Scholar 

  17. Gao B, Woo WL (2014) Wearable audio monitoring: content-based processing methodology and implementation. IEEE Transactions on Human-Machine Systems 44(2):222–233

    Article  Google Scholar 

  18. Gu J, Gao B, Chen Y, Jiang L, Gao Z, Ma X, Ma Y, Woo WL, Jin J (2017) Wearable social sensing: content-based processing methodology and implementation. IEEE Sensors J 17(21):7167–7176

    Article  Google Scholar 

  19. von Gunten A, Favre M, Gurtner C, Abderhalden C (2011) Vocally disruptive behavior (vdb) in the institutionalized elderly: a naturalistic multiple case report. Arch Gerontol Geriatr 52(3):e110–e116

    Article  Google Scholar 

  20. Hearst MA (1998) Support vector machines. IEEE Intell Syst 13(4):18–28. https://doi.org/10.1109/5254.708428

    Article  Google Scholar 

  21. Huang X, Baker J, Reddy R (2014) A historical perspective of speech recognition. Commun ACM 57 (1):94–103

    Article  Google Scholar 

  22. Jalbert JJ, Daiello LA, Lapane KL (2008) Dementia of the alzheimer type. Epidemiol Rev 30(1):15–34

    Article  Google Scholar 

  23. König A., Sacco G, Bensadoun G, Bremond F, David R, Verhey F, Aalten P, Robert P, Manera V (2015) The role of information and communication technologies in clinical trials with patients with alzheimer’s disease and related disorders. Front Aging Neurosci 7:1–5

    Google Scholar 

  24. Krijnders J, Niessen M, Andringa T (2010) Sound event recognition through expectancy-based evaluation ofsignal-driven hypotheses. Pattern Recogn Lett 31(12):1552–1559

    Article  Google Scholar 

  25. Krishnamurthy N, Hansen JH (2009) Babble noise: modeling, analysis, and applications. IEEE Trans Audio Speech Lang Process 17(7):1394–1407

    Article  Google Scholar 

  26. Liaqat D, Nemati E, Rahman M, Kuang J (2017) A method for preserving privacy during audio recordings by filtering speech. In: Life sciences conference (LSC), 2017 IEEE, IEEE, pp 79–82

  27. Mahesha P, Vinod D (2012) Feature based classification of dysfluent and normal speech. In: Proceedings of the Second international conference on computational science, engineering and information technology, ACM, pp 594–597

  28. Mitrović D, Zeppelzauer M, Breiteneder C (2010) Features for content-based audio retrieval. Adv Comput 78:71–150

    Article  Google Scholar 

  29. Navarretta C (2014) The automatic identification of the producers of co-occurring communicative behaviours. Cogn Comput 6(4):689–698

    Article  Google Scholar 

  30. Navarro RF, Rodriguez M, Favela J (2014) Intervention tailoring in augmented cognition systems for elders with dementia. IEEE Journal of Biomedical and Health Informatics 18(1):361–367

    Article  Google Scholar 

  31. O’Shaughnessy D (2000) Speech communications: human and machine. Institute of Electrical and Electronics Engineers. https://books.google.com.mx/books?id=yHJQAAAAMAAJ

  32. Oshima C, Itou N, Nishimoto K, Yasuda K, Hosoi N, Yamashita H, Nakayama K, Horikawa E (2013) A music therapy system for patients with dementia who repeat stereotypical utterances. Information and Media Technologies 8(2):605–616

    Google Scholar 

  33. Peintner B, Jarrold W, Vergyriy D, Richey C, Tempini MLG, Ogar J (2008) Learning diagnostic models using speech and language measures. In: Conference proceedings : ... Annual international conference of the ieee engineering in medicine and biology society. IEEE Engineering in Medicine and Biology Society. Annual Conference, pp 4648–51

  34. Potamitis I, Ganchev T (2008) Multimedia services in intelligent environments: advanced tools and methodologies, chap. Generalized recognition of sound events: approaches and applications. Springer, Berlin, pp 41–79

    Google Scholar 

  35. Qawaqneh Z, Mallouh AA, Barkana BD (2017) Deep neural network framework and transformed mfccs for speaker’s age and gender classification. Knowl-Based Syst 115:5–14

    Article  Google Scholar 

  36. Rabbitt SM, Kazdin AE, Scassellati B (2015) Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clin Psychol Rev 35:35–46

    Article  Google Scholar 

  37. Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  38. Rincon E, Beltran J, Tentori M, Favela J, Chavez E (2013) A context-aware baby monitor for the automatic selective archiving of the language of infants. In: 2013 mexican international conference on computer science (ENC), IEEE, pp 60–67

  39. Sakar BE, Isenkul ME, Sakar CO, Sertbas A, Gurgen F, Delil S, Apaydin H, Kursun O (2013) Collection and analysis of a parkinson speech dataset with multiple types of sound recordings. IEEE Journal of Biomedical and Health Informatics 17(4):828–834

    Article  Google Scholar 

  40. Serizel R, Bisot V, Essid S, Richard G (2018) Acoustic features for environmental sound analysis. In: Computational analysis of sound scenes and events, Springer, pp 71–101

  41. Sezgin MC, Gunsel B, Kurt GK (2012) Perceptual audio features for emotion detection. EURASIP Journal on Audio, Speech, and Music Processing 2012(1):1–21

    Article  Google Scholar 

  42. Smith D, Ma L, Ryan N (2006) Acoustic environment as an indicator of social and physical context. Pers Ubiquit Comput 10(4):241–254

    Article  Google Scholar 

  43. Thomas C, Keselj V, Cercone N, Rockwood K, Asp E (2005) Automatic detection and rating of dementia of alzheimer type through lexical analysis of spontaneous speech. In: 2005 IEEE international conference Mechatronics and automation, vol 3. pp 1569–1574. https://doi.org/10.1109/ICMA.2005.1626789

  44. Wang D, Brown G (2006) Computational auditory scene analysis: principles, algorithms, and applications. Wiley-IEEE Press

  45. Yusupov A, Galvin JE (2014) Vocalization in dementia: a case report and review of the literature. Case reports in Neurology 6(1):126–133

    Article  Google Scholar 

  46. Zhuang X, Huang J, Potamianos G, Hasegawa-Johnson M (2009) Acoustic fall detection using gaussian mixture models and gmm supervectors. In: 2009. ICASSP 2009. IEEE international conference on acoustics, speech and signal processing, pp 69–72. https://doi.org/10.1109/ICASSP.2009.4959522

Download references

Author information

Authors and Affiliations

  1. CONACYT IPN-CITEDI, Av. Instituto Politécnico Nacional No. 1310 Nueva Tijuana, Tijuana, Mexico

    Jessica Beltrán

  2. Department of Industrial Engineering, University of Sonora, Hermosillo, Mexico

    René Navarro

  3. Computer Science Department, Center for Scientific Research and Higher Education at Ensenada (CICESE), Ensenada, Mexico

    Edgar Chávez, Jesús Favela & Catalina Ibarra

  4. Research Center on Applied Mathematics, Autonomous University of Coahuila, Saltillo, Mexico

    Valeria Soto-Mendoza

Authors
  1. Jessica Beltrán
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. René Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edgar Chávez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jesús Favela
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Valeria Soto-Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Catalina Ibarra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jessica Beltrán.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beltrán, J., Navarro, R., Chávez, E. et al. Recognition of audible disruptive behavior from people with dementia. Pers Ubiquit Comput 23, 145–157 (2019). https://doi.org/10.1007/s00779-018-01188-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-018-01188-8

Search

Navigation