Skip to main content
SpringerLink
Log in

Toward standardizing the clinical testing protocols of point-of-care devices for obstructive sleep apnea diagnosis

  • Sleep Breathing Physiology and Disorders • Original Article
  • Published:
Sleep and Breathing Aims and scope Submit manuscript

Abstract

Purpose

In recent years, point-of-care (POC) devices, especially smart wearables, have been introduced to provide a cost-effective, comfortable, and accessible alternative to polysomnography (PSG)—the current gold standard—for the monitoring, screening, and diagnosis of obstructive sleep apnea (OSA). Thorough validation and human subject testing are essential steps in the translation of these device technologies to the market. However, every device development group tests their device in their own way. No standard guidelines exist for assessing the performance of these POC devices. The purpose of this paper is to critically distill the key aspects of the various protocols reported in the literature and present a protocol that unifies the best practices for testing wearable and other POC devices for OSA.

Methods

A limited review and graphical descriptive analytics of literature—including journal articles, web sources, and clinical manuscripts by authoritative agencies in sleep medicine—are performed to glean the testing and validation methods employed for POC devices, specifically for OSA.

Results

The analysis suggests that the extent of heterogeneity of the demographics, the performance metrics, subject survey, hypotheses, and statistical analyses need to be carefully considered in a systematic protocol for testing POC devices for OSA.

Conclusion

We provide a systematic method and list specific recommendations to extensively assess various performance criteria for human subject testing of POC devices. A rating scale of 1–3 is provided to encourage studies to put a focus on addressing the key elements of a testing protocol.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Engleman H, Douglas N (2004) Sleep 4: sleepiness, cognitive function, and quality of life in obstructive sleep apnoea/hypopnoea syndrome. Thorax 59(7):618–622

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Giles TL et al (2006) Continuous positive airways pressure for obstructive sleep apnoea in adults. Cochrane Database Syst Rev 1

  3. Terán-Santos J, Jimenez-Gomez A, Cordero-Guevara J (1999) The association between sleep apnea and the risk of traffic accidents. N Engl J Med 340(11):847–851

    PubMed  Google Scholar 

  4. Lee W, Nagubadi S, Kryger MH, Mokhlesi B (2008) Epidemiology of obstructive sleep apnea: a population-based perspective. Expert Rev Respir Med 2(3):349–364

    PubMed  PubMed Central  Google Scholar 

  5. Kapur V, Blough DK, Sandblom RE, Hert R, de Maine JB, Sullivan SD, Psaty BM (1999) The medical cost of undiagnosed sleep apnea. Sleep 22(6):749–755

    CAS  PubMed  Google Scholar 

  6. Partinen M, Palomaki H (1985) Snoring and cerebral infarction. Lancet 326(8468):1325–1326

    Google Scholar 

  7. Koskenvuo M, Kaprio J, Telakivi T, Partinen M, Heikkila K, Sarna S (1987) Snoring as a risk factor for ischaemic heart disease and stroke in men. Br Med J (Clin Res Ed) 294(6563):16–19

    CAS  Google Scholar 

  8. Hung J, Whitford EG, Hillman DR, Parsons RW (1990) Association of sleep apnoea with myocardial infarction in men. Lancet 336(8710):261–264

    CAS  PubMed  Google Scholar 

  9. Kripke DF, Ancoli-Israel S, Klauber MR, Wingard DL, Mason WJ, Mullaney DJ (1997) Prevalence of sleep-disordered breathing in ages 40-64 years: a population-based survey. Sleep 20(1):65–76

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Duran J et al (2001) Obstructive sleep apnea–hypopnea and related clinical features in a population-based sample of subjects aged 30 to 70 Yr. Am J Respir Crit Care Med 163(3):685–689

    CAS  PubMed  Google Scholar 

  11. Young T et al (2002) Predictors of sleep-disordered breathing in community-dwelling adults: the Sleep Heart Health Study. Arch Intern Med 162(8):893–900

    PubMed  Google Scholar 

  12. Young T, Skatrud J, Peppard PE (2004) Risk factors for obstructive sleep apnea in adults. Jama 291(16):2013–2016

    CAS  PubMed  Google Scholar 

  13. Mazzocchi S How much does a sleep study cost? Well, $600 or $5,070. 2013; Available from: https://clearhealthcosts.com/blog/2013/04/how-much-does-a-sleep-study-cost-well-600-or-5070/. Accessed April 2020

  14. Deutsch PA, Simmons MS, Wallace JM (2006) Cost-effectiveness of split-night polysomnography and home studies in the evaluation of obstructive sleep apnea syndrome. J Clin Sleep Med 2(2):145–153

    PubMed  Google Scholar 

  15. Ahmadi N, Shapiro GK, Chung SA, Shapiro CM (2009) Clinical diagnosis of sleep apnea based on single night of polysomnography vs. two nights of polysomnography. Sleep Breath 13(3):221–226

    PubMed  Google Scholar 

  16. Flemons WW, Douglas NJ, Kuna ST, Rodenstein DO, Wheatley J (2004) Access to diagnosis and treatment of patients with suspected sleep apnea. Am J Respir Crit Care Med 169(6):668–672

    PubMed  Google Scholar 

  17. Collop NA (2002) Scoring variability between polysomnography technologists in different sleep laboratories. Sleep Med 3(1):43–47

    PubMed  Google Scholar 

  18. White DP, Gibb TJ (1998) Evaluation of a computerized polysomnographic system. Sleep 21:188–196

    CAS  PubMed  Google Scholar 

  19. Newell J, Mairesse O, Verbanck P, Neu D (2012) Is a one-night stay in the lab really enough to conclude? First-night effect and night-to-night variability in polysomnographic recordings among different clinical population samples. Psychiatry Res 200(2):795–801

    PubMed  Google Scholar 

  20. Mosko SS, Dickel MJ, Ashurst J (1988) Night-to-night variability in sleep apnea and sleep-related periodic leg movements in the elderly. Sleep 11(4):340–348

    CAS  PubMed  Google Scholar 

  21. Chesson AL Jr, Berry RB, Pack A (2003) Practice parameters for the use of portable monitoring devices in the investigation of suspected obstructive sleep apnea in adults. Sleep 26(7):907–913

    PubMed  Google Scholar 

  22. Su S, Baroody FM, Kohrman M, Suskind D (2004) A comparison of polysomnography and a portable home sleep study in the diagnosis of obstructive sleep apnea syndrome. Otolaryngol Head Neck Surg 131(6):844–850

    PubMed  Google Scholar 

  23. Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernandez-Blasco L, Monasterio C, Alonso A, Chiner E, Rubio M, Garcia-Ledesma E, Cacelo L, Carpizo R, Sacristan L, Salord N, Carrera M, Sancho-Chust JN, Embid C, Vazquez-Polo FJ, Negrin MA, Montserrat JM (2011) Effectiveness of home respiratory polygraphy for the diagnosis of sleep apnoea and hypopnoea syndrome. Thorax 66(7):567–573

    PubMed  Google Scholar 

  24. Kim RD, Kapur VK, Redline-Bruch J, Rueschman M, Auckley DH, Benca RM, Foldvary-Schafer NR, Iber C, Zee PC, Rosen CL, Redline S, Ramsey SD (2015) An economic evaluation of home versus laboratory-based diagnosis of obstructive sleep apnea. Sleep 38(7):1027–1037

    PubMed  PubMed Central  Google Scholar 

  25. Berry RB et al (2008) Portable monitoring and autotitration versus polysomnography for the diagnosis and treatment of sleep apnea. Sleep 31(10):1423–1431

    PubMed  PubMed Central  Google Scholar 

  26. Victor LD (1999) Obstructive sleep apnea. Am Fam Physician 60(8):2279–2286

    CAS  PubMed  Google Scholar 

  27. Whittle AT, Finch SP, Mortimore IL, MacKay TW, Douglas NJ (1997) Use of home sleep studies for diagnosis of the sleep apnoea/hypopnoea syndrome. Thorax 52(12):1068–1073

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Price CP, Kricka LJ (2007) Improving healthcare accessibility through point-of-care technologies. Clin Chem 53(9):1665–1675

    CAS  PubMed  Google Scholar 

  29. Anderson DA, Crowe SM, Garcia M (2011) Point-of-care testing. Current HIV/aids Reports 8(1):31–37

  30. Ventola CL (2014) Mobile devices and apps for health care professionals: uses and benefits. Pharmacy and Therapeutics 39(5):356–364

  31. EarlySense. Cited 2018 June 15; Available from: https://www.earlysense.com/. Accessed April 2020

  32. VitalConnect. Cited 2018 June 15; Available from: https://vitalconnect.com/solutions/vitalpatch/. Accessed April 2020

  33. Itamar-Medical . Cited 2018 June 15; Available from: http://www.itamar-medical.com/watchpat-main/. Accessed April 2020

  34. Stoohs R, Guilleminault C (1992) MESAM 4: an ambulatory device for the detection of patients at risk for obstructive sleep apnea syndrome (OSAS). Chest 101(5):1221–1227

  35. Ancoli-Israel S, Clopton P, Klauber MR, Fell R, Mason W (1997) Use of wrist activity for monitoring sleep/wake in demented nursing-home patients. Sleep 20(1):24–27

  36. Paquet J, Kawinska A, Carrier J (2007) Wake detection capacity of actigraphy during sleep. Sleep 30(10):1362–1369

  37. de Souza L, Benedito-Silva AA, Pires MLN, Poyares D, Tufik S, Calil HM (2003) Further validation of actigraphy for sleep studies. Sleep 26(1):81–85

  38. Beniczky S, Ryvlin P (2018) Standards for testing and clinical validation of seizure detection devices. Epilepsia 59(Suppl 1):9–13

    PubMed  Google Scholar 

  39. Van de Water AT, Holmes A, Hurley DA (2011) Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography—a systematic review. J Sleep Res 20(1 Pt 2):183–200

    PubMed  Google Scholar 

  40. Kushida CA, Littner MR, Morgenthaler T, Alessi CA, Bailey D, Coleman J Jr, Friedman L, Hirshkowitz M, Kapen S, Kramer M, Lee-Chiong T, Loube DL, Owens J, Pancer JP, Wise M (2005) Practice parameters for the indications for polysomnography and related procedures: an update for 2005. Sleep 28(4):499–523

    PubMed  Google Scholar 

  41. Driver HS, Pereira EJ, Bjerring K, Toop F, Stewart SC, Munt PW, Fitzpatrick MF (2011) Validation of the MediByte(®) type 3 portable monitor compared with polysomnography for screening of obstructive sleep apnea. Can Respir J 18(3):137–143

    PubMed  PubMed Central  Google Scholar 

  42. Adult Obstructive Sleep Apnea Task Force of the American Academy of Sleep, M (2009) Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med 5(3):263–276

    Google Scholar 

  43. Tal A, Shinar Z, Shaki D, Codish S, Goldbart A (2017) Validation of contact-free sleep monitoring device with comparison to polysomnography. J Clin Sleep Med 13(3):517–522

    PubMed  PubMed Central  Google Scholar 

  44. O’Driscoll DM et al (2013) Energy expenditure in obstructive sleep apnea: validation of a multiple physiological sensor for determination of sleep and wake. Sleep Breath 17(1):139–146

    PubMed  Google Scholar 

  45. Kanady JC, Drummond SPA, Mednick SC (2011) Actigraphic assessment of a polysomnographic-recorded nap: a validation study. Journal of Sleep Research 20(1pt2):214–222

    PubMed  Google Scholar 

  46. Shambroom JR, Fabregas SE, Johnstone J (2012) Validation of an automated wireless system to monitor sleep in healthy adults. J Sleep Res 21(2):221–230

    PubMed  Google Scholar 

  47. de Zambotti M, Baker FC, Colrain IM (2015) Validation of sleep-tracking technology compared with polysomnography in adolescents. Sleep 38(9):1461–1468

    PubMed  PubMed Central  Google Scholar 

  48. Jean-Louis G, Kripke DF, Cole RJ, Assmus JD, Langer RD (2001) Sleep detection with an accelerometer actigraph: comparisons with polysomnography. Physiol Behav 72(1):21–28

    CAS  PubMed  Google Scholar 

  49. Meltzer LJ et al (2012) Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep 35(1):159–166

    PubMed  PubMed Central  Google Scholar 

  50. Meng L, Xu H, Guan J, Yi H, Wu H, Yin S (2016) Validation of a novel sleep-monitoring system for diagnosing obstructive sleep apnea: a comparison with polysomnography. Exp Ther Med 12(5):2937–2941

    CAS  PubMed  PubMed Central  Google Scholar 

  51. Zaffaroni A, Kent B, O'Hare E, Heneghan C, Boyle P, O'Connell G, Pallin M, de Chazal P, McNicholas WT (2013) Assessment of sleep-disordered breathing using a non-contact bio-motion sensor. J Sleep Res 22(2):231–236

    PubMed  Google Scholar 

  52. Zhou Y, Shu D, Xu H, Qiu Y, Zhou P, Ruan W, Qin G, Jin J, Zhu H, Ying K, Zhang W, Chen E (2020) Validation of novel automatic ultra-wideband radar for sleep apnea detection. J Thorac Dis 12(4):1286–1295

    PubMed  PubMed Central  Google Scholar 

  53. Saha S, Kabir M, Montazeri Ghahjaverestan N, Hafezi M, Gavrilovic B, Zhu K, Alshaer H, Yadollahi A (2020) Portable diagnosis of sleep apnea with the validation of individual event detection. Sleep Med 69:51–57

    PubMed  Google Scholar 

  54. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31(4):337–350

    PubMed  PubMed Central  Google Scholar 

  55. Lwanga SK, Lemeshow S, W.H. Organization (1991) Sample size determination in health studies: a practical manual

  56. Ayappa I et al (2008) Validation of a self-applied unattended monitor for sleep disordered breathing. J Clin Sleep Med 4(1):26–37

    PubMed  PubMed Central  Google Scholar 

  57. Barger LK, Rajaratnam SMW, Wang W, O'Brien CS, Sullivan JP, Qadri S, Lockley SW, Czeisler CA, for the Harvard Work Hours, Health and Safety Group (2015) Common sleep disorders increase risk of motor vehicle crashes and adverse health outcomes in firefighters. J Clin Sleep Med 11(03):233–240

    PubMed  PubMed Central  Google Scholar 

  58. Xu L, Han F, Keenan BT, Kneeland-Szanto E, Yan H, Dong X, Chang Y, Zhao L, Zhang X, Li J, Pack AI, Kuna ST (2017) Validation of the Nox-T3 portable monitor for diagnosis of obstructive sleep apnea in Chinese adults. J Clin Sleep Med 13(5):675–683

    PubMed  PubMed Central  Google Scholar 

  59. Chen H, Lowe AA, Bai Y, Hamilton P, Fleetham JA, Almeida FR (2009) Evaluation of a portable recording device (ApneaLink) for case selection of obstructive sleep apnea. Sleep Breath 13(3):213–219

    PubMed  Google Scholar 

  60. Stehling F, Keull J, Olivier M, Große-Onnebrink J, Mellies U, Stuck BA (2017) Validation of the screening tool ApneaLink® in comparison to polysomnography for the diagnosis of sleep-disordered breathing in children and adolescents. Sleep Med 37:13–18

    PubMed  Google Scholar 

  61. Le TQ et al (2013) Wireless wearable multisensory suite and real-time prediction of obstructive sleep apnea episodes. IEEE J Transl Eng Health Med 1:2700109–2700109

    PubMed  PubMed Central  Google Scholar 

  62. Zou D, Grote L, Peker Y, Lindblad U, Hedner J (2006) Validation a portable monitoring device for sleep apnea diagnosis in a population based cohort using synchronized home polysomnography. Sleep 29(3):367–374

    PubMed  Google Scholar 

  63. Heneghan C et al (2008) A portable automated assessment tool for sleep apnea using a combined Holter-Oximeter. Sleep 31(10):1432–1439

    PubMed  PubMed Central  Google Scholar 

  64. Hajian-Tilaki K (2013) Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 4(2):627–635

    PubMed  PubMed Central  Google Scholar 

  65. Guggenmoos-Holzmann I (1996) The meaning of kappa: probabilistic concepts of reliability and validity revisited. J Clin Epidemiol 49(7):775–782

    CAS  PubMed  Google Scholar 

  66. Cairns A, Wickwire E, Schaefer E, Nyanjom D (2014) A pilot validation study for the NOX T3TM portable monitor for the detection of OSA. Sleep Breath 18(3):609–614

    PubMed  Google Scholar 

  67. Aloia MS, Arnedt JT, Stanchina M, Millman RP (2007) How early in treatment is PAP adherence established? Revisiting night-to-night variability. Behav Sleep Med 5(3):229–240

    PubMed  Google Scholar 

  68. Zucconi M, Ferini-Strambi L, Castronovo V, Oldani A, Smirne S (1996) An unattended device for sleep-related breathing disorders: validation study in suspected obstructive sleep apnoea syndrome. Eur Respir J 9(6):1251–1256

    CAS  PubMed  Google Scholar 

  69. To KW et al (2009) Validation study of a portable monitoring device for identifying OSA in a symptomatic patient population. Respirology 14(2):270–275

    PubMed  Google Scholar 

  70. Garde A, Dehkordi P, Karlen W, Wensley D, Ansermino JM, Dumont GA (2014) Development of a screening tool for sleep disordered breathing in children using the phone Oximeter. PLoS One 9(11):e112959

    PubMed  PubMed Central  Google Scholar 

  71. O’Driscoll DM, Turton AR, Copland JM, Strauss BJ, Hamilton GS (2013) Energy expenditure in obstructive sleep apnea: validation of a multiple physiological sensor for determination of sleep and wake. Sleep Breath 17(1):139–146

    PubMed  Google Scholar 

  72. Wongdhamma W, Le TQ, Bukkapatnam STS (2013) Wireless wearable multi-sensory system for monitoring of sleep apnea and other cardiorespiratory disorders. in 2013 IEEE International Conference on Automation Science and Engineering (CASE).

  73. Le TQ, et al. (2013) Prediction of sleep apnea episodes from a wireless wearable multisensor suite. In 2013 IEEE Point-of-Care Healthcare Technologies (PHT). IEEE

  74. Bukkapatnam ST, Le T, Wongdhamma W, Device and method for predicting and preventing obstructive sleep apnea (osa) episodes. 2014, Google Patents

  75. Xu H, Zhao X, Shi Y, Li X, Qian Y, Zou J, Yi H, Huang H, Guan J, Yin S (2019) Development and validation of a simple-to-use clinical nomogram for predicting obstructive sleep apnea. BMC Pulm Med 19(1):18

    PubMed  PubMed Central  Google Scholar 

  76. Lu M, Fang F, Sanderson JE, Ma C, Wang Q, Zhan X, Xie F, Xiao L, Liu H, Liu H, Wei Y (2019) Validation of a portable monitoring device for the diagnosis of obstructive sleep apnea: electrocardiogram-based cardiopulmonary coupling. Sleep Breath 23(4):1371–1378

    PubMed  Google Scholar 

  77. Fassbender P et al (2019) Validation of a photoplethysmography device for detection of obstructive sleep apnea in the perioperative setting. J Clin Monit Comput 33(2):341–345

    PubMed  Google Scholar 

Download references

Funding

This study was funded by NSF-PFI-AIR-TT 1543226.

Author information

Authors and Affiliations

  1. Department of Industrial & Systems Engineering, Texas A&M University, 101 Bizzell St, College Station, TX, 77843, USA

    Vivek Tangudu, Kahkashan Afrin & Satish T. S. Bukkapatnam

  2. Department of Biomedical Engineering, Texas A&M University, College Station, TX, 77843, USA

    Sandy Reddy & Satish T. S. Bukkapatnam

  3. Center for the Translation Research in Aging and Longevity, Human Clinical Research Facility, Texas A&M University, College Station, TX, 77843, USA

    Nicolaas E.P. Deutz

  4. Department of Educational Psychology, Texas A&M University, College Station, TX, 77843, USA

    Steven Woltering

Authors
  1. Vivek Tangudu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kahkashan Afrin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sandy Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolaas E.P. Deutz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven Woltering
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Satish T. S. Bukkapatnam
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Vivek Tangudu was the primary author of the paper. Kahkashan Afrin provided guidance to the primary author on the specific sources of literature and organization of the contents into tabular forms, and was responsible for detailing the contents of various sections in the manuscript. Sandy Reddy was responsible for conceptualizing the graphical representation as well as detailing the contents of the various sections in the manuscript. Nicolas Deutz and Steven Woltering contributed to the human subject testing and the review methodology. Satish Bukkapatnam is responsible for the organization, structure, and contents of the paper. All the authors were involved with the revision and editing process.

Corresponding author

Correspondence to Satish T. S. Bukkapatnam.

Ethics declarations

Disclosure

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Presentation at a conference: None

Electronic supplementary material

ESM 1

(DOCX 49 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tangudu, V., Afrin, K., Reddy, S. et al. Toward standardizing the clinical testing protocols of point-of-care devices for obstructive sleep apnea diagnosis. Sleep Breath 25, 737–748 (2021). https://doi.org/10.1007/s11325-020-02171-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11325-020-02171-5

Keywords

Search

Navigation