Skip to main content
Log in

From lab to life: Evaluating the reliability and validity of psychophysiological data from wearable devices in laboratory and ambulatory settings

  • Original Manuscript
  • Published:
Behavior Research Methods Aims and scope Submit manuscript

Abstract

Despite the increasing popularity of ambulatory assessment, the reliability and validity of psychophysiological signals from wearable devices is unproven in daily life settings. We evaluated the reliability and validity of physiological signals (electrocardiogram, ECG; photoplethysmography, PPG; electrodermal activity, EDA) collected from two wearable devices (Movisens EcgMove4 and Empatica E4) in the lab (N = 67) and daily life (N = 20) among adults aged 18–64 with Mindware as the laboratory gold standard. Results revealed that both wearable devices’ valid data rates in daily life were lower than in the laboratory (Movisens ECG 82.94 vs. 93.10%, Empatica PPG 8.79 vs. 26.14%, and Empatica EDA 41.16 vs. 42.67%, respectively). The poor valid data rates of Empatica PPG signals in the laboratory could be partially attributed to participants' hand movements (r = – .27, p = .03). In laboratory settings, heart rate (HR) derived from both wearable devices exhibited higher concurrent validity than heart rate variability (HRV) metrics (ICCs 0.98–1.00 vs. 0.75–0.97). The number of skin conductance responses (SCRs) derived from Empatica showed higher concurrent validity than skin conductance level (SCL, ICCs 0.38 vs. 0.09). Movisens EcgMove4 provided more reliable and valid HRV measurements than Empatica E4 in both laboratory (split-half reliability: 0.95–0.99 vs. 0.85–0.98; concurrent validity: 0.95–1.00 vs. 0.75–0.98; valid data rate: 93.10 vs. 26.14%) and ambulatory settings (split-half reliability: 0.99–1.00 vs. 0.89–0.98; valid data rate: 82.94 vs. 8.79%). Although the reliability and validity of wearable devices are improving, findings suggest researchers should select devices that yield consistently robust and valid data for their measures of interest.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We abbreviated 'Empatica E4' as 'Empatica' throughout this paper for brevity, but it should be noted that our findings are based on the E4 model that we tested and may not necessarily apply to other products from Empatica.

References

Download references

Author Note

This research was supported by the National Institute of Mental Health (R01MH118218). During preparation of this manuscript, Dr. Northrup was also supported by K23MH127420. The authors have no conflicts of interest to report.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lauren M. Bylsma.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Practice Statement

The index-level data analyzed in the current study are available in the OSF repository, https://osf.io/28usc/?view_only=6508177708fb4ef380bdbf5dd24b0b1b. As this was a validation study, no hypotheses were pre-registered.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 388 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, X., Sgherza, T.R., Nothrup, J.B. et al. From lab to life: Evaluating the reliability and validity of psychophysiological data from wearable devices in laboratory and ambulatory settings. Behav Res (2024). https://doi.org/10.3758/s13428-024-02387-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.3758/s13428-024-02387-3

Keywords

Navigation