The validation of measured and self-reported sleep duration and perceived sleep quality: an empirical study with three generations of smartwatches

Introduction: Sleep is routinely assessed in the management of mental health conditions. Wearable technologies like smartwatches offer a non-intrusive method to quantitatively measure sleep. However, there are limited empirical benchmarks for sleep duration and sleep quality measured by wearables against user reports. This study aims to evaluate the concordance between user-reported and smartwatch-measured hours of sleep and sleep quality. Methods: Participants were recruited from two decentralized digital health well-being studies and completed a 7-day sleep diary while simultaneously wearing their smartwatch to sleep (November 7, 2023 - June 30, 2024). Participants self-reported sleep timestamps and perceived sleep quality using the Sleep Quality Scale. Sleep timestamps and quality were also derived from their smartwatches (Garmin Vívoactive 4 2019, Garmin Venu 2 Plus 2022, and Garmin Venu 3/3S 2023). Statistical analyses included paired t-tests, equipercentile linking, and chi-square tests to assess agreement between smartwatch and self-reported sleep parameters. Exploratory analyses established the difference between reported and recorded sleep duration in healthcare shift workers. Results: From 841 sleep instances reported by 130 participants wearing three different generation smartwatches, the mean difference in sleep duration between smartwatch-recorded and participant-reported was 21.22 (Garmin Vívoactive), 11.67 (Garmin Venu 2 Plus), and 6.58 (Garmin Venu 3/3S) minutes, respectively. There were statistically significant between-group differences in mean sleep durations assessed by participant self-report vs. Vívoactive 4 smartwatches, but not self-report vs. Venu 2 Plus or Venu 3/3S smartwatches. Equipercentile linking revealed concordance between smartwatch sleep scores and self-reported sleep quality using the Sleep Quality Scale (SQS) between 4 and 7, with disagreements observed at the SQS ranges from 0-4 and 7-10. Conclusions: These results suggest that wearables can reliably measure sleep duration, and future research warrants improvements in algorithms that estimate sleep quality with validations across different wearable vendors.