Accuracy Of Step Counting Methods: The Skyrocket Study

The Consumer Technology Association (CTA) recommends at least two researchers verify, through video recordings, observed steps taken during treadmill-based validation studies of wearable technologies. To our knowledge, there is no evidence to support this recommendation. The duplicative efforts of performing additional recounts to enhance reliability requires significant resources to cover personnel time and costs. PURPOSE: To analyze whether additional reviews of video recordings are necessary to verify observed steps counted during treadmill testing. METHODS: Twenty treadmill walking bouts (ranging in speeds from 0.5mph to 4.0mph) were randomly selected from a larger sample performed by 18-20-year-old participants in the NIH-funded R01 SKyRoCKeT Study of cadence and intensity. Directly observed, hand-tallied steps taken during the bouts served as the original step count. Video recordings were reviewed and recounted by a second researcher. To implement the CTA recommendations, a third, independent researcher conducted a third count on the first five walking bouts. Mean absolute percentage deviation (MAPD = 1/n Σ | (recount - original)/original| x 100%) was calculated, where n was the number of bouts. RESULTS: The MAPD was 0.09% with bout-specific deviations ranging between 0 - 0.97%, indicating that there was less than a 1% difference between the original and first recount for the same bout. The biggest difference between the original and first recount was 6 steps in a single five-minute walking bout at 4.0mph resulting in a 0.97% deviation compared to the original count. The third recount produced a 0% deviation between the two recounts. CONCLUSIONS: Although a second recount of video recorded steps has been previously recommended, the calculated MAPD value showed an average deviation of less than 0.1% across all recounts. Multiple recounts of observed steps taken during treadmill-based studies are redundant given the apparently very low deviations identified from an original count. This information is useful for planning large-scale studies considering the personnel time and costs involved. NIH NICHD - R01HD105768