Exploring body tracking sensors in longitudinal ambient display studies in the wild

Abstract Body tracking sensors (i.e., depth cameras) such as Microsoft Kinect have been utilized in ambient display research for more than a decade. They facilitate a deeper understanding of phenomena occurring throughout interactions, aid the investigation of ambient displays within a broader context, and effectively complement existing qualitative methods such as on-site observations. Although these sensors have made significant contributions to research, there are still challenges with regard to data collection and analysis, particularly in light of recent advances in artificial intelligence. Further research is needed into how these sensors can contribute to a better understanding of how ambient displays are used in practice long term and how they can help to develop this understanding. In this article, we expand on the potentials and limitations of body tracking sensors that we experienced in our own in-the-wild research. To this end, we present insights from a small fleet of long-term, real-world installations of ambient displays we manage, which incorporate multiple body tracking sensors. The present article is concluded with a discussion of future directions for the field. In particular, the present study explores the potential contributions of body tracking sensors to recent methodological developments in the field of Human-Computer Interaction.