ChatAnalysis revisited: can ChatGPT undermine privacy in smart homes with data analysis?

Abstract Large Language Models (LLMs) have demonstrated potential in automating data-driven tasks, enabling non-experts to analyze raw inputs such as tables or sensor data using conversational queries. Advances in Machine Learning (ML) and Human-Computer Interaction (HCI) have further reduced entry barriers, pairing sophisticated model capabilities and background knowledge with user-friendly interfaces like chatbots. While empowering users, this raises critical privacy concerns when used to analyze data from personal spaces, such as smart-home environments. This paper investigates the capabilities of LLMs, specifically GPT-4 and GPT-4o, in analyzing smart-home sensor data to infer human activities, unusual activities, and daily routines. We use datasets from the CASAS project, which include data from connected devices such as motion sensors, door sensors, lamps, and thermometers. Extending our prior work, we evaluate whether advances in model design, prompt engineering, and pre-trained knowledge enhance performance in these tasks and thus increase privacy risks. Our findings reveal that GPT-4 infers daily activities and unusual activities with some accuracy but struggles with daily routines. With our experimental setup, GPT-4o underperforms its predecessor, even when supported by structured CO-STAR prompts and labeled data. Both models exhibit extensive background knowledge about daily routines, underscoring the potential for privacy violations in smart-home contexts.