Fusing Point Clouds and Wireless Signals for Seamless Interaction in Smart Spaces

The convergence of Mixed Reality (MR) and Internet of Things (IoT) presents unprecedented opportunities for intuitive smart environment interaction, yet existing approaches suffer from fragmented control schemes and cumbersome device localization methods that rely on static infrastructure or extensive manual calibration. We present ViewIoT, a seamless MR-IoT framework implemented on a Meta Quest 3S that unifies accurate device localization with natural multimodal control through a single head-mounted display system. Our approach treats IoT devices as fixed wireless signal sources while leveraging the MR headset as a mobile receiver that simultaneously collects timestamped RSSI measurements and constructs real-time 3D spatial maps through mesh reconstruction and pose tracking. ViewIoT introduces a novel fusion algorithm that combines spatial point cloud data with calibrated signal strength measurements to automatically estimate IoT device positions in 3D space, eliminating the need for vision-based detection, pre-installed anchors, or device name memorization. Beyond localization, our framework enables users to discover, visualize, and control IoT devices directly within their MR field of view through intuitive voice commands and gesture interactions, replacing fragmented app-based control with a unified spatial interface.