Utility-Privacy Aware Mobile Diminished Reality Framework for 3D Visual Privacy

Engaging in immersive technologies has obvious benefits, but the increasing pervasiveness of these technologies, specifically Augmented Reality (AR) and Mixed Reality poses privacy risks, especially since AR cameras are continually capturing information about users’ surroundings, including people and personal places. Current privacy-preserving techniques are either designed for a two-dimensional setting where private information is always considered to be in the background, or they struggle to maintain good performance during significant camera motion. Moreover, existing techniques measure the privacy-utility tradeoff by relying on a utility measure specific to the task at hand. We propose a Utility-Privacy Tradeoff Algorithm (UPTA) on top of a real-time mobile privacy-preserving Diminished Reality system that enables the user to obfuscate sensitive objects in a three-dimensional environment. The UPTA adjusts the size and position of the obfuscation region such that it achieves a proper balance between privacy and utility during free camera motion. The algorithm obfuscates the necessary parts of the screen only, to preserve privacy while maintaining a proper level of utility. In addition, we propose a generic utility measure that is inversely proportional to the size of the obfuscation. We demonstrate our framework’s ability to achieve a proper balance between privacy and utility, during translational and rotational motion patterns, where UPTA improves the privacy-utility tradeoff figure-of-merit by an order of magnitude (more than 8x) relative to existing methods.