Real-Time Object Detection and Boundary Extraction in Augmented Reality Using Lightweight Deep Learning Models with Unity Sentis

In this study, an efficient augmented reality (AR) system is developed to real-time detect and extract boundary of the target using lightweight deep learning models MobileNet and Unity Sentis, for resource constrained devices. The system integrates optimized model inference, video input and smooth AR content rendering, and enhances the efficiency of the operation while ensuring accuracy through techniques like quantization and pruning. The result shows that the system can detect objects with a mean accuracy (mAP) of 72.3%, with an intersection-to-integration ratio (IoU) of 0.85 and an average inference time of 35ms on smartphones and 20ms on PCs, which guarantees that the AR interactions are smooth and that the system is stable at 25 FPS and 45 FPS respectively. This technology can be applied in many sectors including gaming, industrial monitoring and education and many others. Although there is much work to be done in complex environments and low-end devices, this research provides a foundation for developing real-time scalable AR solutions. Future work will be directed towards extending the study to include multimodal data fusion, advanced models and more complex application scenarios.