Semantic-R2L: A Self-Supervised Real-Time 3D Semantic Segmentation Method Based on Neural Light Fields

This paper presents an innovative neural light field architecture for real-time 3D semantic understanding, challenging conventional scene interpretation paradigms through a fundamental re-architecture of feature learning processes. Our methodology circumvents traditional 3D reconstruction pipelines by establishing a direct ray-semantic correlation framework, enabling concurrent acquisition of geometric relationships and semantic contexts via a refined light field parameterization scheme. The technical contributions of our system are threefold: ‐A geometry-semantic co-learning mechanism employing cross-modal attention to enforce mutual consistency between spatial coordinates and categorical distributions, utilizing a joint optimization loss function that preserves topological fidelity. ‐An adaptive ray sampling strategy incorporating spatial occupancy prediction to selectively process rays in geometrically significant regions, reducing computational load by 62% compared to uniform sampling approaches. ‐A resource-aware network architecture employing channelwise pruning and quantized tensor operations, specifically optimized for ARM-based edge computing platforms through hardware-aware compilation. By directly encoding the plenoptic function as a conditional probability distribution of semantic labels, our approach bypasses conventional rendering pipelines and their associated computational bottlenecks. Experimental results demonstrate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3.8 \times$</tex> faster inference speeds compared to contemporary neural radiance field implementations, while maintaining comparable segmentation accuracy (mIoU 74.3 vs. 76.1) on the ScanNet benchmark. This efficiency breakthrough enables continuous operation on mobile processors under strict thermal constraints, addressing critical challenges in power-constrained applications. The framework's design principles provide new insights for developing perceptive systems in augmented reality interfaces and autonomous robotic platforms that require simultaneous geometric and semantic comprehension.