MHyNet-PC: A Multi-Modal Hybrid No-Reference Framework for Point Cloud Quality Assessment

The increasing application of 3D point clouds in various technological fields such as interactive eXtended Reality (XR) and robotic interactions requires robust quality assessment methods where immediate quality feedback is necessary. Thus, traditional methods such as subjective studies or objective Full-Reference (FR) metrics are not suited for the task. In such cases, Reduced-Reference (RR) and No-Reference (NR) metrics, where limited to no access to the original content is required to assess the quality become relevant. Current NR methods lack generalization to different datasets and real-time assessment. This paper introduces MHyNet-PC, a novel multi-modal hybrid deep learning-based framework designed for NR quality assessment of point clouds, aiming to address this gap. Our approach leverages features extracted from the two main modalities in which humans experience 3D data, these being both 2D and 3D. 3D geometric features are extracted using statistical analysis, while deep learning is used to extract features from 2D projections. These features are fused to predict quality scores. MHyNet-PC is evaluated using three point cloud datasets with synthetically introduced distortions to simulate real-world imperfections. Results show a 10% improvement in accuracy compared to state-of-the-art methods, while also greatly reducing the number of computations, which decreases the inference time by 40%.