Fringe projection profilometry with lightweight transformer-CNN network for high-dynamic-range scenes

Abstract 3D reconstruction of high-dynamic-range (HDR) scenes using a minimal number of images is a critical challenge for fringe projection profilometry. Deep learning (DL) has demonstrated significant potential, but previous approaches have never considered a balance between accuracy and speed, resulting in a slow inference latency for networks. A lightweight U-shaped transformer-CNN architecture is proposed that integrates dual attention mechanisms (channel and pixel attention) which enable the network to focus more attention on HDR regions. In addition, an efficient transformer (ET) layer establishes a longer dependence that enables compensation for the HDR regions by utilizing their surrounding pixels’ phase information. To address the data scarcity challenge, the network is first pre-trained on a synthetic dataset and subsequently fine-tuned using a real-world dataset that contains 720 HDR scenes. Experiments have proven that the proposed network achieves a phase error as low as 5.32 <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:mo>×</mml:mo> <mml:mrow> <mml:msup> <mml:mn>10</mml:mn> <mml:mrow> <mml:mo>−</mml:mo> <mml:mn>4</mml:mn> </mml:mrow> </mml:msup> </mml:mrow> </mml:mrow> </mml:math> rad with it only containing 0.55 million parameters and a real-time processing speed of 16.56 frames per second (FPS) on an RTX 4060 GPU. Compared with other networks, the proposed network provides an effective phase recovery performance with faster inference speed for HDR scene reconstruction, which is crucial for actual industry applications.