Optics guided thermal image super-resolution via alignment fusion network

Unmanned Aerial Vehicle (UAV) platforms often equip dual-modal cameras that capture paired optical and thermal infrared images and utilizing optical information to guide the Super-Resolution (SR) of thermal images has become an effective way to obtain High-Resolution (HR) thermal images from UAV viewpoint. However, due to the different positions and resolutions of optical and thermal sensors, the acquired optical and thermal images are usually in a non-aligned state, which leads to less effective guidance for thermal image SR. This paper proposes an Alignment Fusion Network (AFNet), which aligns and fuses optical features in thermal image SR to avoid the effect of the non-aligned issue. First, the Feature Alignment Module (FAM) is designed to align optical images at the feature level with thermal images using deformable convolution and to enhance features using the channel attention mechanism. A Bidirectional Fusion Module (BFM) is then developed to effectively fuse optical and thermal features. Extensive experiments on public datasets show that our AFNet effectively mitigates the effect of the non-aligned issue and provides better HR thermal images both numerically and visually against state-of-the-art methods.