RANF: Neural Field-Based HRTF Spatial Upsampling With Retrieval Augmentation and Parameter Efficient Fine-Tuning

This paper gives an in-depth description of our submission to Task 2 of the Listener Acoustic Personalization (LAP) challenge 2024, which aims to reconstruct head-related transfer functions (HRTFs) with dense spatial grids from sparse measurements. Neural fields (NFs) with parameter-efficient fine-tuning (PEFT) have led to dramatic performance improvements in HRTF spatial upsampling and personalization. Despite these advances, spatial upsampling performance remains limited in scenarios with very sparse measurements. Our proposed system, named retrieval-augmented NF (RANF), incorporates HRTFs retrieved from a dataset as auxiliary inputs. We leverage multiple retrievals via transform-average-concatenate and adopt a PEFT technique tailored for retrieval augmentation. Furthermore, we capitalize on the results of a signal-processing-based spatial upsampling method as optional inputs. By incorporating these auxiliary inputs, our system demonstrated state-of-the-art performance on the SONICOM dataset and placed first in Task 2 of the LAP challenge 2024.