ThermalSense: A Sensory Substitution System for Enabling Perception of Thermal Information

We present the development process, design considerations and generalized system engineering for the ThermalSense sensory substitution system. The system utilizes visual-to-auditory sensory substitution to convey thermal properties through auditory cues. In an online proof of concept study evaluating the system, participants demonstrated a high level of accuracy in recognizing and localizing thermal information which is otherwise invisible in visual scenes. The ability of users to gain proficiency within a short training period was accompanied by a positive user experience evaluation. These results provide an indication of the system’s potential to enable an extended sense, serving as an extension of visual perception to the range of thermal information beyond its natural capacities. The proposed system architecture, together with these findings, set the ground for the further development of this and similar systems, as well as general advances in the study of sensory perception.