Method for modifying haptic feedback by displaying onomatopoeia

Haptic feedback, which enhances operability and realism, is extensively employed in smartphones and controllers. One notable tactile presentation method involves the use of visual stimuli to evoke tactile sensations, exemplified by the concept of pseudo-haptics. In these methodologies, resistance and force are simulated by modulating the velocity of the avatar’s finger or the operation pointer. Currently, there exists a discrepancy between the user’s inherent sensory perception and the visual information presented. In this study, we propose a novel approach to modifying the perception of tactile stimulation by concurrently presenting an onomatopoeic word with the tactile stimulus. For the experiment, we developed a smartphone application that, upon tapping a displayed button, triggers both a vibration stimulus and the presentation of an onomatopoeic word that conveys a sense of touch and sound. We employed six switches with varying tactile sensations to evaluate whether the user’s perception would be influenced by the type of onomatopoeia displayed in the application. The experimental results demonstrated that five onomatopoeic words elicited distinct tactile sensations. Additionally, we observed that four of these words enhanced the perceived realism of the button-press sensation. This method diverges from existing techniques by altering the tactile perception through visually presented linguistic information. Although this approach is constrained to scenarios where the haptic object is within the visual field, it is straightforward to implement and can be readily applied to existing smartphones and virtual reality devices.