Application of reinforcement learning in phase control of mid-air ultrasonic haptics

Abstract Mid-air ultrasonic haptics is a technology that uses ultrasonic waves in air to provide tactile sensations without physical contact. To deliver a strong tactile sensation at a desired position, it is essential to create a focal point through beamforming, which involves controlling the phases of the multiple ultrasonic waves. In this study, we propose a method for deriving the optimum phase shift using reinforcement learning without the use of external equipment. The phase shifts are treated as discrete values between 0 and 360, and focal points are generated using deep Q -Network and SARSA. The evaluation results confirmed that focus was formed; however, issues were observed where the deviation from the target position increased when the phase decimation interval was increased. On the other hand, by narrowing the search range, almost 100% of the target sound pressure was achieved and highly accurate focus formation was possible.