Semi-Automatic Production and Distribution System for Audio Descriptions in Live Baseball Broadcasts

We developed a system that provides audio descriptions for live baseball broadcasts in real time to enrich the program content. From an accessibility perspective, many visually impaired people would like to have audio descriptions for all broadcast programs. Audio description is usually produced for recorded programs such as dramas. In contrast, there are few opportunities to produce audio descriptions for live programs such as live sports broadcasts because it is high in cost and requires effort. To address this problem, we developed an audio description production and distribution system. It can semi-automatically provide audio descriptions for live baseball programs in real time. Users can listen to audio descriptions via the application installed on their smartphones along with TV main audio. For the automatic generation of audio description, we extract four types of information from a broadcast video: Pitch Speed, Count, Batter's Information, and Pitch Location using a framework of deep learning. Furthermore, one of the system's modules, the manual input assistance tool, makes it easy for anyone to manually create audio descriptions. Experimental results showed that the auto-recognition functions we have implemented are superior to major applications and models. Moreover, a subjective showed that providing audio descriptions with our system is effective for the visually impaired.