Enhanced Cricket Commentary Using AI Vision and Multilingual Translation

This paper presents a novel approach to generating cricket commentary using AI vision and multilingual support. The system utilizes GPT-4 Vision to process short, audio-free cricket videos as input, analyzing frames in video to create detailed visual descriptions of the ongoing action. Based on these frame-by-frame descriptions, commentary is automatically generated using GPT-40 Mini, which ensures the commentary is structured and relevant. Gemma 2B, fine-tuned specifically for cricket commentary, is then used to translate this commentary into multiple Indian languages, ensuring accurate and culturally appropriate translations. Finally, Eleven Labs is integrated to provide natural-sounding text-to-speech synthesis, converting the translated text into clear, human-like commentary in the user’s preferred language. This project aims to enhance the accessibility of cricket commentary, making it available in multiple languages and offering an immersive experience for a wider audience.