Automatic Cricket Commentary Generation using Vision Transformers

Abstract: This project presents a novel framework for automated cricket commentary generation using a combination of deep learning, computer vision, and natural language processing techniques. The system is designed to analyze cricket match footage and generate relevant play-by-play commentary without human intervention. Leveraging Vision Transformers (ViT) for framelevel visual feature extraction, the framework accurately identifies key game events such as "Four", "Six", and "Bowled". For each detected event, the system retrieves or generates contextually appropriate commentary using pre-trained language models like GPT-2, enhanced with a curated commentary dataset. The commentaries are evaluated using precision, recall, and F1-score against ground truth data. The application includes a user-friendly Streamlit interface that enables users to upload videos, view extracted events, hear generated commentary via gTTS, and assess model performance. Designed for both professional and amateur-level cricket games—especially those lacking live commentary—this framework aims to enhance viewer engagement, accessibility, and post-game analysis through automated, intelligent commentary