TranscribeSight: Redefining ASR Evaluation for Industry 4.0 Knowledge Capture

This paper presents TranscribeSight, a benchmarking platform for systematic evaluation of automatic speech recognition (ASR) technologies in industrial knowledge management contexts. As Industry 4.0 transforms manufacturing, organisations require flexible frameworks for comparing traditional and multimodal large language model solutions. TranscribeSight provides an extensible platform integrating multidimensional evaluation methodologies, encompassing semantic preservation metrics, operational efficiency analysis, and automated qualitative assessment capabilities. The platform’s modular architecture enables integration of diverse ASR services and customisation of evaluation criteria for specific industrial requirements. We demonstrate capabilities through controlled evaluation of eight ASR services using LibriSpeech audio samples with English-language content and default API configurations, though the platform supports multilingual extension and customised configurations. The evaluation identified performance variations across services, with different technologies excelling in accuracy, semantic preservation, or operational efficiency. TranscribeSight provides a foundation for systematic, reproducible ASR assessment adapting to technological advancement and diverse operational requirements in Industry 4.0 environments.