E<i>d</i>ge<i>S</i>can for IoT Contextual Understanding With Edge Computing and Image Captioning

The emergence of Edge Computing has shifted the processing capabilities in proximity to the Internet of Things (IoT) data sources, offering solutions to latency and bandwidth constraints applications. This shift complements Cloud Computing, especially in handling real-time data processing and enhancing processing. Image processing, particularly image captioning for smart monitoring systems, benefits greatly from this synergy. Image captioning plays a crucial role in understanding visual data. While early methods excelled in encoder-decoder frameworks and attention mechanisms, they often overlooked semantic representations which are essential for comprehensive image understanding. To address this gap, we introduce the EdgeScan framework, leveraging Edge Computing for image analysis and semantic feature extractions closer to data sources. EdgeScan integrates visual and semantic features to create more informative and enriched image captions that enhance image captioning accuracy. The EdgeScan image captioning model architecture is capable of 1) learning the salient image region’s specific feature representation and 2) co-embedding visual attention and semantic attributes in one space for feature fusion. This improves models’ ability to interpret and respond to data in a meaningful way, which is particularly valuable for IoT applications that require a deep understanding of the semantics of diverse and constantly changing data for efficient operation. Extensive experiments were conducted on the MS-COCO dataset to demonstrate the superiority of EdgeScan in both quantitative and qualitative performance, achieving highest consensus-based image description evaluation score of 120.9, as well as notable scores of 78.6 for BLEU@1 and 57.7 for recall-oriented understudy for gisting evaluation metrics, promising advancements in IoT-driven image understanding and competitiveness against the state of the art.