Poster: Mixture of Class-aware Experts for Efficient AIoT Inference

Deep neural networks (DNNs) have enabled a wide range of artificial intelligence of things (AIoT) applications, but their increasing complexity poses challenges for deployment on resource-constrained devices. Model compression techniques such as pruning and quantization have been widely adopted to address these challenges; however, they inevitably incur accuracy loss due to information loss. Recently, class-aware pruning has emerged as a promising approach, but existing methods often lack flexibility, as they are typically tailored to fixed target class sets and fail to generalize well to dynamic or broad class distributions. To address this limitation, we propose Mixture of Class-aware Experts (MoCE), a novel framework that combines class-aware pruning with a Mixture of Experts (MoE) architecture. MoCE constructs multiple lightweight experts using class-aware pruning, each specialized for a subset of classes, and employs a shared encoder and a lightweight router to dynamically select the appropriate expert at runtime. Our preliminary results demonstrate the potential of combining class-aware pruning and expert selection to enable accurate and efficient inference on resource-limited AIoT devices.