ProSE: Probing Reconstruction Error for Semantic-guided Expert Skipping in MoE-MLLMs
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer scalable inference through sparse expert activation, yet substantial computational redundancy remains among the activated experts. Existing expert skipping methods typically calibrate thresholds based on target skipping ratios or final-output divergence, overlooking the accumulation of skipping-induced error across layers and the varying semantic importance of individual tokens. To address that, we propose ProSE, a training-free, semantic-guided expert skipping framework that probes layer-wise reconstruction error to promote representation consistency compared with default activation. Specifically, we construct query-aware semantic masks to identify salient visual tokens and progressively calibrate modality-specific skipping thresholds for each MoE layer. By minimizing masked reconstruction error under a given budget, ProSE maintains semantic consistency after skipping and allows adaptive expert allocation under limited activation budgets. Extensive experiments on 9 benchmarks demonstrate consistent improvements on average over existing expert skipping methods, particularly under aggressive skipping settings. ProSE maintains over 96\% performance on KimiVL-A3B-Inst with 83\% expert skipping and accelerates prefilling by $1.7\times$.
Authors
- Matthew B. Blaschko (ORCID: https://orcid.org/0000-0002-2640-181X)
- Minye Wu (ORCID: https://orcid.org/0000-0002-8163-9513)
- Jiameng Li
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.48550/arxiv.2611.20885
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- preprint