TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling

The burgeoning scale of Pre-trained Large Models (PLMs) has intensified the demand for efficient inference without compromising performance, while existing large model collaborative frameworks have shown promise, they often suffer from functional redundancy among experts and limited robustness in complex cross-domain scenarios. In this paper, we propose Tri-gate Routing for Inference via Decoupled Efficient Network Technologies (TRIDENT), a highly efficient and robust heterogeneous collaborative inference framework. TRIDENT leverages the complementary inductive biases of MLP (for statistical patterns) and KAN (for symbolic logic) to maximize reasoning potential with minimal parametric overhead. To address feature homogenization in traditional distillation, we introduce Orthogonal Feature Decoupling Distillation, utilizing an orthogonality loss Lorth for functional decoupling and a reconstruction loss Lrecon to anchor decoupled features to the PLM knowledge manifold. During inference, a Dual-Threshold Arbiter effectively detects expert hallucinations by integrating individual confidence τcon and heterogeneous consistency τagree. Extensive experiments on CIFAR-100-LT, XNLI, and GSM8K demonstrate that TRIDENT significantly reduces the Invocation Rate (IR) of PLMs while maintaining high accuracy. Our findings reveal a distinct Pareto optimal balance and validate the spontaneous division of labor between heterogeneous experts. By transcending the limitations of single-architecture systems, TRIDENT provides a robust and interpretable pathway for efficient collaborative intelligence.