Efficient and Accurate Inference in Edge-Cloud Framework via Large and Small Model Collaboration

With the rapid advancement of edge-cloud collaboration, deploying efficient and intelligent models on resource–constrained edge devices has become a critical challenge. To tackle the issue, in this paper, we propose a collaborative framework that leverages the complementary strengths of large and small models to achieve high-performance inference while maintaining computational efficiency. Specifically, we first detect hard samples by the local lightweight model and upload them to the cloud for processing, followed by re-training the local lightweight model. Our approach provides a more accurate and efficient solution for real-time computing scenarios by combining the advantages of both cloud computing and edge computing via large and small models. Experiments on two tasks show that by using only around 10% scale of hard examples, our approach maintains a better balance of inference efficiency and accuracy compared to relying on a single model.