EfficientMixer: A Lightweight Vision Architecture via Complementary Design Principles

Lightweight vision models must maximize representational capacity under strict constraints on parameters and computational cost. We investigate whether principle-level architectural composition—the systematic integration of complementary design principles rather than optimization of individual components in isolation—can provide a more effective approach to lightweight model design. Motivated by this perspective, we propose EfficientMixer, a lightweight vision architecture that combines hierarchical feature scaling, isotropic spatial–channel mixing, and lightweight channel–spatial attention refinement, with each principle serving a complementary role within a unified architecture. We evaluate this composition under identical training settings and comparable parameter budgets across multiple image-classification datasets using baseline comparisons, component ablations, and computational-efficiency measurements. EfficientMixer consistently outperforms representative lightweight models while maintaining a constrained resource budget and achieves further performance gains when combined with Self-Competitive Distillation, showing that the architecture can benefit from improved training methods. These results show that systematically composing complementary architectural principles can improve representation quality and generalization in lightweight vision models, providing a practical design strategy for performance-constrained settings.