Swarm of Specialists: Can Multi-Model Ensembles of Fine-Tuned 7-8B Specialists Surpass Monolithic 70B+ Foundation Models?

The dominant paradigm in large language model deployment favors monolithic architectures: single large models (70B–400B+ parameters) trained on diverse data and tasked with general-purpose reasoning. This paper presents an alternative approach developed at Hayula Labs: ensembles of fine-tuned 7-8B parameter specialist models, each optimized for a narrow domain, coordinated by a lightweight routing layer.