Vertical Scaling of Subdomain-Specific Small Language Models in Complex Aerospace Engineering: Synthetic QA Generation, Fine-Tuning, and LLM-as-a-Judge Evaluation

While Large Language Models (LLMs) excel across broad general tasks, high-stakes engineering and physical science domains require strict numerical precision, structural safety compliance, and deep contextual reasoning. In this paper, we demonstrate the power of vertical scaling in Small Language Models (SLMs) adapted across 30 specialized aerospace engineering subdomains—including Aerodynamics, Hypersonic Flow, Aeroelasticity, Rocket Propulsion, Re-entry Thermodynamics, Flight Dynamics, and Structural Health Monitoring (SHM). By employing a targeted programmatic synthetic Question-Answering (QA) generation engine, we convert ∼1,000 raw seed topics into a high-density training corpus of 2,000–2,500 multi-turn instruction pairs (∼65–100 Q&As per subdomain). We fine-tune a localized 3.8B parameter SLM using Quantized Low-Rank Adaptation (QLoRA). Evaluating our fine-tuned SLM against state-of-the-art commercial baselines (gpt-4o-mini and gpt-3.5-turbo) via an automated gpt-4o LLM-as-a-Judge protocol across Accuracy, Completeness, and Reasoning (1–5 scale), our model achieves an overall average score of 4.53/5.00, outperforming gpt-4o-mini (4.22/5.00) and gpt-3.5-turbo (3.69/5.00). Crucially, our results confirm that hyper-fine-tuned localized SLMs can outperform leading commercial generalist LLMs at less than a quarter of the operational and inference cost. All model weights, evaluation datasets, generation scripts, and benchmarking code are open-sourced on GitHub and Hugging Face.