Hayula Gateway: A Unified OpenAI-Compatible API Layer for Multi-Model Local Inference

We present Hayula Gateway, a unified API routing layer that provides OpenAI-compatible endpoints for multiple locally-hosted large language models running on a single Apple M2 Ultra workstation (192GB unified memory). The gateway orchestrates three distinct model services—SAIF (a multi-expert cybersecurity suite on port 8443), Averroes-Q (a bilingual Arabic-English model on port 8080), and DeepSeek R1 (a reasoning-optimized model on port 8083)—exposing them through a single /v1/chat/completions interface. Hayula Gateway implements intelligent request routing, load-aware dispatching, LRU-based model lifecycle management with hot-swap capability, and configurable fallback chains (SAIF → DeepSeek R1 → OpenRouter) for reliability. The system achieves zero cloud dependency for primary inference while maintaining sub-200ms routing overhead on the M2 Ultra's 76-core GPU. We describe the architecture, routing strategies, memory management under the 192GB unified memory constraint, and production deployment via launchd on macOS. Hayula Gateway is the central inference fabric of the Hayula ecosystem, demonstrating that a single consumer workstation can replace multiple cloud API subscriptions for diverse LLM workloads.