SLAFlow: Profile-Guided SLA-Aware Runtime Orchestration for Multi-Stage Edge AI Services

Multi-stage edge AI services must balance output quality against the latency of retrieval, generation, and refinement. SLAFlow uses profiles measured on the execution platform to jointly select retrieval depth, generation-token budget, and refinement depth under a request’s remaining deadline and revisits the feasibility and benefit of optional refinement during execution. We evaluate an ophthalmic workflow on a CPU-only workstation at a doctor’s desk, combining physical measurements with trace-driven policy simulation. Under nominal arrivals and the tight SLA, SLAFlow achieves a deadline-weighted quality (DWQ) of 0.668 with 2.7% violations, compared with 0.479 and 33.1% for the queue-based heuristic. Across four evaluated conditions, SLAFlow has observed mean DWQ differences of +0.004 to +0.022 relative to Token-Only; under Bursty/Tight, violations increase from 9.0% to 11.4%. Under Bursty/Tight, however, its observed DWQ gain over Static-Small is only 0.003, while violations increase from 1.5% to 11.4%. Independent calibration observes seven prediction exceedances in 180 records and three false-feasible decisions in 160 predicted-feasible cases. Matched queued executions have physical p95 end-to-end latencies 3.2% and 4.4% above simulation.