TRIASE-SLO: Risk-Gated and SLO-Aware Routing Between FP16 and INT4 Language-Model Workers for Radiology Impression Serving

Language models can draft radiology impressions, but a 4-bit copy of a model does not always reproduce the output of its full-precision version. TRIASE-SLO routes each request between FP16 and AWQ INT4 copies of one LoRA-adapted Qwen2.5-3B-Instruct model. A frozen rule-based hazard score computed from the FINDINGS text sends every request scoring at least 0.5 to FP16, and a secondary latency-aware score places the remaining requests. We evaluated the design in a locked confirmation in which every arm reserved the same two A100 MIG slices: four arms, seven loads and 12 paired deployment blocks of 450 previously inspected test reports (151,200 requests, none failed). The gate improved agreement with FP16. With full TRIASE-SLO routing, agreement with finding-bearing FP16 references was 99.11–99.49%, compared with 66.15–82.31% under hazard-blind routing, while 34–48% of requests still used INT4. A consistent serving-efficiency advantage was not demonstrated: full routing did not improve tail latency or throughput over two FP16 workers (median p95 change +0.33%; slower at C = 2 after Holm correction), and a consistent speed advantage of INT4 appeared only under hazard-blind routing. The demonstrated benefits of the INT4 copy are a smaller checkpoint and a larger KV-cache capacity. The incremental value of the score remains uncertain, because its prespecified C = 8 throughput benefit over gate-only routing (+1.86%, 95% CI −3.27% to +7.27%) was not confirmed. No INT4-using arm met the zero-drift engineering floor. All quality measures are rule-based FP16-consistency proxies from one public dataset, not evidence of clinical correctness or safety.

Authors

Institutions

Publication Details

Journal
Journal of Computing Theories and Applications
Published
2026-11-19
DOI
https://doi.org/10.62411/jcta.18198
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

TRIASE-SLO: Risk-Gated and SLO-Aware Routing Between FP16 and INT4 Language-Model Workers for Radiology Impression Serving

Guruh Fajar Shidik, Junta Zeniarja, Aris Marjuni, Heru Seok Suhartanto
1 links
Journal of Computing Theories and Applications
Advanced Neural Network Applications
article

TRIASE-SLO: Risk-Gated and SLO-Aware Routing Between FP16 and INT4 Language-Model Workers for Radiology Impression Serving

Guruh Fajar Shidik, Junta Zeniarja, Aris Marjuni, Heru Seok Suhartanto
article en

Abstract

Language models can draft radiology impressions, but a 4-bit copy of a model does not always reproduce the output of its full-precision version. TRIASE-SLO routes each request between FP16 and AWQ INT4 copies of one LoRA-adapted Qwen2.5-3B-Instruct model. A frozen rule-based hazard score computed from the FINDINGS text sends every request scoring at least 0.5 to FP16, and a secondary latency-aware score places the remaining requests. We evaluated the design in a locked confirmation in which every arm reserved the same two A100 MIG slices: four arms, seven loads and 12 paired deployment blocks of 450 previously inspected test reports (151,200 requests, none failed). The gate improved agreement with FP16. With full TRIASE-SLO routing, agreement with finding-bearing FP16 references was 99.11–99.49%, compared with 66.15–82.31% under hazard-blind routing, while 34–48% of requests still used INT4. A consistent serving-efficiency advantage was not demonstrated: full routing did not improve tail latency or throughput over two FP16 workers (median p95 change +0.33%; slower at C = 2 after Holm correction), and a consistent speed advantage of INT4 appeared only under hazard-blind routing. The demonstrated benefits of the INT4 copy are a smaller checkpoint and a larger KV-cache capacity. The incremental value of the score remains uncertain, because its prespecified C = 8 throughput benefit over gate-only routing (+1.86%, 95% CI −3.27% to +7.27%) was not confirmed. No INT4-using arm met the zero-drift engineering floor. All quality measures are rule-based FP16-consistency proxies from one public dataset, not evidence of clinical correctness or safety.

Journal of Computing Theories and ApplicationsVol. 4(2)
University of Indonesia (ID), Universitas Dian Nuswantoro (ID)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon