Beyond Model Choice: A Harness-Centered Framework for Designing Clinical Large Language Model Agents

Clinical large language model agents depend on both model capability and the surrounding system that selects information, constrains actions, and verifies execution. In this Viewpoint, we argue that this harness should be an explicit unit of clinical agent design, evaluation, and reporting. Our aim is to help clinical informatics and development teams specify this layer and to help evaluators and institutional governance teams assess its contributions and risks. Selected benchmarks show substantial performance differences across harness configurations, but architecture comparisons, bundled improvements, and within-model component comparisons support different levels of causal attribution. We propose a revisable 3-tier framework: tier 1 specifies general orchestration functions, including prompts, tools, output schemas, control flow, memory, and generic verification; tier 2 specifies their clinical adaptations for knowledge integration, differential reasoning, missingness and temporality, safety verification, and workflow alignment; and tier 3 specifies a failure-driven evaluation cycle applied to both. Our contribution is a framework that makes these connections explicit. We provide classification rules, an illustrative clinical example, a proposed evaluation protocol, and a minimum reporting checklist. Observable execution traces are distinguished from private model reasoning, with provenance and privacy safeguards supporting retrospective reconstruction. Evaluation should separate task success, clinical correctness, process safety, and operational reliability. Lightweight fine-tuning can complement harness design; their interaction requires controlled study. The framework is a proposal for empirical testing, not a validated taxonomy or evidence of deployment safety.

Authors

Publication Details

Journal
Journal of Medical Internet Research
Published
2026-11-09
DOI
https://doi.org/10.2196/101269
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Beyond Model Choice: A Harness-Centered Framework for Designing Clinical Large Language Model Agents

Siru Liu, Jialin Liu
Journal of Medical Internet Research
Artificial Intelligence in Healthcare and Education
article

Beyond Model Choice: A Harness-Centered Framework for Designing Clinical Large Language Model Agents

Siru Liu, Jialin Liu
article en

Abstract

Clinical large language model agents depend on both model capability and the surrounding system that selects information, constrains actions, and verifies execution. In this Viewpoint, we argue that this harness should be an explicit unit of clinical agent design, evaluation, and reporting. Our aim is to help clinical informatics and development teams specify this layer and to help evaluators and institutional governance teams assess its contributions and risks. Selected benchmarks show substantial performance differences across harness configurations, but architecture comparisons, bundled improvements, and within-model component comparisons support different levels of causal attribution. We propose a revisable 3-tier framework: tier 1 specifies general orchestration functions, including prompts, tools, output schemas, control flow, memory, and generic verification; tier 2 specifies their clinical adaptations for knowledge integration, differential reasoning, missingness and temporality, safety verification, and workflow alignment; and tier 3 specifies a failure-driven evaluation cycle applied to both. Our contribution is a framework that makes these connections explicit. We provide classification rules, an illustrative clinical example, a proposed evaluation protocol, and a minimum reporting checklist. Observable execution traces are distinguished from private model reasoning, with provenance and privacy safeguards supporting retrospective reconstruction. Evaluation should separate task success, clinical correctness, process safety, and operational reliability. Lightweight fine-tuning can complement harness design; their interaction requires controlled study. The framework is a proposal for empirical testing, not a validated taxonomy or evidence of deployment safety.

Journal of Medical Internet ResearchVol. 28
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon