Stress-testing consentAI: a pipeline for evaluating a conversational educational adjunct to surgical informed consent

Informed consent is delivered inconsistently, and the comprehension it should produce is often poor. We built ConsentAI, a conversational educational adjunct that guides patients through a surgeon-authored consent form and gates progression on demonstrated understanding. Before any patient-facing trial, we asked how such an agent can be evaluated at scale, and which methods can be trusted. Eight clinicians scored 50 simulated laparoscopic cholecystectomy sessions on a 10-item rubric. We tested four LLM judge architectures against this clinician reference under a pre-specified, imbalance-robust plan. No single judge reproduced clinicians’ failure calls. Only jury aggregation improved sensitivity (0.421; 95% CI, 0.26–0.61), and no architecture reached a usable triage point. A constant pass-everything labeler topped raw agreement and AC1 yet scored zero on Matthews correlation and Cohen’s kappa. Evaluating clinical conversational agents therefore requires multiple clinician ratings and imbalance-robust metrics, treating clinician adjudication as a measurement layer, not ground truth.