Preliminary evaluation of a typed decision model for counselling dialogue coding using motivational interviewing

Typed decision models return probabilities over predefined options, potentially supporting automated coding of motivational interviewing transcripts. The study compared Jev 1.13 with ten experts and two large language models (LLMs), Claude Opus 5.5 and GPT-6 sol, on seven motivational interviewing demonstrations. LLMs reported confidence as probabilities. Jev showed high panel agreement, although Claude Opus 5.5 had higher point estimates on both tasks. Against the ten-expert plurality, its estimated calibration error was lower than both LLMs', clearly so only against Claude Opus 5.5. Calibration comparisons depended on the reference; a prespecified target was missed against single coders. Retained successful LLM calls cost 64–175 times as much as Jev’s, with median response times 7–8 times as long. Costs and latencies were measured through one provider on one day. These preliminary findings motivate evaluation in independent clinical sessions and workflows combining automated coding with human review.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-20
DOI
https://doi.org/10.21203/rs.3.rs-11596861/v1
Primary Topic
Psychotherapy Techniques and Applications
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Preliminary evaluation of a typed decision model for counselling dialogue coding using motivational interviewing

Mor Bar, Zohar Elyoseph, Elad Refoua, David Piterman et al.
5 links
Research Square
Psychotherapy Techniques and Applications
preprint

Preliminary evaluation of a typed decision model for counselling dialogue coding using motivational interviewing

Mor Bar, Zohar Elyoseph, Elad Refoua, David Piterman, Hadar Dery, Ziv Ben-Zion, Shaked Seker
preprint en

Abstract

Typed decision models return probabilities over predefined options, potentially supporting automated coding of motivational interviewing transcripts. The study compared Jev 1.13 with ten experts and two large language models (LLMs), Claude Opus 5.5 and GPT-6 sol, on seven motivational interviewing demonstrations. LLMs reported confidence as probabilities. Jev showed high panel agreement, although Claude Opus 5.5 had higher point estimates on both tasks. Against the ten-expert plurality, its estimated calibration error was lower than both LLMs', clearly so only against Claude Opus 5.5. Calibration comparisons depended on the reference; a prespecified target was missed against single coders. Retained successful LLM calls cost 64–175 times as much as Jev’s, with median response times 7–8 times as long. Costs and latencies were measured through one provider on one day. These preliminary findings motivate evaluation in independent clinical sessions and workflows combining automated coding with human review.

Research Square
Harvard University (US), Bar-Ilan University (IL), Universität Trier (DE), University of Haifa (IL)
Azrieli Foundation
Psychotherapy Techniques and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Preliminary evaluation of a typed decision model for counselling dialogue coding using motivational interviewing — Mor Bar, Zohar Elyoseph, et al. · Research Square (2026) | TGRS Research Map | TGRS