From Multimodal Observables to Latent Trustiness: Mathematical Modeling of Human Trust in AI Interaction

Continuous assessment of human trust in artificial intelligence (AI) is fundamentally a latent-state identification problem: trust cannot be observed directly, while its behavioral manifestations are noisy, multimodal, and often shared with other internal states such as cognitive load, uncertainty, and attention. This work introduces trustiness as an operational latent variable for characterizing the time-varying trust-related state of a user interacting with an AI system. Rather than treating behavioral measurements as direct proxies for trust, we formulate their relationship with trustiness within a multimodal state-space framework that explicitly separates latent trust, cognitive load, contextual factors, and noisy observations. Candidate gaze, head-motion, and interaction-derived features are characterized through marginal, conditional, and incremental mutual information to determine which observables retain trust-related information after accounting for confounding and multimodal redundancy. The resulting informative observation set is integrated into a recursive latent-state estimator, instantiated through an Extended Kalman Filter (EKF) for continuous inference. In a controlled study with 60 participants, gaze entropy, revisit behavior, verification actions, response latency, and inspection time provided the strongest independent trust-related evidence, while conditioning on cognitive load substantially reduced several otherwise apparent associations. The selected multimodal model achieved a mean absolute error (MAE) of 0.112, a root mean square error (RMSE) of 0.149, and a Pearson correlation of 0.79 with self-reported trust across all 60 participants, evaluated on post-calibration interactions after participant-specific Trustiness Calibration Experiment (TCE) calibration. Participants were held out from population-level feature selection and hyperparameter tuning. Calibration was conclusive for 57 of 60 participants, for whom the corresponding MAE was 0.108, RMSE was 0.143, and Pearson correlation was 0.81. Joint modeling of trustiness and cognitive load reduced MAE by 18.2% relative to a trustiness-only model in the conclusive-calibration evaluation, while conditional feature selection improved generalization compared with the complete unfiltered observation set. Temporal experiments further showed coherent degradation and recovery of the inferred latent state following controlled changes in AI reliability. These results indicate that continuous trust assessment is better formulated as the identification of a partially observable latent state from confounded multimodal evidence than as direct prediction from behavioral proxies, providing a mathematical framework for multimodal trust estimation in human–AI interaction.

Authors

Institutions

Publication Details

Journal
Mathematics
Published
2026-11-20
DOI
https://doi.org/10.3390/math14224274
Primary Topic
Human-Automation Interaction and Safety
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

From Multimodal Observables to Latent Trustiness: Mathematical Modeling of Human Trust in AI Interaction

Fabíola da Silva Rosa, Massimo Villari, Antonino Galletta, Pierluigi Dell’Acqua
2 links
Mathematics
Human-Automation Interaction and Safety
article

From Multimodal Observables to Latent Trustiness: Mathematical Modeling of Human Trust in AI Interaction

Fabíola da Silva Rosa, Massimo Villari, Antonino Galletta, Pierluigi Dell’Acqua
article en

Abstract

Continuous assessment of human trust in artificial intelligence (AI) is fundamentally a latent-state identification problem: trust cannot be observed directly, while its behavioral manifestations are noisy, multimodal, and often shared with other internal states such as cognitive load, uncertainty, and attention. This work introduces trustiness as an operational latent variable for characterizing the time-varying trust-related state of a user interacting with an AI system. Rather than treating behavioral measurements as direct proxies for trust, we formulate their relationship with trustiness within a multimodal state-space framework that explicitly separates latent trust, cognitive load, contextual factors, and noisy observations. Candidate gaze, head-motion, and interaction-derived features are characterized through marginal, conditional, and incremental mutual information to determine which observables retain trust-related information after accounting for confounding and multimodal redundancy. The resulting informative observation set is integrated into a recursive latent-state estimator, instantiated through an Extended Kalman Filter (EKF) for continuous inference. In a controlled study with 60 participants, gaze entropy, revisit behavior, verification actions, response latency, and inspection time provided the strongest independent trust-related evidence, while conditioning on cognitive load substantially reduced several otherwise apparent associations. The selected multimodal model achieved a mean absolute error (MAE) of 0.112, a root mean square error (RMSE) of 0.149, and a Pearson correlation of 0.79 with self-reported trust across all 60 participants, evaluated on post-calibration interactions after participant-specific Trustiness Calibration Experiment (TCE) calibration. Participants were held out from population-level feature selection and hyperparameter tuning. Calibration was conclusive for 57 of 60 participants, for whom the corresponding MAE was 0.108, RMSE was 0.143, and Pearson correlation was 0.81. Joint modeling of trustiness and cognitive load reduced MAE by 18.2% relative to a trustiness-only model in the conclusive-calibration evaluation, while conditional feature selection improved generalization compared with the complete unfiltered observation set. Temporal experiments further showed coherent degradation and recovery of the inferred latent state following controlled changes in AI reliability. These results indicate that continuous trust assessment is better formulated as the identification of a partially observable latent state from confounded multimodal evidence than as direct prediction from behavioral proxies, providing a mathematical framework for multimodal trust estimation in human–AI interaction.

MathematicsVol. 14(22)
University of Messina (IT)
Human-Automation Interaction and Safety
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon