Exploring trust in AI-supported military teams using sentiment analysis

Abstract Examining sentiment in team communications can provide information about trust among teammates. Natural language processing (NLP) models provide an efficient means of sentiment analysis. However, military teams and other professional teams use language that differs from what NLP models are trained on, leading to potentially inaccurate sentiment analysis. This study investigates the novel application of two advanced NLP models, DistilBERT and GPT-2, for sentiment analysis of expert military teams conducting AI-supported combat missions in a high fidelity simulation environment. Our fine-tuning process resulted in improved sentiment classification accuracy. The sentiment measures also correlated with measures of team trust and trust in the AI systems, providing valuable insight into the relationship between sentiment and trust in human-AI teaming scenarios. The generalized approach we describe may be useful for adapting sentiment analysis and NLP techniques to military teams, and may help measure trust dynamics and team states in human machine integrated teams.