Effects of AI explanations on trust and reliance: a study in job shop scheduling

Trust calibration is critical for productive human-AI collaboration in production management, yet the impact of brief explanations on <i>trust</i> and <i>reliance</i> for black-box schedulers remains unclear. We conducted a between-subjects online experiment in flexible job shop scheduling (FJSS; <i>N</i> = 253 professionals and graduate engineers), comparing a deep reinforcement learning (DRL) scheduler augmented with natural-language rationales to a transparent first-in-first-out (FIFO) heuristic while holding recommendations identical to isolate explanation effects. In a five-step, path-dependent scheduling task, participants relied more on DRL with rationales, but <i>attitudinal trust</i> did not differ. Instead, trust increased through mediation of perceived ability. This indirect trust effect was smaller on <i>harder tasks</i> and larger among <i>domain experts</i>, whereas <i>reliance</i> was not moderated. Practically, short rationales function primarily as <i>ability cues</i> that raise adoption without necessarily deepening understanding. To maintain <i>calibrated trust</i>, these cues should be complemented with audience- and task-specific guidance on uncertainty and limitations.