TurnShift: Sequential Detection of Mid-Conversation Author Substitution via Behavioral User Profiling

Large language models deployed in conversational systems face a class of threats that content-based safety mechanisms cannot adequately address: multi-turn request sequences that appear benign in isolation yet pursue account takeover, credential harvesting, or social engineering when aggregated across a user's interaction history. TurnShift adapts UEBA-style behavioral profiling (EMA embedding centroids, online linguistic statistics, temporal patterns) to conversational-LLM anomaly detection, with emphasis on sequential detection. A first-pass per-message evaluation looked strong (F1 up to 0.794) but collapsed to F1 = 0.008 at realistic anomaly prevalence (0.29%) — a base-rate artifact diagnosed and analyzed in the paper. Sequential CUSUM accumulation over per-user embedding residuals recovers the signal, detecting mid-conversation author substitution with AUC 0.974 (63.0% episode detection at a 1-per-1,000 false-alarm budget) on PersonaChat, replicated on Blended Skill Talk, with a length-matched placebo control confirming the effect is authorship-driven. The paper's contribution is twofold: a diagnosed evaluation-methodology failure common in sparse behavioral anomaly detection, and evidence that cross-session behavioral security should be framed as sequential episode-level detection rather than per-message classification.