Can a Transcript Alone Find What Viewers Replay? A Negative Result for Text-only LLM Highlight Scoring

Tools that cut short clips from long talks and webinars often hand the transcript to a large language model (LLM) and ask it to pick the best moments. We test whether that choice matches what viewers actually do. On 196 long, speech-heavy YouTube videos from the YTMR500 set, we score every 30-second window of the Whisper transcript with a local 7B instruction model and compare the scores to YouTube's "Most Replayed" heatmap. We report rank correlation, precision at the top 10%, and partial correlation with window position and word count controlled, next to random, position-only and word-count baselines. The LLM does slightly worse than random (mean Spearman -0.053, 95% CI [-0.090, -0.016]), while a position-only baseline reaches 0.163. The LLM's scores rise with the amount of speech in a window (rho = 0.19), but viewers replay windows with less speech (rho = -0.28), and once position and word count are controlled the LLM adds nothing (-0.002). It also rates the opening of a video lowest, where replay is highest. Scoring 8-window chunks without position cues does not help, and the results hold with 60-second windows. We release video IDs, window boundaries, labels, scores, prompts and code.