The nature and computation of attentional effort: A peak/end rule integrating over moment-by-moment effort during multiple-object tracking

So much of perception is effortless, but a hallmark property of sustained visual attention is a vivid sense of effort. Nowhere is this more evident than during multiple object tracking (MOT), where keeping track of a group of moving targets amidst identical moving distractors involves a distinct sense of exertion. But where does this sense of effort come from? The answer is not immediately obvious, in part because of the dynamic nature of MOT: each MOT trial is almost an experiment unto itself, with a moment-by-moment ebb and flow of effort, as the proximities of targets and distractors constantly change. Accordingly, we asked a straightforward question (with a surprising answer): how does the feeling of retrospective effort (at the end of a trial) relate to the moment-by-moment experience of effort during a trial? To find out, we augmented MOT in two ways. First, during tracking, subjects reported their moment-by-moment sense of effort using a continuous dial (with the continuously varying pitch of a tone providing feedback that did not interfere with tracking). Second, immediately after each trial, subjects used a slider to report how effortful tracking was overall. Retrospective effort was not simply the average of moment-by-moment effort, but rather was best explained by certain brief moments — especially the *peak* effort, and the effort near the *end* of each tracking interval. These moments provided maximal predictive power: adding the other moment-by-moment effort ratings did not improve prediction of retrospective effort, and this was not true for any other temporal windows. This peak/end pattern is characteristic of retrospective reports of many other properties at longer time-scales — from the joy of a vacation, to the pain of a surgery. These results thus demonstrate a striking convergence between a hallmark effect of cognition and the moment-by-moment dynamics of visual phenomenology.