An In-Depth Examination of Interruptions’ Effect on Email Classification Behavior

Identifying and avoiding phishing emails is a critical task email users engage in, and performance directly impacts the degree of risk to which users are exposed. Though the impact of email, task, and participant characteristics on email classification performance have been studied extensively, the effect of environmental factors, such as interruptions, have been understudied. We recently demonstrated that limited task interruptions improved email classification ability, but increased time on task (Slifkin & Neider, 2023). The present studies build upon this work by evaluating the effect of interruptions on email classification and analyzing eye movements to provide an in-depth characterization of behavior. In three experiments, we assessed the effects of interruption length (Experiment 1), interruption difficulty (Experiment 2), and task-interruption similarity (Experiment 3) on email classification performance. Participants evaluated the legitimacy of 100 real emails (50% phishing), indicating their response by button press. 20% of trials included an interrupting math task 3s after trial onset. Within each experiment, participants experienced both interruption types, and uninterrupted trials. Preliminary results show across all three experiments, email classification accuracy was unaffected by interruption, regardless of interruption type (all ps>.087), but response time reliably increased (all ps<.003). To further characterize interruptions’ impact, we analyzed the number of fixations post-interruption to locations fixated pre-interruption. Higher refixation rates indicate a disruptive effect of interruptions on previously encoded email information; lower refixation rates indicate better preservation of previously encoded information. Results were mixed across experiments; Experiments 1&3 showed an increase in refixations (all ps<.039), while Experiment 2 did not (p=.075). Overall, participants maintained accuracy when interrupted, but took longer to classify emails. In Experiments 1&3 interruptions appear to have disrupted representation of email information accrued prior to interruption; Experiment 2 revealed a better-preserved representation. Combined, these findings suggest that certain interruption types may disproportionately impair email classification performance.