Beyond heuristics: predicting user attention with computational salience

Abstract Predicting user attention within complex interfaces presents a significant challenge for designers. Traditional heuristic approaches, such as visual hierarchy, have demonstrated limited empirical validity. We present computational salience modeling as a potential alternative to a classic heuristic approach. In doing so, we summarize major advances in salience modeling and explore what is needed for this computational approach to become an effective tool. Previous research has demonstrated the robustness of salience model predictions within the context of interface design and task manipulations. Salience models have reliably predicted the deployment of overt attention across diverse interface contexts, including webpages and mobile displays. Notably, they remain predictive under both free-viewing and goal-directed search tasks, and across different levels of visual clutter. Higher levels of visual salience consistently improve search efficiency. Recent models show enhanced predictive performance by integrating computational salience with empirically derived spatial convention maps reflecting learned experiences. Computational salience modeling can provide designers with a useful, data-driven method for predicting attentional guidance. It offers objective identification of visually salient elements and provides insight into improved search efficiency.