Understanding Visual Saliency of Outlier Items in Product Search

In two-sided marketplaces, items compete for user attention, which translates to revenue for suppliers. Item exposure, indicated by the amount of attention items receive in a ranking, can be influenced by factors like position bias. Recent work suggests that inter-item dependencies, such as outlier items in a ranking, also affect item exposure. Outlier items are items that observably deviate from the other items in a ranked list w.r.t. task-specific, presentational features. Understanding outlier items is crucial for determining an item's exposure distribution. In our previous work, we investigated the impact of different presentational features on users’ perception of outlierness in e-commerce search result pages. By modeling the problem as visual search tasks, we compared the observability of three main features: price, star rating, and discount tag. We found that participants perceive these features differently in terms of attention and reaction times. Various factors, such as visual complexity (e.g., shape, color), discriminative item features (e.g., a solitary discount tag), and value range, affect item outlierness. These factors can be categorized into two main classes: bottom-up and top-down . Bottom-up factors are driven by visual properties such as color, contrast, and brightness, while top-down factors are influenced by cognitive processes such as expectations and prior knowledge. In this extension of our previous work, we deepen our analysis of user perceptions of outliers. In particular, we focus on two key questions left unanswered by our previous work: (i) What is the effect of isolated bottom-up visual factors on item outlierness in product lists? (ii) How do top-down factors influence users’ perception of item outlierness in a realistic online shopping scenario? We start with bottom-up factors and employ visual saliency models to evaluate their ability to detect outlier items in product lists purely based on visual attributes. Then, to examine top-down factors, we conduct eye-tracking experiments on the same task as our previous visual search experiment: online shopping. This time, we design the task as a simulated e-commerce environment, mimicking a popular European online shopping platform to be more representative of real-world scenarios. Moreover, we employ eye-tracking to not only be closer to the real-world case but also to address the accuracy problem of reaction time in the visual search task. In our experiments, participants interact with realistic product lists, some containing outliers w.r.t. different presentational features, such as image, price, and discount tag, at different positions. Our experiments show the ability of visual saliency models to detect bottom-up factors, consistently highlighting areas with strong visual contrasts and attention hotspots. While the well-known Itti & Koch model detects general visual attention patterns in an image, a graph-based visual saliency model (GBVS) identifies visual anomalies more effectively. However, one should be cautious about the limitations of these models. Visual saliency models only rely on bottom-up factors, making them naive in that they do not distinguish between separate product features or compare them against each other. The results of our eye-tracking experiment for lists without outliers show that despite being less visually attractive, product descriptions captured attention the fastest, indicating the importance of top-down factors and user knowledge of the task. Our observations in lists with visual outliers suggest that outliers and their immediate neighbors attracted attention faster (in terms of time to first fixation), which is in line with our findings from the visual search task. However, in our eye-tracking experiments, we observed that outlier items engaged users for longer durations (in terms of fixation count and time spent) compared to non-outlier items. This effect was consistent across different outlier features (image, price, discount tag) and various positions within the list.