Review based Recommendation with Pre-trained Transformer and Hierarchical Attention Networks

On E-commerce platforms, users not only rate items using numerical scoring systems, but also frequently share their opinions about item experiences through textual reviews. Multiple studies suggest that examining user reviews is an effective way to understand consumer tastes and identify key item characteristics, thereby enabling more effective recommendations. The core problems that this approach needs to address include effectively utilizing textual reviews for recommendations, specifically how to represent the reviews, identify essential information within them, and aggregate this information to obtain representations for both users and items. In this work, we introduce SBRec, a novel approach that employs pre-trained models to extract meaningful user and item profiles from textual reviews. SBRec incorporates two levels of attention networks to capture the hierarchical structure of information within a collection of reviews. At the lowest level, sentences within reviews are modeled using a pretrained transformer to capture the full context. The sentences in each review are combined through an attention mechanism to create a review representation. At the higher level, another attention mechanism aggregates information from all of a user’s reviews to generate their overall representation. The item’s representation is constructed in a similar way and compared with the user’s representation to estimate the rating. We experimentally evaluated the proposed method on five benchmark Amazon datasets. Our experimental results demonstrate that our method surpasses previous baselines and cutting-edge review-based recommendation techniques.