The Effect of Reviewer Heterogeneity Between One-Time and Multi-Review Reviewers, Price Sentiment, and Pre-Purchase Information on Star Ratings for Amazon Subscription Boxes

Abstract This study examines the presence, prevalence, and negative polarity of one-time reviewers compared to those who have left multiple reviews across all categories, based on an analysis of 571,544,746 Amazon product reviews. Since sellers and consumers may be interested in specific products relevant to their business or purchasing interests within a category rather than across all categories, this research focuses on the Subscription Boxes category for econometric analysis. This category exhibits an unusual rating distribution, likely presents fewer incentives for promotional reviews, and faces fewer constraints. However, price data is unavailable in this category. Therefore, this study develops price sentiment variables from review text data using generative LLMs and manual annotation. Specifically, the fine-tuned GPT-4.1 model outperforms other GPT models. Moreover, pre-purchase information from past reviews is extracted through data mining. Because star ratings are ordinal responses that indicate the strength of consumer preference, this study employs an ordered probit model and marginal effects analysis to interpret the results. The findings reveal variability in the marginal effects of reviewer type, purchase verification, and price sentiment on the likelihood of giving 1- or 5-star ratings. Additionally, consumers consider pre-purchase information from previous reviews when assigning star ratings. Thus, this approach can help market players make data-driven decisions when relevant consumer and business data are limited, and it can help bridge the information gap with Amazon, which operates both as a marketplace and as a seller on its own platform.