Leveraging LLM generated labels to reduce bad matches in job recommendations

Negative signals are increasingly employed to enhance recommendation quality. However, explicit negative feedback is often sparse and may disproportionately reflect the preferences of more vocal users. Commonly used implicit negative feedback, such as impressions without positive interactions, has the limitation of not accurately capturing users’ true negative preferences because users mainly pursue information they consider interesting. In this work, we present an approach that leverages fine-tuned Large Language Models (LLMs) to evaluate recommendation quality and generate negative signals at scale while maintaining cost efficiency. We demonstrate significant improvements in our recommendation systems by deploying a traditional classifier trained using LLM-generated labels.