A Supervised BERT Model for Identifying Core-Intent Bearing Phrases in e-Commerce Queries

In the realm of e-Commerce, a fundamental problem is accurate interpretation of users' core intent. The intent is often subtly expressed implicitly or stated explicitly with the usage of verbose tokens or key phrases in a user query. In this work, we focus on the later class of problems where we identify a subset of query tokens which are the primary intent bearing phrases that convey explicit intents. We did not solve this as an intent detection problem but rather an immutable component detection problem because we believe that discovering the immutable phrases in a query entails that those are the intent bearing phrases. Furthermore, identifying a certain set of query tokens as immutable ensures better downstream processing in terms of unprecedented token handling, query category detection or query rewrites. We have developed a BERT based supervised learned model which can identify core-intent tokens, thereby improving F1 score over the baseline by over 35%. Furthermore, we integrated our proposed approach for a query recovery strategy which produces approximately 11.9% improvement in offline relevance scores compared to the production model.