Usability of LLM-driven loss function in Automatic Prompt Engineering for e-commerce

This study explores the idea of automating the Prompt Engineering process using optimization techniques. It introduces the approach of evaluating generated descriptions using artificial customers that are powered by an LLM. The environment in which the proposed system was tested was in the context of e-commerce web pages, where customers are exposed to product descriptions. Prompt optimization was performed by generating evaluations on how good the description matched the customer’s preferences. Then, based on them system used LLM to improve the instructions to generate better fitting descriptions. To prove the usability of the achieved prompt, an A/B test was conducted on a retailer’s website, which supports the described theory with a real-world example.