Evaluation of Generative AI in E-commerce Product Description Generation: An Experimental Study

Generative AI has emerged as a powerful tool for content generation in e-commerce, enabling the automated creation of product descriptions, reviews, and marketing copy. This paper presents a systematic evaluation of state-of-the-art generative AI models for e-commerce content generation using the WANDS dataset. We conduct a series of experiments to compare commonly used AI models based on their BLEU and ROUGE scores, assessing their effectiveness in generating high-quality, coherent, and contextually relevant content. Our analysis reveals distinct strengths for each model: GPT-4 excels in producing high-conversion SEO-driven sales copy, Claude generates engaging, storytelling-rich product descriptions, DeepSeek provides precise, factual descriptions for technical products, LLaMA offers simple, cost-effective product descriptions, and OpenHermes-2.5-Mistral-7B creates engaging and informative product descriptions suitable for e-commerce platforms. These findings highlight the importance of aligning AI model selection with specific e-commerce use cases and content objectives. By providing a comprehensive benchmark and targeted recommendations, this study offers valuable insights for businesses and researchers seeking to optimize AI-driven content generation strategies in online retail, enabling more effective and tailored implementation of generative AI in e-commerce contexts.