DRAM: Dynamic Range Modulation for Multimodal Attribute Value Extraction on E-Commerce Product Data

With the prosperity of e-commerce applications, the web data of products are presented by multiple modalities, e.g., vision and language. For mining the product characteristics, multimodal attribute values are crucial, which are extracted from textual descriptions, assisted by helpful image regions. However, most previous works (1) fuse the multimodal information within a newly learned range based on co-occurrence rather than language meanings and (2) predict the outputs within a range of all attributes rather than the product-related ones. These issues yield unsatisfactory results; thus, we propose a novel approach via Dynamic Range Modulation (DRAM): (1) First, we propose an Information Range Calibration (IRC) method to dynamically fuse multimodal features of related meanings as Text-Related Embeddings (TEM) within a language range, which is calibrated from the range to fuse language features by a powerful attention mechanism of a pretrained language model. (2) Moreover, an Attribute Range Minimization (ARM) method is proposed to minimize the output attribute range based on the adaptive selection of product-related attribute prototypes. Experiments on the popular multimodal e-commerce benchmarks show that our DRAM performs well compared with previous methods.