Patent Text Representation Method Based on Masked Augmentation and Contrastive Learning

Patent text representation is an important task in patent analysis and mining. Patent texts not only contain technical terms but also involve complex semantic structures. Therefore, to deeply mine key information in patent texts, a patent text representation method is proposed. First, to address the issue of insufficient diversity among positive samples in contrastive learning, a keyword repetition method is used for data augmentation to construct positive examples. Second, the masked augmentation task in the ERNIE model is applied to mask keywords in patent texts, enabling the model to predict the masked words based on context and enhance text feature extraction. Then, the SimCSE strategy is employed for contrastive learning, improving text representation by leveraging sample differences. Additionally, the model undergoes domain-specific pretraining on patent corpora to acquire domain knowledge and enhance its capability in patent text representation. Experiments on the CCF Chinese patent classification dataset demonstrate that the proposed method improves the F1-score by 3.7% compared to the average, validating its effectiveness.