Disentangled Abstract Meaning Representation Attention Augmented DeBERTa Model

We propose a DeBERTa model with disentangled abstract meaning representation attention to improve the model's semantic understanding and adaptability on complex downstream tasks. With this novel attention mechanism, the model can encode textual information along with abstract meaning representation, enhancing the understanding of complex semantics. To fully utilize the graph structural features, we also introduce a novel multi-scale graph aggregation method to model and propagate graph vertex features from different distances, inducing efficient information aggregation and semantic modeling. A joint masked language and abstract meaning modeling task is also designed for a pre-training stage, jointly predicting masked words and abstract meaning representations, preparing the model for downstream finetuning. The experiments show that our DeBERTa model with the novel attention mechanism exhibits significant performance improvement, providing a promising solution for various downstream tasks.