Enhancing conversational agent responses with EXLNetT using learnable enhanced Laplacian kernel attention mechanism, deep bi-affine network, and hybrid positional encoding

Abstract In natural language computing, AI-based Conversational Agents are pivotal in enhancing human-computer interaction. Although transformer models like XLNet and GPT-3 have shown strong performance, conventional self-attention mechanisms still struggle to capture intricate token relationships, limiting text generation and comprehension. To address these limitations, the Enhanced XLNet Transformer (EXLNetT) introduces several architectural innovations. Self-attention is replaced with Learnable Enhanced Laplacian Kernel Attention to better model complex dependencies. The standard Feed Forward Network is substituted with a Deep Bi-affine Network (DBAN) to strengthen contextual relationship learning. Additionally, hybrid positional encoding enhances token order representation, while a Deep Neural Network (DNN) is utilized to fine-tune attention weight scaling during training. EXLNetT demonstrates substantial performance gains across four benchmark QA datasets: NarrativeQA, SQuAD, TriviaQA, and InsuranceQA, achieving BLEU scores of 0.771, 0.714, 0.728, and 0.743, respectively, with reduced Word Error Rate (WER) scores of 0.187, 0.208, 0.213, and 0.186. ROUGE-W scores also improve, reaching 0.919, 0.923, 0.905, and 0.920. Compared to GPT-3, EXLNetT shows BLEU gains up to 4.61%, WER reductions up to 16.22%, and ROUGE-W improvements up to 4.43%, highlighting its effectiveness in generating accurate and coherent responses.