ENHANCING CONVERSATIONAL AGENTS USING ROTATIONAL ATTENTION AND GATED SPLINE MODULES

In natural language understanding, transformer models like T5 and GPT have achieved strong results in generating contextually relevant responses. However, limitations such as static self-attention in T5 and unidirectional context in GPT hinder their ability to capture deeper inter-token dependencies and nuanced semantics. To address these challenges, we propose an enhanced T5 (ET5) architecture integrating two novel modules: contextual dual-axis rotational attention (CDARA) and neural-spline gated linear units (NS-GLU). CDARA facilitates attention across both token and feature dimensions, while NS-GLU introduces adaptive spline-activated gating for improved nonlinear representation. Experiments on NarrativeQA, SQuAD, MultiWOZ, and DailyDialog show that ET5 consistently outperforms PEGASUS, GPT-3, and T5-LSTM FusionNet. ET5 achieves superior BERTScore (up to 0.971), BLEU (up to 0.77), and lower word error rate (WER) (as low as 0.13), confirming its effectiveness in generating fluent, accurate, and semantically rich responses. These results position ET5 as a promising advancement in transformer-based conversational AI systems.