Bilinear Attention Based Learning on Graphs

The attention mechanism is a core module in today’s well-established models like Transformers and Graph Attention Networks (GAT). The attention coefficients needed typically requires heavy computational resources for stability and better predictions. This requirement limits the size of the sequence or graph a model can learn from, leading to separating connected graphs or sequences to ease training. In this paper, we introduce an attention mechanism that leverages Bilinear forms to handle sequences of arbitrary length, offering greater flexibility and scalability. This improvement not only optimizes computational resources but also offers better model performance. Furthermore, our Bilinear Attention Mechanism (BAM) can capture more complex relationships between nodes for less parameters count.