SparseLinTab: Sparse linear self-attention for efficient feature interaction in tabular data

Tabular data supports intelligent decision-making in key areas such as finance and healthcare, but its heterogeneous feature interaction modeling and large-scale computing efficiency issues have long restricted the application of deep learning technology. This paper proposes SparseLinTab, an efficient tabular data modeling framework based on bidirectional sparse linear self-attention. By decoupling the interactions between rows (sample level) and columns (feature level), SparseLinTab reduces the quadratic computational complexity of traditional Transformer self-attention O ( N 2 M + N M 2 ) to linear O ( N M ) , while retaining the ability to model global dependencies. SparseLinTab outperforms all traditional gradient boosting models and deep learning models in terms of average performance across 7 public datasets. Specifically, the core metrics are calculated separately for each dataset (accuracy is used for classification tasks, and MAE for regression tasks), and then the simple arithmetic mean of the metric scores from all datasets is computed to balance the impact of different scenario characteristics on the overall performance evaluation. In addition, ablation experiments and attention visualization show that the sparse mechanism significantly enhances the robustness and generalization of the model by filtering noise interactions and focusing on key feature combinations.