Transformer-based feature integration for predictive modeling in multi-source business environments

Abstract Predictive analytics in multi-source business environments encounter significant challenges when incomplete data sources compromise analytical accuracy. This study presents transformer-based feature integration mechanisms for handling missing data in prediction tasks, validated through a representative case study of real-world business data where 47.4% of records lack engagement information due to customer behavioral patterns. Within this framework, we systematically compare four integration architectures: self-attention, dual-stream, adaptive feature weighting, and fusion integration through adaptive gating mechanisms. In the internal architecture comparison, the proposed Gated Fusion architecture demonstrated statistically significant performance improvements over all three competing variants. Finally, we benchmarked the Gated Fusion architecture against XGBoost with native missing-value handling and the SAINT tabular Transformer as external baselines. While XGBoost demonstrated highly competitive overall performance, the fusion model showed a statistically significant adaptive advantage on records with missing engagement data and outperformed SAINT across all evaluation metrics. The findings demonstrate that the proposed architecture provides a practical and adaptive solution for handling systematic missingness in the specific context of multi-source transactional prediction tasks.