Transformer Based Ensemble Framework For Sequential User Behavior Prediction

The goal of this research is to develop a sequence modeling approach to predict future user behavior over the next several days based on an analysis of the past behavior of users from the previous several days. By leveraging historical user data, we aim to accurately predict how users will interact with any given platform feature in the future, allowing for more personalized and targeted campaign strategies to be implemented. This predictive modeling approach provides valuable insights for optimizing user engagement, retention, and ultimately driving business growth. We start with LSTM models, variations of LSTMs, and experiment with attention mechanisms, exploring the latest transformer based sequence prediction models. We also propose a novel sequence model based on an ensemble of transformers and evaluate it.