Destination Heat Prediction Driven by Social Media Using Transformer-Based Temporal Modeling

Accurately modeling temporal dependencies in large-scale, high-dimensional datasets has become a critical focus in the field of machine learning. This paper proposes a novel Transformer-based framework designed for temporal modeling in irregularly spaced time-series data, addressing limitations in traditional methods such as ARIMA, RNNs, and LSTMs. The model introduces a temporal attention mechanism to dynamically weight time steps based on their relevance, as well as a dynamic positional encoding scheme that effectively represents irregular time intervals. These innovations allow the model to capture both long-term trends and short-term eventdriven anomalies, offering superior flexibility and precision. Extensive experiments conducted on large-scale datasets demonstrate the framework’s ability to significantly outperform baseline models, achieving lower RMSE and MAE scores across multiple temporal granularities. The results validate the scalability and interpretability of the proposed Transformerbased approach, highlighting its potential for advancing timeseries prediction tasks in diverse computational domains.