Artificial intelligence-based analysis of digital media engagement and user sentiment using machine learning techniques

Abstract Digital media datasets consist of both text content and platform, campaign, temporal, and engagement data, but precomputed sentiment and engagement data can lead to target leakage and overestimation of model performance. This study introduces the Hybrid Textual Sentiment Engagement Learning Model (HTSELM), a leakage-controlled late-fusion model for sentiment and engagement analysis. The experiments are conducted with the machine-generated Social Media Engagement Dataset, which has 28 raw columns and 12,000 records. Deterministic Affective Proxies are removed prior to training for the primary sentiment task. The text branch is a TF-IDF Linear Support Vector Machine with a weight of 0.70, and the structured branch is a HistGradientBoosting branch with a weight of 0.30, assigned by validation. On the held-out test set, HTSELM achieves an accuracy of 0.9392, macro-precision of 0.9471, macro-recall of 0.9336, macro-F1 of 0.9398, weighted-F1 of 0.9392, Matthews correlation coefficient of 0.9043, Cohen’s kappa of 0.9040, macro ROC-AUC of 0.9923, and macro PR-AUC of 0.9862. The Mean macro-F1 score for Exact 5-fold cross-validation is 0.9398 with an SD of 0.0069. Ablation shows that the dominant signal is text, and the validation-tuned fusion branch offers a relatively small but stable improvement over the text-only branch. There is minimal degradation due to numeric noise, but missing features and text corruption result in gradual degradation. A prospective engagement experiment (excluding engagement-rate components and post-response variables) is similarly close to chance, and the released pre-response variables are not reliable predictors of engagement. The results highlight the importance of leakage auditing, clear feature fusion and careful interpretation when analyzing digital media.