TransForeCaster: In-and-Cross Categorized Feature Integration in User Representation Learning

This paper introduces TransForeCaster, a novel user representation learning approach to improving prediction accuracy in purchase and churn predictions via a two-stage feature integration process: In-Category Integration (ICI) and Cross-Category Integration (CCI). The ICI stage employs a Time-Series Feature Mixer (TSFM) to capture the temporal dynamics of features within the same categories, resulting in compact and continuous category representations. The CCI stage utilizes a Meta-Conditioned Transformer (MCT) to integrate the representations with multi-task learning, capturing complex relationships across categories and improving interpretability through attention mechanisms. Empirical evaluations of real-world datasets demonstrate significant improvements over conventional models, supported by qualitative analyses using feature importance assessments and UMAP visualizations. TransForeCaster's robustness is validated by its superior performance over other models in multiple in-house deployments across various games and applications. The source code is available at https://github.com/bagelcode-data-science-team/TransForeCaster.