From Idiographic to Nomothetic Prediction and Back: Leveraging big data for real-time stress prediction

The failure of population derived models to adequately predict early warning signals (EWS) for mental health conditions has led to an increased interest in personalized or idiographic approaches. However, idiographic methods face two large challenges: 1) scaling of methods to big data, and 2) generalizing model results to new participants. Here we address these challenges in EWS detection by leveraging the WARN-D dataset (n = 1,212) to predict 3 months of momentary stress data, collected via smartphones up to four times a day, based on a combination of smartphone-derived affective and context measures as well as smartwatch-derived passive sensing data. We fit three models: First, idiographic Long Short-Term Memory (LSTM) networks using an offline prediction framework, in which each participant's data was randomly partitioned into training and test sets (80/20 split) to optimize within-person predictions. Second Online version of the same models, capable of continuously updating as new data became available, allowing for real-time adaptation and forecasting of subsequent stress levels. Third, an ensemble model to aggregate predictions from 494 idiographic LSTM models to identify generalizable patterns applicable to unseen individuals. Step four, applying these models to unseen individuals, is still work in progress. At the group level, both offline (AUC=0.51, range=0.37-0.99) and online (AUC=0.50, range=0.380.84) models performed poorly, but had high variability in performance across individuals. Offline models performed best in individuals who reported higher levels of stress. Ensemble models delivered slightly improved performance (0.57, range=0.40-0.89), indicating potential to pool resources from idiographic models for out-of-sample prediction. In conclusion, we show that idiographic approaches vary greatly in their success of predicting stress across participants, and that this variation may be key for models to improve prediction accuracy. Combining insights from multiple participants results in better predictions, showcasing a way back from idiographic to nomothetic science.