Fast and trustworthy nowcasting of dengue fever: A case study using attention-based probabilistic neural networks in São Paulo, Brazil

Nowcasting methods are crucial in infectious disease surveillance, as reporting delays often lead to underestimation of recent incidence and can impair timely public health decision-making. Accurate real-time estimates of case counts are essential for resource allocation, policy responses, and communication with the public. In this paper, we propose a novel probabilistic neural network (PNN) architecture, named NowcastPNN, to estimate occurred-but-not-yet-reported cases of infectious diseases, demonstrated here using dengue fever incidence in São Paulo, Brazil. The proposed model combines statistical modelling of the true number of cases, assuming a Negative Binomial (NB) distribution, with recent advances in machine learning and deep learning, such as the attention mechanism. Uncertainty intervals are obtained by sampling from the predicted NB distribution and using Monte Carlo (MC) Dropout. Using proper scoring rules for the prediction intervals, NowcastPNN achieves nearly a 30% reduction in losses compared to the second-best model among other state-of-the-art approaches. While our model requires a large training dataset (equivalent to two to four years of incidence counts) to outperform benchmarks, it is computationally cheap and outperforms alternative methods even with significantly fewer observations as input. These features make the NowcastPNN model a promising tool for nowcasting in epidemiological surveillance of arboviral threats and other domains involving right-truncated data.