A Compact Probabilistic Recurrent Operator for Forecasting Climate-Sensitive Disease Incidence under Partial Observability

Public health agencies need reliable forecasts of infectious diseases to prevent epidemics with limited routinely available epidemiological and climate data. In this research, we introduce a compact probabilistic operator to forecast disease incidence under real-world information constraints. The operator encodes climate inputs separately using one-way encoding, retains recent dengue incidence with a gradually decaying memory, bounds month-wise changes in the internal disease state, and anchors the next-month forecast to the most recently observed incidence. We combined historical dengue surveillance and meteorological data covering 90 countries from 1990 to 2024. Data from 75 countries contained at least one pair of 12 months of history and next-month dengue incidence. Using a threshold of 1.5 standard deviations above the mean, we found that 4.23% of the 4,449 histories from 73 countries with at least one eligible non-outbreak history were followed by an outbreak month. We empirically measured how much variability in next-month dengue incidence can be predicted from observed disease incidence and climate data using a nearest-neighbor analysis ($k=25$). The log-scale $R^2$ on the held-out set was $0.730$ (country-bootstrap 95% confidence interval: $0.586$–$0.808$). Our forecaster has 89,652 parameters. We trained it independently five times using different random initializations and report ensemble forecasts by averaging the predicted outputs of the five fitted operators. When evaluated against observations on the original observation scale, the ensemble attained $R^2=0.713$ on the test set. After split-conformal calibration, its nominal 90% prediction intervals achieved empirical marginal coverage of 95.6%. We also compared the operator with a matched recurrent model lacking the four constraints. The operator reduced disagreement across independently trained models by $2.78\times$ and sensitivity to small input perturbations by $1.84\times$. It also outperformed negative-binomial and SARIMAX-style models in matched assessments. Across seeds 42–46 on fold 0, the operator achieved the lowest mean SMAPE in a comparison with 34 baseline models on the data-rich cohort. Performance worsened when the historical data length was short. On 156 matched test country-months from 13 countries, all-period outbreak average precision (AP) was 0.958 for SPECTRA and 0.907 for the pooled SARIMAX-style comparator. Among 64 months following a non-outbreak month, sudden-onset AP was 0.657 and 0.695, respectively, against an event prevalence of 0.219. We also assessed model accuracy when transferring the model to countries not used during training. The model achieved positive log-scale $R^2$ in 12 of 13 countries included in training and in 15 of 17 previously unseen countries with at least 10 years of historical data and at least 30% of months having nonzero target values, without country-specific retraining. We showed that the operator can also be used in resource-constrained settings.