acceptodds
Under review as a conference paper at ICLR 2027

CHARM: Probabilistic Forecasting with Uncertain Historical Context

Abstract

Under changing predictive dynamics, a forecaster must make predictions while the relevance of its historical context remains uncertain. Our key insight is to reuse learned predictive knowledge across multiple candidate contexts, maintaining uncertainty over which historical context should inform the next prediction. We introduce CHARM (Changepoint-Aware Recurrent Memory), which maintains a recurrent state for each candidate history under a shared, frozen predictor. Bayesian online changepoint detection updates run-length probabilities using predictive likelihoods, and CHARM combines the candidate predictive densities under these probabilities to account for uncertainty over context boundaries. We evaluate CHARM on financial realized volatility, where persistent dependence makes history valuable while changing dynamics make its relevance uncertain. Experiments show that CHARM improves average predictive log scores over the same forecaster with continuous history, with gains varying across markets.

Primary area:
learning on time series and dynamical systems
Keywords:
Probabilistic forecasting, Bayesian online changepoint detection, historical context uncertainty, recurrent neural networks, online adaptation.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VrDYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CHARM, a probabilistic forecasting framework that maintains multiple candidate historical contexts using recurrent states under a shared frozen predictor. Bayesian online changepoint detection updates the probabilities of these contexts, and their predictive densities are marginalized accordingly. Experiments on financial realized volatility show improvements over continuous-history forecasting, particularly on Oxford-Man data.

Critical strengths
  1. CHARM provides a principled and relatively clean way to propagate uncertainty over historical context while reusing a single learned predictor.

  2. The experimental analysis is unusually careful, including shared-predictor controls and explicit comparisons of online weighting, static mixing, MAP selection, and predictive marginalization.

Critical weaknesses
  1. The empirical scope is narrow. The real-data evaluation is limited to financial realized volatility, and the shared-predictor benefit is clear on Oxford-Man but essentially absent on Binance.

  2. CHARM incurs computation proportional to the number of retained context hypotheses, yet its benefit depends on candidate contexts producing meaningfully different forecasts. The paper does not provide a clear criterion for when this additional inference cost is likely to be worthwhile.

Questions for authors
  1. Could the authors evaluate CHARM on a non-financial time-series domain to clarify whether the benefit generalizes beyond realized-volatility forecasting?

  2. Can the authors provide a practical diagnostic for determining when context marginalization is likely to help enough to justify its additional computation, together with runtime or memory results as Rmax⁡R_{\max} changes?

1 comment
AuthorsOpenReview ↗

Forecasting Beyond Finance and the Cost of Context Marginalization

Thank you for asking when context marginalization justifies its cost. We added two hourly forecasting tasks outside finance, a held-out validation diagnostic, and benchmarks across four run-length caps (§5.6 and the appendix).

Q1. Do the gains extend beyond financial volatility?

For next-hour Australian electricity demand, the equal-state average favored CHARM over GRU-global in all five seeds (mean NLL difference −0.03002, adjusted interval [−0.03951, −0.02046]). South Australia accounted for 91% of that net gain and also favored CHARM over tuned static (−0.02856, adjusted interval [−0.04495, −0.01053]). Across all five states, the gain over tuned static was uncertain (−0.00327, adjusted interval [−0.00746, +0.00166]). On both hourly transformer oil-temperature series, ETTh1 and ETTh2, both controls had lower NLL than CHARM in every seed.

Q2. When is the additional inference cost worthwhile?

We first checked whether different histories produced different forecasts. On validation targets supporting the long-history comparison, the mean CDF discrepancy between the 128-step candidate and GRU-global was 0.0221 on Oxford-Man and 0.000075 on Binance. The latter comparison covered 648 of 1,880 targets. A disagreement-based switch selected on early validation still required the candidate forecasts and did not improve on always using CHARM on later validation. On that later split, CHARM minus tuned static NLL was −0.01484 on Oxford-Man and −0.00045 on Binance.

For deployment, we compare CHARM, global, and tuned static by paired NLL on held-out validation dates and choose the best-scoring policy that meets the latency budget. The table gives median milliseconds per forecast on the same CPU, using two threads, batch size one, and five repetitions on one representative checkpoint per dataset.

DatasetGRU-globalCap 5Cap 22Cap 128Cap 256
Oxford-Man0.4850.5520.6730.8381.026
Binance0.2660.4590.5120.8310.957

Increasing the cap from 128 to 256 raised latency on both datasets. In the original two-seed test evaluation, it changed mean NLL by less than 0.0001 on either dataset.

Reviewer_D5YtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CHARM (Changepoint-Aware Recurrent Memory) for probabilistic forecasting when the relevant historical context is uncertain. It maintains multiple run-length-specific recurrent states under a shared, frozen GRU density predictor, updates their weights through Bayesian online changepoint detection, and marginalizes the candidate predictive densities without online parameter updates. The paper provides idealized predictive analysis and evaluates one-step daily log-realized-volatility forecasting on Oxford-Man and Binance, supplemented by shared-predictor controls and controlled AR(1) interventions.

Critical strengths

S1. Informative controls under a shared predictive model. The comparisons with continuous history, fixed windows, static mixtures, and MAP prediction hold the recurrent network and density head fixed. This design helps isolate the effects of context choice, likelihood-based weighting, and predictive marginalization. On Oxford-Man, the reported comparisons provide evidence for these contributions without attributing improvements to a stronger neural predictor. These controls are an informative part of the empirical evaluation.

S2. Careful statistical evaluation and transparent reporting. The evaluation separates fitting, validation, and testing chronologically and reports paired block-bootstrap intervals with multiplicity adjustments. The paper also explicitly acknowledges the limited improvement on Binance and the stronger aggregate synthetic performance of an independently tuned static mixture. This transparency strengthens confidence in the stated scope of the findings and helps distinguish supported conclusions from exploratory observations.

Critical weaknesses

W1. The methodological novelty beyond prior work is insufficiently established. CHARM combines a shared recurrent density predictor with Bayesian run-length inference and predictive marginalization. Closely related work, particularly MOCA (Harrison et al., Continuous Meta-Learning without Tasks, NeurIPS 2020), already maintains run-length-specific model states and marginalizes predictions over run lengths, including with recurrent predictors. While CHARM trains its predictor separately rather than through the changepoint filter, the substantive methodological advance provided by this distinction remains insufficiently explained. A more explicit comparison of the formulations and training objectives would help clarify the contribution.

W2. The comparison with MOCA does not isolate the proposed contribution. The SH-MOCA baseline uses frozen neural features of logHAR inputs and a Bayesian predictor, whereas CHARM uses a GRU with a mixture-density head. Their performance difference therefore cannot be attributed solely to the context-inference mechanism. Although the shared-GRU controls are useful, a recurrent MOCA comparison with matched architecture, density head, and comparable training and tuning budgets would better establish the advantage over this closely related approach.

W3. Robustness to the recurrent predictor’s training configuration remains unclear. The shared-predictor experiments use a GRU, with training horizons of 128 and 32 steps on the two real-data datasets. The paper also notes that its diagnostics cannot distinguish sufficient recent information from limitations of learned memory. Sensitivity to training context length, or validation with another recurrent backbone, would help establish whether the findings extend beyond the particular memory characteristics of the fitted predictor. This is a secondary concern about the scope of the evidence rather than an objection to using GRU itself.

Questions for authors

Please address the concerns outlined in the Critical Weaknesses section. A clearer methodological comparison and targeted supporting evidence maybe could improve my assessment.

1 comment
AuthorsOpenReview ↗

Comparison with Recurrent MOCA and Sensitivity to Training Length

Thank you for highlighting the shared-predictor controls. We clarified the training objectives in §3.2 and added a matched recurrent MOCA comparison and a training-length study in §5.6 and the appendix.

W1. What distinguishes CHARM from recurrent MOCA?

CHARM uses recurrent MOCA’s inference procedure. Both maintain a state per run length, update its weight using predictive likelihoods and a hazard, and marginalize the forecasts. The distinction is in training. MOCA differentiates the predictive mixture loss through the filter. CHARM scores the next observation at each training-crop prefix. A prefix constructs the same state as a test candidate of that depth, so its loss directly trains the density used for reweighting. Fitting the predictor separately lets us compare online and static weighting with identical candidate densities.

W2. Matched architecture and tuning budget

Since SH-MOCA uses a different predictor, we added an Oxford-Man comparison matching the 32-unit GRU, four-component Gaussian mixture head, initialization, minibatches, 600 updates, checkpoint schedule, and test filter. Giving both methods the same validation grid for the inference hazard yielded the same result. Across five paired seeds, mean test NLL was 0.89514 for CHARM and 0.89555 for recurrent MOCA. The paired difference was −0.00041 (adjusted date-block interval [−0.00406, +0.00342]). The interval allows an advantage for either objective.

W3. Sensitivity to training length

We varied training crop length from 32 to 128 with five paired seeds per dataset, holding the update count and test cap fixed. On Oxford-Man, CHARM’s mean NLL fell from 0.91024 to 0.89514. Its advantage over both controls grew in all five seeds, with the mean gain over tuned static increasing from 0.00238 to 0.00854. On Binance, NLL rose for CHARM (+0.04768) and GRU-global (+0.07913), and the change in CHARM’s gap to tuned static was uncertain. Length-128 crops score four times as many targets per update and are scarce in early Binance folds. These results therefore characterize sensitivity to the GRU training configuration.

Reviewer_UChYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CHARM, which combines a frozen recurrent density predictor with Bayesian online changepoint detection for forecasting under uncertain historical context. It maintains recurrent states for multiple candidate histories, updates their weights using predictive likelihoods, and marginalizes their predictive distributions. The paper provides idealized theoretical results and evaluates CHARM on financial realized-volatility forecasting and synthetic regime changes. Comparisons using the same recurrent predictor show clear gains on Oxford-Man but negligible improvement on Binance, while synthetic experiments reveal localized benefits from online weighting alongside an aggregate disadvantage relative to tuned static mixing.

Critical strengths

The strongest result is the comparison against methods sharing the same learned predictor on the Oxford-Man dataset. On the same dataset, it also improves over the validation-tuned static mixture and over CHARM-MAP, with adjusted intervals excluding zero. These comparisons substantiate contributions from both likelihood-based weighting and predictive marginalization. The fixed-window controls also help rule out the explanation that simply limiting historical context accounts for the improvement. This is the paper’s strongest acceptance argument: there is a measurable benefit that can be attributed to the context-handling procedure while keeping the underlying predictor fixed.

The chronological separation of fitting, selection, and evaluation, common scored targets, and paired block-bootstrap intervals with multiplicity adjustment strengthen the empirical evidence. The synthetic study also distinguishes the opportunity created by a beneficial reset from the improvement actually recovered by CHARM and the incremental contribution of online feedback. The authors explicitly report non-significant comparisons and cases where simpler alternatives perform better. These choices make the study informative about when context uncertainty matters and support confidence in the narrower positive findings.

Critical weaknesses

The methodological advance over recurrent MOCA is insufficiently established, and the theoretical results add limited technical novelty. MOCA already maintains predictive states for different run lengths, updates their probabilities using predictive likelihoods and a hazard transition, and marginalizes their predictions. Crucially, it also includes an explicit LSTM-based implementation; recurrent states under shared learned parameters are therefore already part of that framework ([1], sections 4.1-4.3). Its training procedure differentiates through the filtering computation.

CHARM’s apparent differences are its particular autoregressive GRU and mixture-density implementation, separate crop-based predictor training, and forecasting evaluation. These may be useful choices, but the paper does not sufficiently establish their significance relative to the existing framework. The SH-MOCA baseline does not resolve this concern: it uses a different shared-feature Bayesian predictor, so its comparison with CHARM changes the predictive model as well as the method under evaluation. A comparison with a matched recurrent implementation is especially important here.

The theory provides interpretation but does little to establish an additional contribution. Lemma 1 is the standard population log-loss optimality argument under realizability. Proposition 1 assumes exact candidate conditionals and a matching changepoint model. Proposition 2 follows from the KL chain rule and marginalization. These statements appear reasonable, but they neither control the errors of the learned, capped implementation nor establish when CHARM improves over simpler context policies. Moreover, section 3.2 explicitly acknowledges that the conditionals learned from training crops need not match those assumed in Proposition 1.

The presentation makes this contribution boundary harder to assess. Although the combination is specified mathematically in section 3, the paper would benefit from a direct explanation of what is inherited from MOCA, what changes, and why the changed training procedure should produce useful likelihoods for comparing candidate histories. A self-contained algorithm description or a data flowing image would help in the readability and overall understanding of the method.

The overall ranking in Table 1 is more favorable than the evidence attributable specifically to CHARM’s context mechanism. In Table 2, on Binance, CHARM and GRU-global achieve essentially the same NLL. Thus, the Binance results do not establish an improvement from adding context inference to the recurrent predictor, leaving the clearest evidence for the proposed mechanism concentrated on Oxford-Man.

The limited number of training seeds is a substantial concern: the real-data experiments use only two neural training seeds, and the synthetic experiments use one. The reported bootstrap intervals are conditional on these fitted models and therefore do not establish robustness to initialization and training randomness. Evaluation with more independent training seeds is necessary, together with seed-level paired comparisons and measures of variability, to determine whether the observed advantages are reproducible across trained predictors.

The evaluation also covers a single forecasting task (daily log realized volatility) across four equity indices and two cryptocurrency pairs. Additional datasets, preferably covering other nonstationary forecasting settings, are necessary to establish whether the method’s incremental benefit extends beyond Oxford-Man. Given the close relationship to MOCA, the paper needs stronger empirical evidence of a consequential and reproducible advantage. The current combination of limited training replication and narrow dataset coverage does not sufficiently establish that advantage.

As a minor presentation comment (which does not affect my recommendation), I find the opening quotation unnecessary: it adds little to the scientific motivation and makes the introduction less direct. I would prefer the paper to begin with a precise statement of the forecasting problem, the limitations of existing approaches, and the specific gap that CHARM addresses.

Questions for authors
  • Which inference or modeling step substantively distinguishes CHARM from recurrent MOCA when both use the same predictive architecture? Can you isolate the effect of separately training the predictor on crops versus training through the filtering objective, with matched architecture and training budget? The current SH-MOCA comparison does not isolate this difference.
  • Can the authors evaluate the method with more independent training seeds and report seed-level paired differences against GRU-global and the static mixture? With only two training seeds for the real-data experiments and one for the synthetic study, it is difficult to assess whether the conclusions are robust to initialization and training randomness. The reported bootstrap intervals, which condition on the fitted models, do not resolve this uncertainty.
  • Does CHARM provide a reproducible improvement over the same recurrent predictor and tuned static mixing on additional datasets, preferably beyond daily financial realized-volatility forecasting? Since the clearest incremental benefit is concentrated on Oxford-Man, broader evaluation would materially affect my assessment of the method’s significance.
1 comment
AuthorsOpenReview ↗

Matched Comparison with Recurrent MOCA, Replication Across Seeds, and Evaluation Beyond Finance

Thank you for highlighting the shared-predictor comparisons. We clarified the relationship to recurrent MOCA (§§3–4) and added a matched comparison, more training seeds, and two hourly forecasting tasks outside finance (§5.6 and the appendix). Δ is CHARM minus comparator test NLL. Negative values favor CHARM.

Q1. What distinguishes CHARM from recurrent MOCA with the same predictive architecture?

Both methods use recurrent MOCA’s inference steps. Its LSTM version already maintains a state per run length, updates weights using predictive likelihoods and a hazard, and marginalizes forecasts. MOCA trains through the filter’s mixture log loss. CHARM scores the next observation at each crop prefix. At depths covered in training, this uses the same state construction as a test candidate and directly trains the density used for reweighting. A crop can cross a change, so its conditional density can differ from the true run-length conditional in Proposition 1.

Since SH-MOCA uses a different predictor, we matched the 32-unit GRU, four-component Gaussian mixture head, initial weights, minibatches, 600 updates, and test filter on Oxford-Man. Each method selected its checkpoint on the same held-out crops using its own objective. Across five paired seeds, mean NLL was 0.895136 for CHARM and 0.895546 for recurrent MOCA. Mean Δ was −0.000409 (SD 0.000595, adjusted date-block interval [−0.004063, +0.003420]). The interval allows either objective to perform better. Fitting the predictor outside the filter lets us compare history policies with its parameters fixed.

Q2. Are the shared-predictor gains reproducible across training seeds?

We expanded from two to five independent seeds per financial dataset. The policies share a fitted GRU within each fold and seed. Rows follow seed-ID order within each dataset. GRU-global uses continuous history. Tuned static uses validation-selected weights on CHARM’s candidate densities.

SeedOxford-Man vs globalOxford-Man vs staticBinance vs globalBinance vs static
1−0.016204−0.007280+0.000454−0.000008
2−0.024900−0.007695−0.000335−0.000502
3−0.020205−0.008753−0.000426−0.000444
4−0.022895−0.009402+0.001526+0.000153
5−0.024201−0.009556+0.000889−0.000090
Mean−0.021681−0.008537+0.000421−0.000178
Sample SD0.0035480.0010150.0008260.000284

All five Oxford-Man seeds favored CHARM over both controls. Both adjusted date-block intervals lie below zero on Oxford-Man and include zero on Binance. The SDs describe retraining variability. The intervals condition on the fitted models and selected policies, with adjustment across five financial comparisons. We distinguish these results from Table 1’s cross-model ranking.

We expanded from one to three synthetic fits. In the first 20 steps after a strong change, mean Δ was −0.012679 against global (3/3 favored CHARM) and −0.000074 against tuned static (2/3). Across six relation-change conditions, full-stream mean Δ was +0.000009 against global and +0.000761 against tuned static, which had lower NLL in every fit. The appendix reports each primary contrast by seed.

Q3. Do the gains extend beyond financial volatility?

For next-hour Australian electricity demand, we trained one GRU jointly across five states per seed. The equal-state average favored CHARM over global (Δ = −0.030018, 5/5, adjusted interval [−0.039514, −0.020455]). The gain over tuned static was uncertain (Δ = −0.003271, 3/5, [−0.007460, +0.001655]). South Australia accounted for about 91% of the net gain over global. There, CHARM also improved on tuned static (Δ = −0.028561, 4/5, [−0.044949, −0.010532]).

For next-hour transformer oil temperature (OT), both controls had lower NLL in all five seeds on each of ETTh1 and ETTh2. The equal-series mean Δ was +0.001991 against global and +0.000956 against tuned static. The appendix gives split timestamps and hour counts, fit-period scaling, and selected configurations.

Theory and presentation

Lemma 1 characterizes the population log-loss optimum. Proposition 1 requires the correct changepoint law, exact candidate conditionals, and an uncapped filter. Proposition 2 bounds predictive KL by weight and component errors. Section 4 also gives two one-step bounds in total variation (TV). A single tail merge changes the forecast by at most the mass moved from beyond the cap. For fixed candidates, the change from reweighting is bounded by the TV between weight vectors times the largest pairwise TV between candidate densities. We assess full-stream NLL gains for the fitted, capped predictor empirically.

Section 3.3 gives the forecast, likelihood update after observation, and reset/growth steps. The appendix gives full pseudocode. We removed the opening quotation and now begin with the forecasting problem of choosing relevant history when change boundaries are unobserved.

Comments on the paper

1 comment
AuthorsOpenReview ↗

CHARM’s Contribution and Summary of Revisions

We thank all reviewers for their careful reading and constructive feedback, and the area chair for coordinating the review. The comments helped us clarify CHARM’s relationship to prior work and the scope of its empirical gains.

Contribution. CHARM builds on recurrent MOCA’s run-length inference and predictive marginalization. We train the recurrent predictor separately by scoring the next observation at each prefix of a training crop. With that predictor fixed, we can isolate two effects. Comparing CHARM with a tuned static mixture of the same candidate densities measures the value of online weighting. Comparing it with the most probable context measures the value of averaging over plausible histories.

Revisions and findings. We clarified the training objectives, added pseudocode, and compared CHARM with recurrent MOCA under matched architecture and training budgets. The two objectives yielded similar test NLL on Oxford-Man. With the predictor held fixed, however, online weighting lowered NLL relative to tuned static mixing in all five Oxford-Man seeds (mean reduction 0.00854). The corresponding Binance result was inconclusive.

We also extended the evaluation beyond finance. On Australian electricity demand, CHARM improved on continuous history, chiefly in South Australia. The five-state gain over tuned static mixing was uncertain, and both transformer-temperature series favored the simpler policies. The revision also reports synthetic replication, sensitivity to training length, and a held-out forecast-disagreement diagnostic alongside latency and memory measurements across run-length caps.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.