acceptodds
Under review as a conference paper at ICLR 2027

Forecasting as a Verifiable Task: What RL and Test-Time Compute Can (and Cannot) Buy

Abstract

Probabilistic forecasting is verifiable: once the future arrives, a proper scoring rule grades every sampled trajectory. Time-series foundation models are therefore natural targets for the reinforcement-learning (RL) post-training and test-time-compute recipes that transformed language models. We ask what these recipes buy when the verifier grades a distribution rather than a single answer. We decompose the excess CRPS of a sampled forecaster into Monte-Carlo, optimisation and information gaps, derive the per-sample reward under which policy gradient is unbiased for CRPS (it is necessarily pairwise), and analyse GRPO under this reward. GRPO's group-relative advantages are provably biased for pairwise rewards: the group-mean baseline lowers the effective dispersion coefficient from 1/2 to (G-2)/(2(G-1)), and per-group standard-deviation scaling adds an O(1) shrinkage that no group size removes. The bias predicts the under-dispersion of every uncorrected run across two model families and three model sizes (coverage 0.63 to 0.55 for Chronos-T5-small); a two-line correction removes it. Corrected RL matches a supervised fine-tuning control at equal budget, with no significant GIFT-Eval gain, while preserving calibration at half the KL divergence. Test-time compute fails for a different reason: its only verifier, a self-backtest, has fidelity rho ≈ 0.1, so every backtest-driven intervention hurts. The bias and its correction apply to any pairwise or set-level reward. Code is available at https://anonymous.4open.science/r/verifiable-forecasting-72ED.

Primary area:
learning on time series and dynamical systems
Keywords:
probabilistic time-series forecasting, time-series foundation models, reinforcement learning with verifiable rewards, GRPO, policy gradient, proper scoring rules, CRPS, calibration, test-time compute, verifiers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kftHOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This article explores what RL post-training and test-time compute can do for probabilistic forecasting, where scores apply to distributions rather than single sampled trajectories. It divides excess CRPS into Monte-Carlo, optimization, and information gaps, derives a pairwise reward for policy gradients, and shows why standard GRPO drives forecasts toward under-dispersion. Experiments on Chronos and Moirai show that the correction largely restores spread and coverage. The corrected RL model performs similarly to the SFT control, with no consistent significant gain on short-term GIFT-Eval configurations. The self-backtest has little ability to predict future performance, and the interventions based on it hurt performance on average.

Critical strengths

The experiments test the mechanism from multiple perspectives. Reward and estimator ablations, SFT comparisons, and differences in corpus, horizon, and model size make the bias diagnosis more convincing. Synthetic examples with known conditional laws help distinguish distributional effects from benchmark scores.

Critical weaknesses

(1) The correction removes a GRPO-specific failure, but the motivation of this paper and the reason to use GRPO remains weak. Table 5 supports the bias diagnosis: corrected runs bring GMD and coverage close to their zero-shot values. This largely reverses the extra under-dispersion caused by standard GRPO; it does not show better calibration than the original model. In pretrain-corpus validation, zero-shot cov80 is about 0.63, and corrected runs are about 0.63-0.66. According to the paper's quantile convention, the calibrated value at N = 20 is about 0.73 (Section 4; Tables 3 and 5). If GRPO is used, the correction matters, but this itself does not explain why GRPO should be used. I do not see a consistent short-term forecasting advantage over simpler choices. Across 55 GIFT-Eval configurations, the WQL ratio of coefficient-corrected GRPO ranges from about 0.987 to 0.991 across seeds, without consistent statistical significance; the batch-normalized version is about 0.993-1.002 (Table 5). In two reported 2,000-step runs on the same pretraining corpus, SFT reaches 0.982 and 0.988, respectively (Table 6), while no post-training avoids rollout cost altogether. Longer corrected training improves in-corpus CRPS but worsens short-term GIFT-Eval WQL (Table 7); the larger Chronos models also show no clear short-term transfer. These results limit the practical case for GRPO, rather than the validity of the bias analysis. There is a real counterpoint: on validation windows, corrected RL has a KL divergence of about 0.012-0.013 nats per token from the zero-shot policy, while SFT control has a KL divergence of 0.025 at the same training-step budget and with roughly comparable in-corpus gain (Section 5.4). I believe that bias diagnosis and correction are meaningful contributions even without large benchmark gains. However, the paper does not show that lower KL leads to better held-out prediction or capability retention, and matched step and window counts do not imply matched rollout tokens, FLOPs, or GPU time. Without a compute-matched comparison, I cannot judge when additional RL optimization is worthwhile. I would not generalize short-term transfer results to all horizons: Appendix D.5 reports qualified Moirai gains on medium- and long-horizon configurations. The motivation for test-time compute is similarly unresolved. The paper evaluates several self-backtest-driven interventions, including conformal rescaling, bias correction, strategy selection, and softmax mixing over candidate strategies. These interventions hurt performance on average, and the reported verifier fidelity is low (ρ ≈ 0.1). This is a useful negative result for the specific backtest signals and intervention strategies studied here, but it does not provide positive evidence that additional test-time computation can improve forecasting. More importantly, taken together with the lack of consistent short-term transfer from corrected GRPO, the results do not yet show a practical regime in which either post-training with corrected GRPO or test-time computation produces a clear forecasting benefit.

(2) The results and conclusions need to be stated more rigorously. For example, the group-std claim goes beyond the analytic result. Corollary 1 derives the group-mean baseline bias for pairwise CRPS rewards. The per-group standard-deviation effect has a different evidential basis: Appendix A says this term lacks a closed-form result, and the persistent shrinkage is supported by Gaussian fixed points at G=8,32,128G=8,32,128. I find this convincing for the studied group sizes, but it does not prove that the bias remains bounded away from zero when G→∞G\to\infty. In particular, O(1)O(1) alone gives a bounded asymptotic upper order, not a nonzero lower bound. Table 4 also shows that correcting the coefficient while retaining group-std normalization leaves the toy policy under-dispersed at G=8G=8. The proved group-mean effect and numerically observed group-std effect should be more clearly distinguished. In addition, test-time and sampling results require narrower attribution. The self-backtest results are well tested within their scope: Section 5.5 and Appendix D.6 include 15 GIFT-Eval configurations, five synthetic families, up to four backtest tails, and a learned ridge verifier. On average, none of the tested interventions helps. I would not extend this result to test-time verifiers with information the evaluated backtests do not have. Additionally, exact sampling masks PAD/EOS, whereas the default top-k sampler does not (Appendix B). Before attributing the full performance gap to top-k sharpening, a matched-masking comparison is needed.

Questions for authors

Please see weaknesses.

2 comments

Thank you for the detailed response. I will maintain my positive score.

Thank you for the careful reading and constructive critiques—your questions substantially improved the rigour and clarity of the paper.

Reviewer_Xd95OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses CRPS as the reward for RL post training of Chronos. Chronos generates its forecast token by token. The authors show that the reward of each sample must include its distances to the other samples. They prove that the group mean and the group standard deviation in GRPO make the forecasts too narrow. The fix changes the weight of this term and uses the standard deviation of the whole batch. After the fix, RL works about as well as SFT with the same budget. The paper also studies drawing more samples and choosing a sampling strategy at test time.

Critical strengths
  1. The comparison with SFT is valuable. Once the estimator is fixed, RL works about as well as SFT with the same budget. The control is strict, so this result is more convincing than work that only compares with the base model.
  2. The diagnosis of the bias is well supported. The predicted bias matches both the toy study and real training. Every uncorrected run makes the forecasts narrower and lowers coverage. After the fix, this effect is gone. The fix only changes two lines.
  3. The test-time analysis gives useful lessons. It shows that the default truncated sampling of Chronos hurts when many samples are drawn. Turning it off gives a clear gain. It also shows that choosing a sampling strategy by backtesting on the history barely works.
Critical weaknesses
  1. The theoretical contribution does not seem very significant to me. The key step is to pick one coefficient so that GRPO follows the CRPS gradient with finite samples. A baseline without the current sample is a known idea, for example in VIMCO and PKPO. It would help to discuss how the result relates to these works.
  2. It is not fully clear to me what corrected RL offers over SFT. The paper points to better calibration and a lower KL. But SFT reaches similar accuracy, and the practical value of these two points is not shown. It would help to explain this more directly.
Questions for authors
  1. How does Corollary 1 relate to the leave one out baselines in VIMCO and PKPO? What is new beyond them?
  2. In what cases would one prefer corrected RL over SFT? Could you show one such case, for example by testing whether the lower KL leads to less forgetting?
Reviewer_c1m7OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether reinforcement learning with verifiable rewards and test-time compute can improve probabilistic time-series forecasting.

It derives the correct pairwise reward for optimizing CRP. It identifies a bias introduced by GRPO normalization and proposes a correction.

Experiments with Chronos models show that corrected RL improves calibration relative to uncorrected RL, but provides little additional benefit over supervised fine-tuning.

Critical strengths

The paper asks a clear question and gives a technically grounded answer, including negative results.

The analysis of GRPO bias is useful.

The theoretical predictions are supported by controlled experiments across several model sizes and two model families.

The paper also shows that default top-k sampling can hurt forecasting quality.

The comparison with supervised fine-tuning is valuable. It helps seperate benefits of the training objective from simply learning on more data.

Critical weaknesses

The conclusions about test-time compute seem somewhat broader than the experiments support. There is overselling.

The study mainly considers self-backtesting, which is a weak verifier. Other verification strategies may behave differently.

The empirical conclusions about RL are primarily based on CRPS and a limited set of forecasting model families. It is not clear how broadly they generalize to other probabilistic forecasting objectives.

Questions for authors

Could other sources of verification (e..g, related time series or auxiliary information) provide sufficiently accurate feedback for test-time selection?

Which parts of the GRPO correction extend to general set-level rewards beyond pairwise CRPS?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.