- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This article explores what RL post-training and test-time compute can do for probabilistic forecasting, where scores apply to distributions rather than single sampled trajectories. It divides excess CRPS into Monte-Carlo, optimization, and information gaps, derives a pairwise reward for policy gradients, and shows why standard GRPO drives forecasts toward under-dispersion. Experiments on Chronos and Moirai show that the correction largely restores spread and coverage. The corrected RL model performs similarly to the SFT control, with no consistent significant gain on short-term GIFT-Eval configurations. The self-backtest has little ability to predict future performance, and the interventions based on it hurt performance on average.
Critical strengths
The experiments test the mechanism from multiple perspectives. Reward and estimator ablations, SFT comparisons, and differences in corpus, horizon, and model size make the bias diagnosis more convincing. Synthetic examples with known conditional laws help distinguish distributional effects from benchmark scores.
Critical weaknesses
(1) The correction removes a GRPO-specific failure, but the motivation of this paper and the reason to use GRPO remains weak. Table 5 supports the bias diagnosis: corrected runs bring GMD and coverage close to their zero-shot values. This largely reverses the extra under-dispersion caused by standard GRPO; it does not show better calibration than the original model. In pretrain-corpus validation, zero-shot cov80 is about 0.63, and corrected runs are about 0.63-0.66. According to the paper's quantile convention, the calibrated value at N = 20 is about 0.73 (Section 4; Tables 3 and 5). If GRPO is used, the correction matters, but this itself does not explain why GRPO should be used. I do not see a consistent short-term forecasting advantage over simpler choices. Across 55 GIFT-Eval configurations, the WQL ratio of coefficient-corrected GRPO ranges from about 0.987 to 0.991 across seeds, without consistent statistical significance; the batch-normalized version is about 0.993-1.002 (Table 5). In two reported 2,000-step runs on the same pretraining corpus, SFT reaches 0.982 and 0.988, respectively (Table 6), while no post-training avoids rollout cost altogether. Longer corrected training improves in-corpus CRPS but worsens short-term GIFT-Eval WQL (Table 7); the larger Chronos models also show no clear short-term transfer. These results limit the practical case for GRPO, rather than the validity of the bias analysis. There is a real counterpoint: on validation windows, corrected RL has a KL divergence of about 0.012-0.013 nats per token from the zero-shot policy, while SFT control has a KL divergence of 0.025 at the same training-step budget and with roughly comparable in-corpus gain (Section 5.4). I believe that bias diagnosis and correction are meaningful contributions even without large benchmark gains. However, the paper does not show that lower KL leads to better held-out prediction or capability retention, and matched step and window counts do not imply matched rollout tokens, FLOPs, or GPU time. Without a compute-matched comparison, I cannot judge when additional RL optimization is worthwhile. I would not generalize short-term transfer results to all horizons: Appendix D.5 reports qualified Moirai gains on medium- and long-horizon configurations. The motivation for test-time compute is similarly unresolved. The paper evaluates several self-backtest-driven interventions, including conformal rescaling, bias correction, strategy selection, and softmax mixing over candidate strategies. These interventions hurt performance on average, and the reported verifier fidelity is low (ρ ≈ 0.1). This is a useful negative result for the specific backtest signals and intervention strategies studied here, but it does not provide positive evidence that additional test-time computation can improve forecasting. More importantly, taken together with the lack of consistent short-term transfer from corrected GRPO, the results do not yet show a practical regime in which either post-training with corrected GRPO or test-time computation produces a clear forecasting benefit.
(2) The results and conclusions need to be stated more rigorously. For example, the group-std claim goes beyond the analytic result. Corollary 1 derives the group-mean baseline bias for pairwise CRPS rewards. The per-group standard-deviation effect has a different evidential basis: Appendix A says this term lacks a closed-form result, and the persistent shrinkage is supported by Gaussian fixed points at . I find this convincing for the studied group sizes, but it does not prove that the bias remains bounded away from zero when . In particular, alone gives a bounded asymptotic upper order, not a nonzero lower bound. Table 4 also shows that correcting the coefficient while retaining group-std normalization leaves the toy policy under-dispersed at . The proved group-mean effect and numerically observed group-std effect should be more clearly distinguished. In addition, test-time and sampling results require narrower attribution. The self-backtest results are well tested within their scope: Section 5.5 and Appendix D.6 include 15 GIFT-Eval configurations, five synthetic families, up to four backtest tails, and a learned ridge verifier. On average, none of the tested interventions helps. I would not extend this result to test-time verifiers with information the evaluated backtests do not have. Additionally, exact sampling masks PAD/EOS, whereas the default top-k sampler does not (Appendix B). Before attributing the full performance gap to top-k sharpening, a matched-masking comparison is needed.
Questions for authors
Please see weaknesses.
2 comments
Thank you for the careful reading and constructive critiques—your questions substantially improved the rigour and clarity of the paper.
Thank you for the detailed response. I will maintain my positive score.