acceptodds
Withdrawn from ICLR 2027

Reasoning Selectively over Forecast Errors with Evidence-Conditioned Probabilistic Correction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A probabilistic forecast can be well calibrated on average yet remain unreliable under a particular structural state. We propose evidence-guided selective probabilistic reasoning, a framework that separates the choice of a residual condition from the estimation and deployment of its numerical correction. A numerical forecaster provides a predictive center and asymmetric interval. A selective multi-agent workflow examines bounded structural evidence and produces a residual ConditionSpec, which deterministic estimators use to retrieve mature residuals and compute center and tail corrections. An outcome-grounded Gate then selects how much of the candidate to deploy using comparable historical candidates frozen before their outcomes were known. Delayed outcomes populate separate views for residual estimation, workflow allocation, and deployment. Experiments on XuanCheng and PEMS-BAY show dataset-dependent effects of evidence conditioning and an explicit accuracy-computation trade-off. Selective reasoning reduces live LLM calls by 38.7% and 60.2% relative to Always FULL, respectively, with higher MAE than Always FULL but lower point-prediction errors than the Base. On PEMS-BAY, suppressed candidates have negative mean interval-score gain, whereas fully deployed candidates have positive mean gain. These results connect structural reasoning to measurable candidate quality and deployment behavior, providing a traceable way to control both reasoning effort and forecast correction.

Primary area:
learning on time series and dynamical systems
Keywords:
selective probabilistic reasoning, evidence-conditioned correction, uncertainty quantification, multi-agent reasoning, outcome-grounded learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QiqxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents an approach for LLM-based time series forecast correction in graph-structured systems. Given a base forecast at a particular node with a nominal coverage target, this method uses an LLM agent to determine what structural information (i.e. combination of dependencies among relevant nodes, together with their historical forecast error patterns) to use in the forecast correction for the node of interest. This correction is applied to both the forecast itself and upper/lower confidence bounds, with the extent of the correction controlled by a gate. The authors test their method on two graph-structured forecasting datasets, showing that point-prediction error decreases compared to the base forecast while maintaining similar empirical coverage levels.

Critical strengths

None

Critical weaknesses

The paper consistently under-specifies core components of the proposed correction method. Layers 1-3 each depend on parameters that are insufficiently defined, including (but not limited to) the PTE graph; the quantity PTE~ℓ\tilde{\textup{PTE}}_\ell that is needed to determine path relevance; the distribution UtW\mathcal{U}_t^W and its dependence on an also undefined WW; the origin of the binary masks mtμm_t^\mu, mt−m_t^-, and mt+m_t^+; and many others. More practically, there is no provided format for the structural evidence that an agent observes or the ConditionSpec that it produces, and no sample LLM prompts to show how agents are instructed to interpret evidence. Experimentally, the ablation that removes evidence conditioning does not specify what information, if any, is passed to the agent in lieu of structural evidence. Beyond making the paper unnecessarily difficult to read, these omissions are significant gaps in the description and evaluation of the proposed algorithm.

Questions for authors

I strongly encourage the authors to provide definitions (not just descriptions) for all quantities used in the presentation of the method. That notwithstanding:

(1) What is the precise role of the LLM in this pipeline? What structural information is it provided (and in what format), what is it instructed to do, and what is it expected to output?

(2) MAE/RMSE differences in the ablation and against the strongest baselines are relatively modest. Given that the proposed method and baselines are stochastic, can the authors report means and standard deviations across multiple runs? This would help to better understand the extent of improvement.

Reviewer_ihNHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes an approach to calibrate numerical forecasts using LLMs. It uses a multi-agent workflow which examines the evidence to determine the condition based evidence. Another LLM performs a gating of how much of the candidate forecast to deploy. They evaluate on 2 traffic datasets - XuanCheng and PEMS-BAY and show a 4-5% improvement in performance.

Critical strengths

The problem formulation of using numerical forecasts for initial forecasts and using LLMs to identify whether to change it and how much to change is quite interesting. The division of the framework to base forecast, conditional correction and outcome-grounded deployment is structurally good.

Critical weaknesses

The writing is extremely hard to understand and is full of jargon. The approach and notations are very hard to follow. The framework figure is also very convoluted. Further, the experimental setup is very weak - only 2 datasets are used which are both traffic datasets. GPT-4o is used for correction but the datasets pre-date the training cutoff of the GPT-4o so there are data leakage concerns as well.

Questions for authors

Can you please explain the notations and the approach in a digestible format?

Reviewer_MJfEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an evidence-guided selective probabilistic correction framework for time-series forecasting. The proposed framework uses structural evidence and a selective multi-agent LLM workflow to identify relevant historical residual conditions. A reasoning controller determines the appropriate workflow depth, deterministic statistical estimators are used to compute corrections to the forecast center and prediction intervals. A deployment gate determines how strongly these corrections should be applied, using historical candidate outcomes. The framework also incorporates delayed feedback to update residual estimates, reasoning allocation, and deployment decisions.

The authors evaluate their approach on two traffic forecasting datasets, XuanCheng and PEMS-BAY, using comparisons against numerical, LLM-based, and probabilistic forecasting baselines. Results demonstrate improvements in point forecasting accuracy over the ridge-based base model, although the proposed method does not consistently outperform stronger forecasting baselines. Evidence conditioning shows dataset-dependent benefits and selective reasoning reduces live LLM calls at the cost of some predictive accuracy.

Critical strengths
  1. Thoughtful methodological design. The paper presents a structured framework that separates reasoning allocation, evidence-conditioned residual estimation, and correction deployment. This separation is a meaningful design choice, as it reduces reliance on unconstrained LLM-generated numerical predictions and improves interpretability and traceability.

  2. Careful treatment of temporal information and outcome-based feedback. A notable strength is the explicit enforcement of forecast-time information boundaries. This is particularly important for an adaptive forecasting framework, where information leakage could otherwise lead to overly optimistic evaluations.

  3. Empirical Evidence. The proposed framework improves point forecasting accuracy over its Ridge-based Base model on both datasets. The experiments also demonstrate that selective reasoning reduces live LLM calls at the cost of some predictive accuracy. The component ablations and deployment analyses further help characterize these trade-offs, providing useful empirical evidence about the framework's behavior.

Critical weaknesses

W1. Limited comparison with alternative correction and reasoning approaches. While the paper evaluates several forecasting baselines and provides component ablations (Table 2), it does not sufficiently isolate the contribution of multi-agent reasoning. The paper would benefit from comparisons with simpler reasoning-free or deterministic condition-selection approaches, as well as a single-agent LLM baseline, to better justify the added complexity of the proposed framework.

W2. Limited evaluation of conditional probabilistic reliability. While the paper motivates evidence-conditioned correction through forecasting unreliability under specific structural conditions, the evaluation primarily reports aggregate probabilistic metrics. The paper would benefit from a more targeted analysis of conditional coverage and interval scores across regimes defined by structural dependency strength or congestion conditions, comparing the proposed approach against base and the variant without evidence conditioning. This would help establish whether the proposed corrections improve probabilistic reliability under the conditions they specifically target.

W3. Limited evaluation across forecasting backbones. The proposed framework is evaluated using a Ridge-based Base forecaster, but its effectiveness may depend on the underlying forecasting model. More expressive models, such as GNNs, may already capture some of the structural dependencies targeted by the proposed corrections. An evaluation with alternative forecasting backbones, or a discussion of this sensitivity, would help establish the generalizability of the framework.

Questions for authors
  1. Evidence-conditioning ablation: Could the authors clarify precisely what is removed in the “w/o Evidence Cond.” variant in Table 2?

  2. Traffic-regime definitions: How are the free-flow and congestion regimes in Table 3 defined, and are these classifications based exclusively on forecast-time information?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.