- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents an approach for LLM-based time series forecast correction in graph-structured systems. Given a base forecast at a particular node with a nominal coverage target, this method uses an LLM agent to determine what structural information (i.e. combination of dependencies among relevant nodes, together with their historical forecast error patterns) to use in the forecast correction for the node of interest. This correction is applied to both the forecast itself and upper/lower confidence bounds, with the extent of the correction controlled by a gate. The authors test their method on two graph-structured forecasting datasets, showing that point-prediction error decreases compared to the base forecast while maintaining similar empirical coverage levels.
Critical strengths
None
Critical weaknesses
The paper consistently under-specifies core components of the proposed correction method. Layers 1-3 each depend on parameters that are insufficiently defined, including (but not limited to) the PTE graph; the quantity that is needed to determine path relevance; the distribution and its dependence on an also undefined ; the origin of the binary masks , , and ; and many others. More practically, there is no provided format for the structural evidence that an agent observes or the ConditionSpec that it produces, and no sample LLM prompts to show how agents are instructed to interpret evidence. Experimentally, the ablation that removes evidence conditioning does not specify what information, if any, is passed to the agent in lieu of structural evidence. Beyond making the paper unnecessarily difficult to read, these omissions are significant gaps in the description and evaluation of the proposed algorithm.
Questions for authors
I strongly encourage the authors to provide definitions (not just descriptions) for all quantities used in the presentation of the method. That notwithstanding:
(1) What is the precise role of the LLM in this pipeline? What structural information is it provided (and in what format), what is it instructed to do, and what is it expected to output?
(2) MAE/RMSE differences in the ablation and against the strongest baselines are relatively modest. Given that the proposed method and baselines are stochastic, can the authors report means and standard deviations across multiple runs? This would help to better understand the extent of improvement.