acceptodds
Under review as a conference paper at ICLR 2027

From External Information to Forecast Adjustment: LLM-Based Constraint Modeling and Reflection Acceleration

Abstract

In complex real-world systems, forecasts based solely on historical data often fail under non-stationary dynamics induced by external information, ranging from stochastic events to rule-based operational constraints. We propose a novel post-forecast adjustment framework that refines existing forecasts by converting unstructured external knowledge into explicit mathematical constraints using large language models (LLMs). Our approach utilizes an LLM-based agent to extract and structure external information, which is then translated into mathematical constraints by a fine-tuned LLM. Given a base forecast from any statistical or machine learning model, the adjusted forecast is derived by solving a convex optimization problem that projects the base prediction onto the feasible region defined by these constraints. We employ a reflection and memory mechanism for iterative refinement, and introduce a novel constraint selection strategy based on mutual information and intervention effect analysis to accelerate the reflection process. Empirical results show that our framework significantly improves forecasting accuracy over base methods, including those that integrate external information via LLMs, while offering superior interpretability compared to direct LLM-based approaches.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
time series forecasting, large language models, post-hoc forecast adjustment, constrained optimization, llm agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_oD3TOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper converts external text into numerical constraints and uses soft-constrained optimization to adjust a base forecast. Reflection and memory refine this process, while CIMI selects constraint types to reduce reasoning cost. The evaluation compares forecasting performance and reasoning cost, includes a same-text LoRA baseline, and audits constraint extraction.

Critical strengths
  • Explicit constraints make it possible to inspect how external information changes a forecast. Soft penalties provide a clear way to balance that information against the base prediction.
  • Giving the LoRA baseline the same text is a useful control. The extraction audit also helps distinguish improvements in forecasting from faithful interpretation of the text, which is important for assessing the proposed mechanism (Section 4.2; Table 4).
Critical weaknesses
  • The repair procedure and convexity claim are not adequately justified. Large slack in a soft solution does not imply that the hard constraints are inconsistent, so the basis for IIS diagnosis is unclear. The probability constraint also challenges convexity for the full catalog, although results using selected constraint types may be unaffected (Sections 3.2–3.3).
  • The evidence for reflection and constraint selection is incomplete. Appendix A.2 excludes access to the current target outcome, but the RMSE-based examples do not explain how reflection is scored or stopped before that outcome is known. Comparing CIMI with all constraint types also leaves its ranking rule confounded with simply using fewer types (B.3.3; Tables 8–11).
Questions for authors
  1. Which hard constraint system is tested for infeasibility before IIS analysis? What happens when the soft solution has large slack even though the hard constraints are feasible?
  2. How is the probability constraint encoded or approximated in the all-constraint experiments, and does that formulation preserve convexity?
  3. What score is available to choose among reflection attempts and decide when to stop at test time, under the information restrictions in Appendix A.2?
  4. If both methods select the same number of constraint types, does CIMI outperform ranking by the intervention score alone?
Reviewer_kkdsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed an LLM-based post-forecast adjustment framework to refine existing time series forecasts. Several agents are employed to extract and structure external information, formulate convex optimization problems, and do reflection for iterative refinement. The proposed method is evaluated on four real world datasets with external information, and the results show improved accuracy over base forecasting methods.

Critical strengths
  1. The proposed framework enables automated forecast-adjustment process with designed agents. The adjustment process is decoupled with base forecasters, which allows the combination of the proposed method with different base methods.
  2. Forecast adjustments are obtained from optimization problems rather than directly generated by LLMs. By transforming external information into optimization constraints, the proposed method achieves explainable and interpretable adjustments.
Critical weaknesses
  1. The CIMI objective seems not related to mutual information, which describes the dependency of two random variables. The effects of Intervention and DMI terms are not separately tested. The selection of hyperparameters is not explained, and no sensitivity analysis is given.
  2. The main experiments only compare the proposed method with base forecasters. The proposed method is not compared to existing baselines [1], while the datasets are obtained from [1].

[1] Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection. Advances in Neural Information Processing Systems, 37:58118–58153, 2024.

Questions for authors
  1. For the Intervention term in CIMI objective, redundant constraints may affect the evaluation of a single constraint type, as removing one of them will not change validation result. Please clarify.
  2. For the DMI term, why is the true trajectory compared with historical observations? I think it may overestimate the trend captured by base forecasting models.
  3. In Section 3.2, chance constraints are approximated with discrete linear inequalities. In this case, is the optimization problem still convex?
  4. In the iterative optimization, there can be large constraint violations when the problem is feasible, as the base forecast may be far from the constraint requirements. What will Stage 2 do if there are no incompatible constraints?
  5. For the evaluation agent, how can it distinguish errors introduced by the reasoning agent and the iterative repair in stage 2 of iterative optimization?
  6. The comparison with existing forecasting methods with external information is missing. Also, can the authors compare against an LLM with the same backbone that directly adjusts the base forecast?
  7. In Section 4.2, the TimeLLM baseline is not stated. In Table 2, why the results are compared with Base?
Reviewer_8bsfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework that refines LLM forecasts by incorporating mathematical constraints such as range and probabilistic limits, which are derived from external knowledge. A novel important score named CIMI is proposed to weigh how important the constraint should be.

Critical strengths
  1. The idea is well motivated.
Critical weaknesses
  1. The fontsize in figures is too small to be readable, especially in Fig. 1 and Table 1.
  2. The presentation of Table 2 is bad, too.
Questions for authors

Questions

  1. What is Human Experience in Fig. 2?
  2. Is there information leak in Fig. 2 when using Chronos-Direct, i.e., does the Reasoning Agent already had the knowledge? Is the performance improvement of Adjustment-Chronos from the additional information or from the mathematical constraint incorporation?
  3. Is the propose CIMI in line 194 heuristic? Does it guarantee certain properties of the improvement?
  4. Does the proposed framework consider restricting how much historical information should or should not impact the forecasts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.