- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses multimodal time series forecasting. Numerical forecasters learn endogenous patterns from history but cannot see the external events described in text, while existing multimodal methods either compress text into latent features, underusing event semantics, or let an LLM generate numbers directly, which suffers from the language–number gap. The authors propose AFTER, which keeps a trained numerical backbone and lets an LLM estimate only an offset to the backbone forecast, without training any new parameters. AFTER works in three steps. History-Aware Event Reasoning (HAER) reads the history and the texts, decides whether any event still affects the future (none / weak / active), and writes an event summary. Complementary Offset Reasoning (COR) then estimates offsets along three routes: LLM reasoning from the current event state, LLM reasoning by analogy with similar cases retrieved from a training-set memory, and direct transfer of the retrieved cases’ continuations. Robust Offset Integration (ROI) takes the element-wise median of the three offsets and adds it to the backbone forecast.
Critical strengths
The most valuable aspect is the division of labor: the numerical model forecasts and the LLM only estimates a correction. The backbone does not need retraining, so AFTER can be attached to any trained forecaster; the authors observe gains on all seven backbones, and MSE decreases in 95 of the 108 main settings. When the text carries no useful information, AFTER leaves the forecast unchanged: on Environment, 94% of the samples are labeled none and the predictions barely move, so it does not degrade a good forecast. Table 3(b) shows that letting the LLM generate the whole trajectory greatly increases the error (Economy MSE rises from 0.193 to 0.369), which supports the claim that anchoring on the numerical forecast is more reliable than asking the LLM for numbers directly. This matters because it offers a training-free, traceable, and low-risk way to connect an LLM’s understanding of events to existing forecasting systems.
Critical weaknesses
-
All test periods end before mid-2024 (e.g., roughly 2009–2024 for Social Good; Health and Traffic both cover the 2020 pandemic), very likely before the knowledge cutoff of DeepSeek-V4-Flash. The HAER and COR prompts contain the absolute input and forecast time ranges and the target variable (e.g., the U.S. unemployment rate), and the texts contain specific figures such as “unemployment increased to 9.5%”. The LLM could therefore recognize the episode and write down what happened next. The de-identification check covers only Economy and Social Good with iTransformer and keeps the specific numbers in the texts, so it does not rule this out. The No Text condition in Table 4 falls back to the backbone by design, so it cannot separate event information in the text from the LLM’s own memory. Since the central claim is that the gains come from reasoning about events, this concern directly determines whether the conclusion holds.
-
The average gains over the backbones are only 3.34% (MSE) and 2.03% (MAE); MSE drops by only about 0.2%–1.9% on Agriculture, Climate, and Energy, and Environment is slightly worse. The paper does not state the number of seeds, reports no standard deviations or significance tests, and does not describe the randomness of LLM outputs, while the monthly test segments are short (about 60 months for Security) and the test windows overlap heavily. Table 1(a) in the main text shows only three “representative” datasets, yet Tables 17–19 show that MM-TSFLib beats AFTER on Climate, Health, and Environment under all three backbones, and Table 2 omits MM-TSFLib and TaTS. Each sample also needs three LLM calls (about 7.7 s versus 1.8 ms for the backbone), and the cost of building the memory on the training set is not reported. The related work also misses directly relevant studies, such as From News to Forecast (NeurIPS 2024), which uses LLMs to analyze news events for forecasting, retrieval-augmented forecasting (e.g., RAF, TimeRAG), and the classic forecasting literature on judgmental adjustment of statistical forecasts, which may lead readers to think this direction has not been explored before.
Questions for authors
Dear authors, the answers to the following questions could directly affect my assessment.
- Could you evaluate on data after the LLM’s knowledge cutoff, for example by collecting 2025–2026 data from the same sources, or use an LLM whose cutoff precedes the test period? A cheaper control is to keep the dates, history, and dataset description but replace the texts with texts from other periods; if the gains remain, they come from the LLM’s memory rather than from the texts.
- How many seeds were run? Are LLM outputs sampled, and at what temperature? Could you report mean ± standard deviation and a paired significance test, and show how much of the gain comes from a few extreme periods such as the 2020 pandemic, and what the results are without them?
- How was the context_guidance written for each dataset, and was it designed using only the training and validation sets? The problem is formulated for C variables; do the experiments forecast only the OT variable?
3 comments
A Kind Follow-up on Our Rebuttal
Dear Reviewer dUhf,
We sincerely appreciate your thoughtful and constructive comments, which have helped us improve our paper.
It has been five days since we submitted our detailed responses, additional experiments, and revised manuscript. As the discussion period is approaching its conclusion, could you kindly review our responses and let us know whether your concerns have been addressed?
We greatly value open and constructive scientific discussion and would welcome the opportunity to address any remaining questions or comments before the discussion period ends.
We fully understand that you may be very busy. Thank you again for your valuable time and effort in reviewing our work.
Best regards,
The Authors
Request for Your Feedback on Our Responses
Dear Reviewer dUhf,
We have carefully addressed every concern raised in your Critical Weaknesses and Questions:
- Historical-knowledge leakage: Following your suggestion, we retained the dates and numerical history while replacing only the event texts with texts from other periods. We also expanded the de-identification experiment.
- Statistical reliability: We reported five-seed mean ± standard deviation, paired significance tests, and the LLM decoding settings, and separately analyzed pandemic and non-pandemic periods.
- Framework comparison: We conducted controlled comparisons with MM-TSflib after removing its additional numerical prior and also compared the frameworks when the same type of prior is available.
- Inference and Memory costs: We reported online token and monetary costs together with offline Memory-construction costs.
- Context guidance and multivariate forecasting: We clarified that
context_guidancecontains no sample-specific information. The main experiments forecast only theOTvariable because they follow the established Time-MMD protocol adopted by prior methods. We further conducted an additional experiment demonstrating AFTER's applicability to multivariate forecasting. - Related work: We added From News to Forecast, RAF, TimeRAG, and judgmental forecast-adjustment studies.
The corresponding revisions have been added to Appendices D, L, M.1, and N, together with the expanded Related Work. As less than one day remains before the discussion period ends, we would greatly appreciate it if you could review our responses and let us know whether they have addressed your concerns. We remain available to promptly answer any further questions.
Best regards,
The Authors
Looking Forward to Your Further Feedback
Dear Reviewer dUhf,
We sincerely thank you for your time and for the thoughtful and constructive comments on our paper. We have carefully addressed each of your questions and concerns, conducted additional controlled experiments, and revised the manuscript accordingly.
As the discussion period approaches its conclusion, we hope that our responses, new experimental evidence, and manuscript revisions have adequately addressed your concerns. If any questions remain, please do not hesitate to let us know. We would be pleased to provide further clarification promptly and in detail.
Best regards,
The Authors