- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors present a detailed analysis for leveraging LLMs for time series forecasting, including the way of aligning LLMs with TS data, the scenarios where LLMs can be better leveraged for TS data, and the reason why LLMs can be helpful for TSF. They also provide the guidelines to apply LLMs to TSF.
Critical strengths
- The writing is quite clear.
- The experiments are comprehensive.
- The investigated problem is interesting.
Critical weaknesses
- I am challenging the motivation of this work. The authors raise a question 'are LLMs tryly indispensable for TSF'. However, they only investigate two kinds of alignment methods to leverage LLMs for TSF. It is noted that there are still other ways to leverage LLMs like Prompt4Cast [1]. I think the authors should narrow down their claims.
[1] PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting
-
I agree the point whether LLMs are necessary for TS data. However, why the authors just compare two kinds of LLM alignment methods? I think it is more reasonable to include non-LLM methods, particularly those TS foundation models. In the current experiments, it is more like the authors aim to investigate which alignment method is better, instead of whether using LLMs is necessary.
-
I am wondering whether the input prompts have large influence on the performance and whether the authors have simple experiments to discuss this effect?
-
The authors used GPT-2 as the LLM backbone. I understand this is for efficient experiments. However, GPT-2 is too small to fully exploit the power of LLMs, and I also doubt whether using GPT-2 is useful to derive the correct conclusion.
-
As an investigation work, I think only numerical comparisons are insufficient to provide an effective solution. The intermediate results can be helpful.
-
Another concern is regarding the parameters. From two aspects, one is on the LLM parameters, and another is on the parameters of the non-LLM part. The first issue is related to my point 4, and I am wondering whether the conclusion still holds with different sizes of LLMs, or, whether different sizes of LLMs have different influences? Regarding the second issue, given you have two alignment methods which might bring different trainable parameters for the non-LLM part, I am wondering whether the authors control this? in my understanding, the reason why the pre-alignment has better performance is probably because this method has additional module as Figure 1, which might affect the performance.
Additionally, I am open to raise my score if my concerns could be addressed
Questions for authors
See above and I also have another minor question: Why did the authors choose to investigate the two kinds of LLM alignment methods instead of comparing with existing LLM driven baselines and non-LLM baselines?