- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors proposed a hierarchical LLM semantic guidance for MTS forecasting, leveraging LLMs to assist the local feature learning and topology learning. The similar challenges have been addressed by an existing work with similar techniques, so I recommend the authors to provide detailed justifications on how this work is different from the existing work and what is the gap the authors are trying to solve.
Critical strengths
The writing and the presentation are quite clear for readers to understand, and the experiments are comprehensive.
Critical weaknesses
-
The authors aim to guide the feature and topology learning with LLMs, and they designed two modules for variable-level learning and topology-level learning after using prompts to extract features from LLMs. However, the similar challenge has been addressed by the existing work [1], and this work also leveraged the two modules (node and edge levels) after extracting prompt features from LLMs. Thus, I think the two works adopted the similar methodology to address the similar challenge. I think the authors should justify the difference and what are the challenges the authors are trying to solve.
-
How the authors ensure the prompts are accurate to guide the feature and topology learning? LLMs always have hallucinations. Meanwhile, I think the prompt designs should have large influences on the guidance and the final performance, so I am wondering whether the authors have detailed analysis on these prompt designs? Additionally, I noticed the authors used GPT-2 to extract features. However, this one is too small to be called as a LLM, so do you have analysis by using different LLMs like the advanced open-sourced LLMs like QWEN or Deepseek?
-
I appreciated the authors to have efficient analysis. However, my concern is that the authors assumed the cached embeddings. When you faced the new situations, you would not have the cached embedding, which means you have to process the embeddings by using the LLM. Thus, I think it is reasonable to conduct the efficiency analysis based on the raw input instead of the cached embeddings. Meanwhile, I think this problem also challenges the first motivation, i.e., 'introduce substantial memory and inference overhead'. In this method, I dont think this motivation has been addressed, because you assume the cached embeddings instead of designing a mechanism to solve it. This is not fair because in real inference, you still need a LLM to extract the embeddings.
[1] Wang, Y., Jin, R., Wu, M., Li, X., Xie, L., & Chen, Z. (2024). K-link: Knowledge-link graph from llms for enhanced representation learning in multivariate time-series data. arXiv e-prints, arXiv-2403.
Questions for authors
See above