- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces EventAlign, an evaluation framework for studying how the assignment of sparse textual records affects time series forecasting. Using financial data and earnings call transcripts, the authors compare different text exposure rules, input encodings, and background information. The results show that exposure rules can change both predictive performance and the relative advantages of different models.
Critical strengths
Clear and meaningful research question. The paper investigates an important but often overlooked issue in multimodal time series forecasting: how sparse textual information should be aligned with prediction samples.
Careful experimental design. The authors conduct controlled comparisons and provide extensive analyses of different exposure rules.
Critical weaknesses
Insufficient baselines. Although several numerical forecasting models are evaluated, the text representation experiments mainly focus on MiniLM, FinBERT and TF-IDF. There are many alternative approaches to multimodal forecasting. Since the paper proposes a new evaluation protocol, comparisons with more representative multimodal methods are important to establish its relevance.
Limited evaluation scope. The main experiments focus on a relatively small set of companies in one financial market. Evaluation across different financial markets and datasets would strengthen the generality of the conclusions.
Limited methodological novelty. The main contribution is an evaluation protocol rather than a new modeling technique. The individual components are relatively straightforward, making the methodological contribution somewhat incremental.
Fragmented experimental presentation. The paper includes many experiments with different evaluation protocols. While comprehensive, this increases the difficulty of understanding the main findings and makes the overall narrative less focused.
Questions for authors
Dependence on the prediction window. The main statistically significant learner-by-exposure interaction appears to depend strongly on the initial overnight return window. For subsequent future-return windows, the interaction becomes much smaller and statistically uncertain. Could the authors explain why this happens and clarify how the main findings generalize to genuine future forecasting?