acceptodds
Under review as a conference paper at ICLR 2027

EventAlign: Auditing Input Exposure in Sparse-Record Forecasting

Abstract

Financial forecasts can be made daily, whereas earnings-call transcripts arrive only occasionally and may be reused for weeks. However, evaluating models under a single reuse rule makes it difficult to separate how well a learner uses text from how that text is assigned across prediction samples. Our key observation is that transcript assignment changes the information supplied to the predictor, making it part of the evaluation rather than a neutral preprocessing step. To isolate this effect, we introduce EventAlign, which varies transcript assignment while keeping prediction samples, labels, numerical histories, and training and model-selection procedures fixed within each exposure comparison. On our financial case study, whose primary close-to-close target includes the initial overnight and same-day intraday movements on first-eligible earnings-call rows, one-day use yields higher mean AP than 30-day reuse across all ten learners. With matched numerical histories and company information, the GRU–logistic-regression gap is 1.08 AP points under one-day use but 3.76 under prolonged reuse; fresh/stale channels change this interaction, while a separate background study shows that stronger company information improves the no-content baseline more than the text model. The main learner-gap interaction is much smaller and uncertain on subsequent return windows. Together, these results show that measured text gains and model advantages depend on transcript assignment, what the predictor already knows, and the target window.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, multimodal forecasting, temporal alignment, evaluation, sparse records, text-augmented forecasting, financial forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kW1rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EventAlign, an evaluation framework for studying how the assignment of sparse textual records affects time series forecasting. Using financial data and earnings call transcripts, the authors compare different text exposure rules, input encodings, and background information. The results show that exposure rules can change both predictive performance and the relative advantages of different models.

Critical strengths

Clear and meaningful research question. The paper investigates an important but often overlooked issue in multimodal time series forecasting: how sparse textual information should be aligned with prediction samples.

Careful experimental design. The authors conduct controlled comparisons and provide extensive analyses of different exposure rules.

Critical weaknesses

Insufficient baselines. Although several numerical forecasting models are evaluated, the text representation experiments mainly focus on MiniLM, FinBERT and TF-IDF. There are many alternative approaches to multimodal forecasting. Since the paper proposes a new evaluation protocol, comparisons with more representative multimodal methods are important to establish its relevance.

Limited evaluation scope. The main experiments focus on a relatively small set of companies in one financial market. Evaluation across different financial markets and datasets would strengthen the generality of the conclusions.

Limited methodological novelty. The main contribution is an evaluation protocol rather than a new modeling technique. The individual components are relatively straightforward, making the methodological contribution somewhat incremental.

Fragmented experimental presentation. The paper includes many experiments with different evaluation protocols. While comprehensive, this increases the difficulty of understanding the main findings and makes the overall narrative less focused.

Questions for authors

Dependence on the prediction window. The main statistically significant learner-by-exposure interaction appears to depend strongly on the initial overnight return window. For subsequent future-return windows, the interaction becomes much smaller and statistically uncertain. Could the authors explain why this happens and clarify how the main findings generalize to genuine future forecasting?

Reviewer_qGHVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EventAlign, an evaluation framework that examines how the temporal assignment of sparse textual records affects forecasting performance and model comparisons. The authors used earnings-call transcripts and financial time-series data to compare different transcript exposure strategies. The experimental results indicate that exposure rules can substantially change the measured performance gaps between models, while temporal encoding and background information further influence these comparisons.

Critical strengths

S1. A meaningful evaluation perspective. The paper highlights that assigning sparse textual records to prediction samples is not merely a preprocessing decision but can influence model comparisons. This perspective is relevant to multimodal time-series forecasting, where textual information arrives irregularly.

S2. Extensive empirical analysis. The paper evaluates multiple forecasting architectures, exposure strategies, temporal representations, and background-information settings. The paired comparisons and additional sensitivity analyses provide useful empirical evidence about the impact of input exposure.

Critical weaknesses

W1. Limited methodological novelty. The central idea of examining temporal alignment and information availability in multimodal forecasting is closely related to existing research on event alignment, temporal fusion, and context-sensitive evaluation. EventAlign primarily reorganizes these considerations into a controlled evaluation protocol rather than introducing a fundamentally new methodology. The paper does not sufficiently establish what new methodological insight distinguishes its framework from prior evaluation approaches.

W2. The main empirical finding is highly dependent on the prediction target. The paper's strongest empirical result is obtained on the close-to-close return target, which includes overnight market movements that may have occurred before the earnings-call transcripts became available to the model. This makes the primary evaluation retrospective rather than strictly predictive. More importantly, the reported model-gap interaction becomes substantially smaller and statistically uncertain when evaluated on subsequent 1-, 3-, and 5-day return windows. The overnight-only analysis further indicates that the strongest effect is concentrated around the initial market reaction. Consequently, the central finding appears highly dependent on a specific retrospective evaluation setting, raising concerns about its generalizability and practical relevance to real-world forecasting.

W3. The experiments do not sufficiently establish the independent value of textual content. Although the paper demonstrates that exposure strategies affect model comparisons, these changes do not necessarily indicate that models are extracting meaningful predictive information from transcripts. In the main experiments, the GRU with textual inputs does not consistently outperform its no-content counterpart, and improvements may partly reflect event timing, company identity, or other metadata rather than transcript semantics. Consequently, the results establish sensitivity to input assignment more clearly than they demonstrate a genuine contribution from textual information.

Questions for authors

Please refer to the above comments.

Reviewer_GLDwOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether the way earnings-call text is reused changes the comparison between forecasting models. It compares using a transcript for one day with using it for 30 trading days. The main study uses 57 firms and six test periods in 2022–2024. The GRU's AP advantage over logistic regression changes from 1.08 to 3.76 percentage points under the two rules. The paper also tests separate channels for fresh and old text, company background features, and other input controls. The main result comes from a return window that partly starts before the text is assumed to be available. Results for later return windows are much smaller and uncertain.

Critical strengths
  1. The controlled comparison is useful. The paper changes text reuse while keeping the numerical inputs, samples, and model-selection rules matched. This directly tests whether the measured advantage of one model depends on how sparse text is assigned to daily rows.

  2. The authors report results that limit the scope of their finding. The future-window interactions are uncertain, and text does not beat the no-content baseline in some settings. Reporting these results helps readers judge where the finding applies.

Critical weaknesses

Independent confirmation remains unclear. Appendix A (p. 14) says the local analysis plans were fixed after the retrospective periods had been inspected. The paper uses chronological test splits and removes labels crossing split boundaries. However, it does not show a confirmation set kept unseen during analysis development. The seed, period, and resampling checks are useful, but use the same periods. The intervals are conditional on fitted predictions, as the authors state, and do not cover repeating model selection and the full fitting process. I therefore have less confidence that the finding would hold on an independent confirmation set.

Questions for authors
  1. Appendix A says the local plans were fixed after the retrospective periods had been inspected. Could you explain when the target windows, model-selection rules, and comparison families were fixed relative to viewing their results? Were any reported evaluation data kept unseen throughout that process?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.