acceptodds
Under review as a conference paper at ICLR 2027

DynaGuard: Reducing Negative Transfer through Controlled Context Injection in Multimodal Time Series Forecasting

Abstract

In time series forecasting, exogenous text such as news and reports can provide predictive signals not yet reflected in numerical histories, but unreliable, stale, or temporally misaligned context can increase forecasting error relative to a matched numerical-only reference, causing negative transfer. To reduce negative transfer, we address three linked questions: what content to retain, how to assess its temporal relevance, and where and how strongly to apply corrections. We propose DynaGuard, a backbone-compatible framework for controlled context injection: it models target dynamics from numerical histories and uses text to provide corrections governed by these three decisions. Time-Slice Context Compression (TCC) selects and compresses forecasting-relevant content within each historical interval. Context-Adaptive Temporal Bias (CATB) uses these compressed representations to predict salience and decay, guiding retrieval according to historical lag and numerical-token position. Content-Aware Gated Injection (CAGI) then uses the numerical state and retrieved context to control correction strength at each numerical token and feature channel. These stages learn jointly while preserving a direct numerical path to prediction. Our main comparisons cover 16 tasks from three data collections and six numerical backbones, with five random seeds per configuration. DynaGuard reduces negative transfer and improves forecasting accuracy. Ablations support each component's contribution, and the components also benefit other text-integration methods. Code is available at https://anonymous.4open.science/r/DynaGuard2027/.

Primary area:
learning on time series and dynamical systems
Keywords:
multimodal time series forecasting, exogenous text, negative transfer, temporal relevance, gated fusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kRgDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In time series forecasting, text can serve as exogenous information that guides predictions alongside the numerical look-back window. However, injecting unreliable text can mislead the forecasting model. To address this problem, the authors propose DynaGuard, which has three components: Time-Slice Context Compression (TCC) compresses the text within each time interval, Context-Adaptive Temporal Bias (CATB) generates an attention bias for each interval embedding and, Content-Aware Gated Injection (CAGI) generates a correction term for each numerical token. Because the final correction term is simply added back to the numerical token, DynaGuard is compatible with a wide range of backbones. Experiments show that DynaGuard reduces negative transfer and improves forecasting accuracy, and ablation studies confirm the contribution of each component.

Critical strengths
  1. The paper targets an important problem in multimodal time series forecasting: how to reduce the impact of irrelevant textual information on forecasting accuracy.
  2. The empirical results show that the approach works with several numerical forecasting architectures, which demonstrates its robustness.
  3. The paper is well written and easy to follow, and the added case study helps explain how the method works.
Critical weaknesses
  1. In the experiments, the main perturbation test randomly replaces text with a placeholder indicating missing information, across three domains. This does not establish resilience to plausible but incorrect text.
  2. Although six backbones are used, the evaluated tasks are all univariate, so it is unclear how well DynaGuard handles multivariate time series.
  3. When content and query are fixed, CATB imposes a relevance that decreases monotonically with the age of the text. This may be unsuitable for events whose effects emerge after a delay or recur periodically (for example, when a stock market is closed over the weekend, news from Friday may only matter on Monday).
Questions for authors
  1. Could the authors please address the three points raised in the "Weaknesses" section?
  2. In the TCC module, does the learnable query qcq_c carry any semantic meaning?
  3. In the CATB module, could the authors provide an intuitive explanation of the learned bias, or use a case study to show its value distribution and how it affects the results?
  4. The layout of the captions in Figures 3 to 5 should be improved to enhance readability.
5 comments
AuthorsOpenReview ↗

Response to Reviewer (kRgD) — Part 1/3

We thank the reviewer for the careful reading and constructive comments. Below, we respond to each concern and question, drawing on the additional controlled experiments and analyses in the revised manuscript. All figure, table, and equation references follow the revised manuscript.

W1 | Weakness 1 (Question 1): Robustness to Fluent but Incorrect Text

We agree that text drop, which replaces text with the missing-text placeholder, tests missing text only; it does not establish robustness to fluent but incorrect text. The revision adds a controlled test of the latter case.

We use controlled semi-synthetic tests to isolate the effects of text content and timing (Section 4.6, Appendices K–L). Events with known direction, magnitude, and onset are added to a real-world Traffic series from the Autoformer archive, each paired with a fixed text template. On Signed, numerical inputs and targets are held fixed while the text undergoes type swap, subtype swap, or reversal of its temporal order. These interventions preserve well-formed text while changing the stated event or its temporal placement. Each intervention increases error (Figure 24, Table 87), showing that the forecasts use event content and timing.

To assess performance under missing or temporally mismatched text, we compare perturbed DynaGuard with a Numerical-only backbone on matched origins and targets. On Base Traffic with PatchTST, horizon-averaged MSE remains below Numerical-only under text drop, within-window shuffle, and global shuffle at tested nominal rates up to 50% (Table 102).

In a separate experiment on Signed, Shuffle training augmentation improves performance under severe corruption. At 100% nominal corruption, it reduces MSE relative to clean-trained DynaGuard on identical test inputs by 7.75% (PatchTST) and 9.12% (iTransformer) under global shuffle, and by 4.48% and 5.39% under within-window shuffle (Figure 32, Table 82).

Base Traffic robustness with PatchTST — selected results from Table 102.

RateText dropWithin-window shuffleGlobal shuffleNumerical-only
0%2.699 ± 0.0562.699 ± 0.0562.699 ± 0.0563.343 ± 0.056
50%3.072 ± 0.0813.099 ± 0.0573.173 ± 0.0413.343 ± 0.056
100%3.927 ± 0.6543.506 ± 0.0643.693 ± 0.0973.343 ± 0.056

Note: MSE (×10⁻⁴), mean ± sample SD over five seeds after averaging four horizons. Nominal rates include placeholders; global shuffle is an offline perturbation.

W2 | Weakness 2 (Question 1): Multivariate Time Series

We thank the reviewer for raising this point. DynaGuard’s formulation accommodates multivariate numerical inputs and targets. The adapter applies residual corrections to numerical representations and gates the retrieved features at each token and feature channel (Equations 4 and 11). Across the six evaluated backbones, the same adapter design operates on variable tokens, patches, encoder states, and future segments.

Our evaluation studies text integration—content selection, temporal weighting, and correction strength—on single-target tasks, using the same task definitions for DynaGuard and all compared methods. Joint multivariate evaluation would examine how these textual corrections interact with dependencies across variables. We identify this evaluation as future work in Appendix A.

AuthorsOpenReview ↗

Response to Reviewer (kRgD) — Part 2/3

W3 | Weakness 3 (Question 1): CATB Monotonic Decay, Delayed Effects, and Recurring Events

With content and query fixed, the CATB bias B=s−λlog⁡(1+Δ)B=s-\lambda\log(1+\Delta) decreases with historical distance. The monotonicity concerns the temporal bias, not the forecast response, which also depends on query–context compatibility, the CAGI gate, and the prediction head (Figure 26). Moreover, ss and λ\lambda are predicted for each slice, so slices of the same age can receive different biases: content can override the age penalty, and the weighting across slices need not be monotone.

The main-paper case study (Figure 9) shows this behavior. In the displayed run, two reports describing past price conditions receive substantial attention, and masking them increases RMSE by 11.4% and 2.4%, although they lie six and seven weeks before the forecast. CATB, in contrast, reduces mean attention to a topically related but non-predictive report from 16.2% to 10.6%, and removing that report lowers RMSE. Relative influence therefore follows content as well as age.

Delayed effects are examined in Appendix K. For D events, whose effect starts eight steps after publication, retaining the bulletin lowers active-phase MSE by 5.48 on PatchTST and 5.45 on iTransformer (×10⁻⁴) and lowers the full 64-step MSE on both backbones (Table 95; Figure 28(d)). On both Signed and Delayed, CATB also lowers error relative to a shared decay within the same logarithmic family at short horizons (Figure 23), and has lower horizon-averaged MSE than both linear-decay alternatives in all nine real-world domains (Table 55). DynaGuard may capture periodic effects implicitly through changing numerical queries and their interactions with text. CATB’s bias itself decreases monotonically with age for fixed content and query and does not explicitly model recurring peaks in an event’s relevance. Appendix A discusses this limitation and identifies more flexible temporal functions as a direction for modeling these patterns explicitly.

Report-removal effects for D (Delayed), H=64H=64 — Table 95.

BackbonePhase / stepsKeep MSE (×10⁻⁴)Remove MSE (×10⁻⁴)ΔMSE (×10⁻⁴)
PatchTSTWait 1–75.233 ± 0.4612.489 ± 0.172−2.743 [−3.174, −2.312]
PatchTSTActive 8–399.838 ± 0.77515.315 ± 1.2375.477 [4.234, 6.720]
PatchTSTAfter 40–6412.788 ± 1.03312.189 ± 0.833−0.599 [−1.023, −0.175]
iTransformerWait 1–75.231 ± 0.8822.778 ± 0.206−2.453 [−3.436, −1.470]
iTransformerActive 8–398.813 ± 1.05914.262 ± 1.5415.449 [3.491, 7.408]
iTransformerAfter 40–6414.876 ± 1.75513.550 ± 1.041−1.326 [−3.654, 1.002]

Note: 14 notices, five seeds. Keep/Remove: mean ± sample SD. ΔMSE = Remove − Keep, mean [95% t interval]; positive favors retaining the notice.

AuthorsOpenReview ↗

Response to Reviewer (kRgD) — Part 3/3

Q2 | Question 2: Semantics of the Learnable TCC Query qcq_c

The learnable query qcq_c learns a preference for selecting text content relevant to the forecasting task. In Equation 5, each head projects it as qa=qcWaQq_a=q_cW_a^Q and matches this vector against text keys. The frozen pretrained input embeddings provide token representations, while qcq_c and the attention projections learn from forecasting error. The query is shared across time slices, and its attention distribution depends on the tokens present in each slice. Its interpretation therefore concerns the content it selects, without requiring individual heads to correspond to predefined semantic categories. Replacing it with a query derived from numerical history raises horizon-averaged MSE in all nine domains on iTransformer (Table 48, Appendix F.1). This result supports using a shared query for compression, with numerical conditioning applied during subsequent retrieval.

Q3 | Question 3: Interpretability of the Learned CATB Bias

The learned bias cannot be read directly from real-world forecasts, where text content, text age, and numerical history vary together. We therefore analyze it on the controlled semi-synthetic data (Section 4.6, Appendices K–L), where the event and its text are known and one factor can be varied at a time.

The bias has two learned components. Salience ss determines the importance of a slice's content; decay λ\lambda determines how quickly this weight decreases with age. On Signed, a sustained event (A1) receives positive salience (1.44 PatchTST, 1.34 iTransformer) and the missing-text placeholder negative, whereas a pulse (B1) has lower salience and a larger λ\lambda (Table 88). For PatchTST, the bias of A1 relative to the placeholder remains positive at all ages, whereas that of B1 changes from positive to negative with age (Figure 25).

We measure the forecast response to one A1 notice while holding weights, numerical histories, other text, and targets fixed. The response is the forecast with the notice minus the forecast with it removed. We compare notice ages 0 and 64 with the CATB bias enabled and disabled; the two ages give identical responses when the bias is disabled (Figure 26). The age contrast D(W)D(W) is the age-0 versus age-64 response difference with CATB enabled, minus the corresponding difference with CATB disabled. For A1, the paired contrast D(5:16)−D(33:64)D(5{:}16)-D(33{:}64) is 15.24 [3.33, 27.14] for PatchTST and 23.00 [18.54, 27.46] for iTransformer (×10⁻³ in target units; Table 93). Brackets give pointwise 95% tt intervals over five seed means, each averaging 37 fixed histories. Both intervals exclude zero, supporting a larger effect of text age over steps 5–16 than over steps 33–64. In this controlled experiment, CATB changes how text age affects corrections across the forecast horizon.

Q4 | Question 4: Caption Layout for Figures 3–5

We thank the reviewer for this suggestion. The captions of Figures 3–5 have been revised: all three use a common ragged-right layout with matched margins and no hyphenated line breaks, and each states its statistic—first-place counts over the 96 settings, including ties (Figure 3); wins, ties, and losses over the same settings (Figure 4); mean error reduction over the original methods across the three domains (Figure 5).

Reviewer_yCoDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method called DynaGuard, designed to reduce negative transfer from exogenous text in time series forecasting. The method combines Time-Sliced Context Compression (TCC), Adaptive Temporal Bias (CATB), and Gated Injection (CAGI). Experiments across 16 tasks and 6 base models demonstrate that the method not only improves forecasting accuracy but also reduces negative transfer.

Critical strengths
  1. The evaluation covers 16 tasks, 6 backbones, and five seeds.
  2. DynaGuard reduces the negative-transfer rate to 14.6%, compared with 37.5–71.9% for the multimodal baselines.
  3. The framework supports different forecasting backbones.
  4. Ablations show that all three components contribute to performance. Additional experiments also show benefits for other fusion methods.
Critical weaknesses
  1. The paper aims to handle unreliable, stale, and misaligned text. However, the experiments mainly use benchmark-aligned text. Random text removal is tested, but irrelevant, stale, and temporally shifted text are not. The claimed robustness is therefore not fully supported in the current version of the manuscript.
  2. The method combines established attention, temporal bias, and gating techniques. CATB changes both content adaptivity and the decay function. The ablations do not isolate these effects. The query-position offset also lacks clear justification across different backbone token types.
Questions for authors
  1. How does DynaGuard perform when handling irrelevant, outdated, or temporally shifted text?
  2. Can the authors compare CATB with a content-agnostic logarithmic bias? How is query position offset justified across different foundation models?
  3. Appendix K.2 notes that the configuration of GPT-2-XL differs from that used in the main experiments. However, Tables 81-82 indicate that the datasets from the main comparative experiments were reused. Which statement is correct? Were all embedding sources evaluated under the same configuration selection protocol?
7 comments
AuthorsOpenReview ↗

Response to Reviewer (yCoD) — Part 1/4

We thank the reviewer for the careful reading and constructive comments. Below, we respond to each concern and question, drawing on the additional controlled experiments and analyses in the revised manuscript. Unless otherwise specified, references to the manuscript follow the revised version.

W1 | Weakness 1 (Question 1): Robustness to Irrelevant, Stale, and Temporally Shifted Text

We thank the reviewer for identifying this gap in the robustness evaluation. We address it with text-corruption tests and a direct evaluation of stale announcements.

On three Time-MMD domains with iTransformer, replacing up to 90% of historical text with the missing-text placeholder produces a mean relative MSE increase below 6%; the increase is 7.2% when all text is missing (Section 4.4, Figure 8).

The controlled semi-synthetic experiments specify event effects and announcement timing explicitly. On Base Traffic, we train on unmodified text and apply three test-time perturbations while keeping numerical inputs and targets fixed: text drop, within-window shuffle, and global shuffle. Within-window shuffle changes the order of text within a history; global shuffle reassigns text across windows as an offline perturbation. Complete text lowers MSE by about 19% relative to Numerical-only, and MSE remains below this reference up to a nominal corruption rate of 50% (Table R1). At complete text deletion, MSE exceeds Numerical-only.

Table R1. Base Traffic robustness with PatchTST (Table 102). MSE (×10⁻⁴).

Nominal corruption rateText dropWindow shuffleGlobal shuffle
0%2.699 ± 0.0562.699 ± 0.0562.699 ± 0.056
30%2.913 ± 0.0342.936 ± 0.0553.048 ± 0.039
50%3.072 ± 0.0813.099 ± 0.0573.173 ± 0.041
100%3.927 ± 0.6543.506 ± 0.0643.693 ± 0.097

Numerical-only: 3.343 ± 0.056. Entries are means ± sample SD over five training seeds, after averaging 20 draws per nonzero rate and four horizons within each seed.

We evaluate training-time augmentation separately on Signed, a Traffic variant with both positive and negative event effects (Appendix K.7). Relative to models trained on unmodified text and evaluated under identical test perturbations, Shuffle augmentation lowers MSE at 100% nominal corruption by 7.75% and 9.12% for global shuffle and by 4.48% and 5.39% for within-window shuffle (PatchTST/iTransformer), with no statistically detectable change on clean inputs (Table 82, Figure 32). With Shuffle augmentation, horizon-averaged MSE remains below the corresponding Numerical-only model on both backbones even when all text is missing.

AuthorsOpenReview ↗

Response to Reviewer (yCoD) — Part 2/4

W1 | Weakness 1 (Question 1): Stale Announcements and Temporal Mechanisms

We also directly evaluate stale announcements on Signed using 177 matched test origins per forecast horizon. Each input contains one candidate announcement, with all other text slices replaced by the missing-text placeholder. Reliable and stale announcements have identical wording and event subtype. The reliable announcement concerns an event affecting the forecast horizon; the stale event has ended before the forecast origin, and its announcement retains its original publication position. Numerical inputs, targets, and model weights remain fixed.

Table R2 reports the resulting prediction errors and fixed-gate controls. Under stale inputs, adaptive injection has lower mean MSE than fully open injection on both backbones.

Table R2. Prediction error with stale announcements on Signed. MSE (×10⁻⁴).

BackboneReliable, adaptiveStale, adaptiveStale, G = 0Stale, G = 0.5Stale, G = 1
PatchTST4.490 ± 0.1904.998 ± 0.2595.532 ± 0.9585.414 ± 0.6427.074 ± 1.323
iTransformer4.769 ± 0.2025.442 ± 0.2204.789 ± 0.1285.281 ± 0.2696.858 ± 0.672

Entries are means ± sample SD over five training seeds, with H ∈ {8, 16, 32, 64} equally averaged within each seed. All columns use single-announcement histories and the same matched origins. The matched subset covers seven subtypes; B3 and C3 have no eligible pairs. Fixed gates modify the same trained checkpoints. G = 0 retains the output-projection bias and fusion layer normalization and is not an independently trained Numerical-only model.

The output-response experiment provides complementary evidence about the temporal mechanism. At fixed weights and numerical histories, moving a sustained-event announcement from age 0 to age 64 suppresses its early-horizon response; the responses at these two ages coincide when CATB is disabled (Figure 26). This measures how report age changes the forecast. In the displayed Energy case (test window 194, seed 2025), CATB lowers a report's mean attention across heads from 16.2% to 10.6%, and masking that report lowers RMSE by 1.4% (Section 4.5, Appendix H.2). On Signed, dropping text, swapping event type, swapping subtype, and reversing the 96 text slices all increase error at fixed weights, confirming that the model uses event content and its temporal placement (Appendix K.4; Table 87 in Appendix L.1).

Together, these experiments quantify performance under missing, shuffled, and stale text. They show that training augmentation improves tolerance to severe corruption on Signed and that controlling injection reduces error relative to fully open injection under stale-announcement inputs.

AuthorsOpenReview ↗

Response to Reviewer (yCoD) — Part 3/4

W2 | Weakness 2 (Question 2): CATB Design and the Query-position Offset

Following your suggestion, we compare CATB with a content-agnostic logarithmic bias. Shared log retains the same logarithmic distance function, removes the content-dependent salience term, and replaces the content-dependent decay coefficient with one learned coefficient per head (Table 78). This comparison tests content conditioning within the same logarithmic decay family.

CATB has lower mean MSE in 15 of the 16 settings in Table R3. After reversing all event effects in Base Traffic, CATB remains lower than Shared log at every horizon on both backbones (Tables 79, 85–86). Against the two linear alternatives, CATB has lower horizon-averaged MSE in all nine Time-MMD domains on both backbones (Table 55). On Electricity and Exchange, CATB has lower MSE than Shared log in 3 of 4 settings. Within each setting, the largest mean MSE among CATB, Shared log, ALiBi, and Learned linear is no more than 6.2% above the smallest (Table 80).

Table R3. CATB versus Shared log on Signed and Delayed (Tables 83–84). MSE (×10⁻⁴).

SettingBackboneHCATBShared log
SignedPatchTST82.638 ± 0.0452.881 ± 0.099
SignedPatchTST163.819 ± 0.0934.051 ± 0.125
SignedPatchTST325.586 ± 0.1385.753 ± 0.148
SignedPatchTST647.051 ± 0.4667.336 ± 0.446
SignediTransformer82.620 ± 0.1522.908 ± 0.085
SignediTransformer163.615 ± 0.2363.948 ± 0.237
SignediTransformer325.228 ± 0.2325.609 ± 0.259
SignediTransformer646.859 ± 0.1437.218 ± 0.174
DelayedPatchTST83.330 ± 0.1453.607 ± 0.068
DelayedPatchTST164.913 ± 0.2765.315 ± 0.150
DelayedPatchTST329.499 ± 0.4479.425 ± 0.727
DelayedPatchTST6412.346 ± 0.79512.547 ± 0.236
DelayediTransformer83.808 ± 0.0774.566 ± 0.136
DelayediTransformer165.016 ± 0.0985.893 ± 0.099
DelayediTransformer328.838 ± 0.6129.464 ± 0.786
DelayediTransformer6412.253 ± 0.48612.859 ± 0.241

Means ± sample SD over five training seeds; matched test windows at each horizon.

Our contribution coordinates text selection, temporal weighting, and where and how strongly retrieved context enters numerical representations. TCC supplies representations for CATB and retrieval. CAGI controls the retrieved features' injection. Numerical-only is the matched evaluation reference.

For the query-position offset, ui=i−1u_i=i-1 indexes the numerical tokens at the fusion interface and lets the temporal bias vary across query positions. In these single-target tasks, iTransformer and SOFTS each provide one token and u=0u=0. PatchTST provides historical patches; TimeXer provides historical patches followed by a global token; Transformer provides historical encoder states; and Crossformer provides future segments from its final decoder layer (Section 3.4, Appendix D.3).

The offset indexes tokens, not elapsed time. For fixed content and attention head, increasing uiu_i increases the penalty λjlog⁡(1+rj+ui)\lambda_j\log(1+r_j+u_i) and decreases its marginal growth with age, λj/(1+rj+ui)\lambda_j/(1+r_j+u_i). At each query, older text retains a larger distance. Attention weights also depend on content-specific salience, decay, and query–key compatibility.

Reviewer_jH2aOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DynaGuard, which serves as a context adapter to unimodal time series forecasters, such as PatchTST, iTransformer, etc. This is done by proposing three components: Time Slice Context Compression (TCC) provides retrieval of the most important and useful context per timestamp for forecasting; Context Adaptive Temporal Bias (CATB) controls the effect of the context over time frames; and Context Aware Gated Injection (CAGI) controls or selectively applies the intended effect of context into the numerical predictions. The empirical results are very strong, and the approach has been tested on a wide range of unimodal forecasting backbones and datasets.

Critical strengths
  1. The paper has a very strong and thorough empirical analysis with required ablations.

  2. The idea is sound and makes sense. The improvements over very recent approaches like TaTS is impressive.

Critical weaknesses
  1. The intuitive effects of individual components can be better elucidated with synthetic toy examples. For instance, can the authors develop a simple toy dataset where the effect of text on the forecast can be controlled so that it is very clear to observe the contribution of CATB with respect to the influence of context over future time frames?
Questions for authors

See the critical weakness section. Additionally,

  1. Can this approach be used to evaluate the usefulness of textual information in text-aided time series forecasting datasets in general? In other words, can this approach be used to determine the predictive power of text or the usefulness of text in a dataset?

  2. I find it a little bit incomprehensible to understand the effect of the text in Figure 9, causing the forecast to reduce. The text inputs do not seem to suggest any drop in gasoline prices. Additionally, the highlighted text inputs are from 6 to 7 weeks prior to the start of the forecast. What is the significance of that figure? In general, can the authors provide a convincing real-world example showcasing useful retrieval and its positive effect on accurate forecast estimation, along with a counterexample?

  3. A nice extension to showcase the usefulness of joint-training would be to compare against zero-shot approaches like simple averaging of the forecasts from tsfms and llms, as shown in [1].

  4. Can TCC, CATB, and CAGI work for large-scale training? In other words, would the domain information diversity affect the extension of TCC, CATB, and CAGI when being applied to iTransformer or PatchTST while being trained on a large-scale collection of datasets? What would be the contribution of these individual components if one model is trained for all 16 tasks considered in this paper?

6 comments
AuthorsOpenReview ↗

Response to Reviewer (jH2a) — Part 1/4

We thank the reviewer for the careful reading and constructive comments. Below, we respond to each concern and question, drawing on the additional controlled experiments and analyses in the revised manuscript. All section, appendix, figure, and table references follow the revised manuscript.

W1 | Weakness 1: Controlled Examples of Component Roles — CATB

Thank you for this suggestion. We constructed controlled semi-synthetic (CS) experiments that add events with known direction, magnitude, and timing to a real traffic background (Section 4.6; Appendices K–L). Events are sustained (A), pulsed (B), reported after onset (C), or announced before onset (D); the Signed variant mixes increases and decreases, and we also compare time biases on Electricity and Exchange.

With the model weights, numerical history, and all other text fixed, we measure the forecast change from adding one report, denoted by RR. Figure 11 isolates CATB's contribution as ΔR=Ron−Roff\Delta R=R_{\mathrm{on}}-R_{\mathrm{off}}, where “on” and “off” indicate whether the CATB bias is enabled. For a sustained event (A1, PatchTST), this contribution is positive and concentrated in early forecast steps for a recent report, and decreases as the report ages. With CATB disabled, RR is unchanged by report age in this intervention. Figures 26–27 report responses and contrasts for A/B/C; Figure 28 additionally examines C and D reports at the first forecast after publication.

Content and future-range contrasts on Signed (H = 64).

EventContrastPatchTST (×10⁻³)iTransformer (×10⁻³)
AAmplitude contrast: +1 − (−0.45), steps 5–1619.69 [3.91, 35.47]31.40 [27.16, 35.65]
AAmplitude contrast: +1.8 − (+1), steps 5–169.04 [1.70, 16.39]12.93 [5.30, 20.57]
A1Future-range contrast: DA1(5:16)−DA1(33:64)D_{\mathrm{A1}}(5{:}16)-D_{\mathrm{A1}}(33{:}64)15.24 [3.33, 27.14]23.00 [18.54, 27.46]

Note: Paired response differences in target units, mean [pointwise 95% t interval] over five seeds; each seed averages 37 histories. Rows 1–2: CATB on, announcement age 0. Source: Tables 91 and 93.

Rows 1–2 show that the correction scales with the announced amplitude, on both backbones. Row 3 turns to age. DA1(W)D_{\mathrm{A1}}(W) is the CATB-related age effect on future range WW: the change in response between ages 0 and 64 with CATB on, minus the same change with CATB off. Both backbones give a positive DA1(5:16)−DA1(33:64)D_{\mathrm{A1}}(5{:}16)-D_{\mathrm{A1}}(33{:}64), with intervals excluding zero: in this setting the effect of age is stronger at early steps.

AuthorsOpenReview ↗

Response to Reviewer (jH2a) — Part 2/4

W1 | Weakness 1: Controlled Examples of Component Roles — TCC and CAGI

We ran the same fixed-checkpoint interventions on the other two components. For TCC, we append neutral background text to event announcements and compare learned pooling with uniform pooling, with event tokens masked, and with the same number of background tokens masked. If learned compression retains event content, the first two should hurt more than the third; they do.

Component interventions on complete Signed histories.

ComponentInterventionMSE increase (%)
TCCUniform pooling+11.47
TCCMask event+18.78
TCCMask background−2.50
CAGIG = 0.5+9.64
CAGIG = 1+41.73

Note: MSE increase relative to learned pooling (TCC) or to adaptive gating (CAGI). Event and background masks remove equal token counts (three background masks averaged). Percentages are per-backbone–horizon increases averaged over two backbones and four horizons; each cell is a five-seed mean. Source: Figures 12–13 and Appendices K.6 and L.4.

Uniform pooling raises error by 11.5% and event masking by 18.8%, while masking an equal number of background tokens slightly lowers it: TCC keeps event content when background text is present. Replacing the adaptive gate with a constant gate also hurts, by 9.6% at G=0.5G{=}0.5 and 41.7% at G=1G{=}1: the gate does set how strongly text enters the forecast. The three tests therefore answer one question each—what to retain (TCC), how text age should shape retrieval (CATB), and how strongly to inject the correction (CAGI).

Q1 | Question 1: Assessing the Predictive Value of Text

Yes, as a conditional measure. With the backbone, adapter, and training protocol fixed, the difference between DynaGuard and TSOnly is what the dataset's observed text adds over a learned memory, and the difference between TSOnly and Uni-modal (the numerical-only baseline without the adapter) is what the adapter adds at all. On Agriculture and Energy, observed text is worth 15.7–22.1% MSE and 10.7–16.1% MAE across the two backbones.

DynaGuard vs. TSOnly on Agriculture and Energy.

DatasetBackboneMSE: TSOnly → DynaGuard (reduction)MAE: TSOnly → DynaGuard (reduction)
AgricultureiTransformer0.302 → 0.254 (15.7%)0.372 → 0.332 (10.7%)
AgriculturePatchTST0.376 → 0.293 (22.1%)0.408 → 0.350 (14.2%)
EnergyiTransformer0.303 → 0.241 (20.5%)0.423 → 0.355 (16.1%)
EnergyPatchTST0.311 → 0.252 (19.1%)0.426 → 0.363 (14.8%)

Note: Mean over four horizons and five seeds; percentages relative to TSOnly. Source: Appendix E.3, Tables 43–44.

The gain varies across domains, and Appendix E.3 reports results for all nine domains. These comparisons quantify the predictive benefit of observed text for the evaluated model and training protocol. At the report level, fixed-checkpoint masking tests whether retaining a report improves accuracy in the evaluated window; editing or shifting a report shows how its content and age affect the forecast.

AuthorsOpenReview ↗

Response to Reviewer (jH2a) — Part 3/4

Q2 | Question 2: Interpreting Figure 9 and Real-world Examples

Thank you for raising both points. We agree: the two highlighted reports describe the price level, not a downward direction, and they sit six and seven weeks before the forecast origin. The figure is about retrieval, not direction: it shows how CATB distributes attention over the 24 historical slices, and the highlighted reports are examples of the context it retrieves. The downward path is the model's final forecast from numerical history and text.

To attach accuracy to that picture, we added fixed-checkpoint interventions in the same window (Section 4.5; Appendix H.2).

Energy case study (sample 194, iTransformer, H = 24).

Input conditionWeeks before originRMSE (USD/gallon)RMSE change (%)
Full input—0.304293—
T1 masked70.338921+11.4
T2 masked60.311728+2.4
February 21 report masked140.299991−1.4

Note: The run shown in Figure 9 (seed 2025); each row changes only the named report, relative to the full input. Five-run results are in Appendix H.2.

Masking T1 or T2 raises RMSE in the displayed run (by 11.4% and 2.4%) and in each of the five trained models, so both reports help this forecast. The February 21 report is the counterexample you asked for: it is topically relevant (rising oil prices and Russian supply), CATB lowers its attention from 16.2% to 10.6%, and masking it lowers RMSE by 1.4%. Attention and contribution also come apart—T2 receives more attention than T1 (28.3% vs 21.5%) but matters less—which is why we test contribution by masking rather than by attention.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We thank the reviewers for their constructive feedback. We have uploaded a revised manuscript incorporating new experiments, extended analyses, and a correction to the embedding ablation tables in the supplementary material. The core methodology and main benchmark comparison results remain unchanged. Unless otherwise noted, all references below refer to the revised manuscript.

  1. New Controlled Experiments and Analyses — Section 4.6, Figures 10–13, and Appendices K–L. We have added controlled semi-synthetic experiments featuring known event effects paired with corresponding text reports. Studies based on the Traffic dataset cover persistent, impulse, retroactively reported, and pre-announced effects, while additional temporal-offset comparisons use the Electricity and Exchange backbones. New additions include a content-agnostic log-offset baseline, fixed-weight interventions on text content and recency, TCC and CAGI controls, text-misalignment tests, training-time text augmentation, and phase-specific evaluations for delayed-onset events. The appendices detail the construction methodology, experimental protocols, and full results.

  2. Extended Real-World Case Study — Section 4.5, Figure 9, and Appendix H.2. We expanded the gasoline price case study to include single-report masking and a counterexample featuring an uninformative report. The discussion distinguishes between attention weights and measured predictive contributions, and clarifies that the historical position of a highlighted report does not establish an equivalent causal lag.

  3. Correction to Embedding Ablation Results in the Supplementary Material — Appendix M.2, Tables 112–113 (corresponding to Tables 81–82 in the submitted version). The table-generation script inadvertently referenced the GPT-2-XL results from the main comparison rather than those from the already completed, unified fixed-configuration embedding ablation. We have fixed the script and replaced the affected entries with the intended ablation results. The corrected tables now consistently report all five embedding sources under the exact same fixed configuration, separate from the configuration selection protocol used in the main comparison. We apologize for this reporting error. We have also audited our remaining experiments and confirmed that no similar issues exist. This correction is confined to the embedding-source analysis in the supplementary material and does not affect the main benchmark results or core conclusions.

  4. Presentation and Documentation. We revised the captions and layouts for Figures 3–5, updated the Conclusion and Reproducibility Statement to reflect the newly added work, and adjusted appendix numbering and cross-references accordingly.

This comment summarizes the changes made to the manuscript. We address each reviewer's specific questions and present supporting evidence in the respective response threads.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.