acceptodds
Under review as a conference paper at ICLR 2027

CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting

Abstract

Although multivariate time series forecasting has broad application prospects, effectively incorporating exogenous variables remains challenging. Prevailing discrete-time architectures often impose rigid temporal alignment across heterogeneous variables, which may limit their ability to capture continuously evolving external effects. To address this limitation, we introduce CTIA-ODE, a continuous-time forecasting framework centered on Continuous-Time Inter-Variable Attention (CTIA). Specifically, CTIA is embedded within an ordinary differential equation (ODE) solver, enabling the continuously evolving endogenous trajectory to dynamically query a static exogenous context along the integration path without requiring explicit temporal alignment. To improve the stability of the continuous latent dynamics, we further introduce a dual-stabilization strategy: architecturally, a Linear Residual Augmentation (LRA) module provides a macroscopic trend anchor, while an Outlier-Robust Hybrid Loss (ORHL) mitigates the influence of heavy-tailed anomalies during optimization. Extensive experiments across eight real-world datasets demonstrate that CTIA-ODE achieves state-of-the-art forecasting performance across diverse datasets and prediction horizons, while maintaining a favorable accuracy-efficiency trade-off with a compact memory footprint and low inference latency, particularly under long-horizon forecasting settings. Anonymous code is available at https://anonymous.4open.science/r/CTIA-ODE-15A6/.

Primary area:
learning on time series and dynamical systems
Keywords:
Multivariate Time Series Forecasting, Neural Ordinary Differential Equations, Continuous-Time Models, Inter-Variable Attention, Long-Term Forecasting, Robust Optimization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_gY9TOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CTIA-ODE for long-term multivariate time series forecasting. Variable-level historical representations are evolved by a latent neural ODE whose vector field combines learned state dynamics with attention to static exogenous keys and values. The model additionally uses shared-weight refinement, a linear residual path, future calendar features, and an outlier-robust hybrid loss. Experiments on eight benchmarks report competitive forecasting accuracy, with component ablations and perturbation studies intended to explain the improvements.

Critical strengths

1.The combination of a linear forecasting path and a context-conditioned nonlinear branch is a useful design, and the reported ablations provide evidence that the components are complementary. For example, on ETTm1 at T=720, Table 2 reports MSE 0.472 for the full model, compared with 0.488 without the ODE branch and 0.548 without the linear path. Table 1 also reports strong MAE results, including the best MAE among the listed methods at all four Traffic horizons.

2.The release provides an inspectable implementation of the proposed architecture and a concrete record supporting part of the empirical evaluation. The available Weather log reports MSE/MAE 0.350809/0.342421 at T=720, consistent with the rounded values in Table 1. This is useful supporting evidence, although it is not an independent reproduction.

Critical weaknesses

1.Important results are inconsistent across tables, and the differences are not explained by the stated settings. For Weather at T=720, Table 1 and Table 6 report 0.351/0.342, whereas Table 7 reports 0.251/0.283 at the default ORHL parameters, δ=1 and λ=5. For ETTh1 with L=96 and T=720, Table 1 reports 0.526/0.504, while Table 8 reports 0.452/0.460. The available Weather log supports the main-table result but does not resolve the alternative result in Table 7; no corresponding ETTh1 log was found in the inspected release. These discrepancies affect the interpretation of the hyperparameter and look-back studies. They may be reporting errors or reflect undocumented protocol differences, but the current manuscript does not establish a consistent experimental account.

2.The evidence does not sufficiently isolate the benefit of continuous latent evolution from that of a nonlinear attention branch. Table 2 compares the full model with a “Pure Linear” variant; this supports adding a nonlinear branch, but does not establish an advantage over a comparable discrete residual or recurrent attention module. In Section 3.3 and the released implementation, integration occurs over a fixed latent interval [0,1] with static exogenous context, rather than following observation timestamps. The masking and shifting results in Appendix B are useful robustness tests, but do not by themselves attribute that robustness to ODE integration. Similarly, the released training loop applies global gradient clipping at norm 5.0, which needs to be accounted for when interpreting the claimed ORHL stability benefit. These gaps leave the central mechanistic explanation less well supported than the full-model accuracy results.

Questions for authors

1.Can you reconcile the Weather results in Tables 1, 6, and 7 and the ETTh1 results in Tables 1 and 8? Please identify whether the entries use the same data split, evaluation targets, preprocessing, input length, seed protocol, and model configuration, and provide the corresponding corrected entries or existing run records.

2.Is there a comparison that replaces the ODE integration with a discrete residual or recurrent attention module while retaining the same encoder, exogenous context, linear path, refinement, loss, and a comparable parameter/compute budget? If available, such a result would materially clarify the specific contribution of continuous integration beyond the current Pure Linear comparison.

3.Was the norm-5 gradient clipping in exp/exp_main.py applied identically to the full model and the MSE/MAE loss ablations? Please clarify which evidence distinguishes the effect of ORHL from that of gradient clipping in the stability analysis.

7 comments

Thank you for taking the time to provide this constructive feedback! We respond below to the identified weaknesses (W) and questions (Q).


W1/Q1: Differing results across tables


Thank you for identifying these inconsistencies. We sincerely apologize for these discrepancies between Table 1 and the appendix tables. The appendix reported results from an earlier version of the code. We did not update those results when we revised Table 1. We reran the relevant experiments and corrected the affected tables in the revised manuscript.

The specific corrections and reporting settings are as follows:

Table 7. The revised result for the default setting δ = 1 and λ = 5 is 0.245 MSE and 0.267 MAE. These values are averaged across T ∈ {96, 192, 336, 720} and match the Weather Avg row in Table 1. We have clarified this averaging protocol in the text.

Table 8 (now Table 13). The revised table reports MSE and MAE averaged across forecast horizons T ∈ {96, 192, 336, 720}. At L=96, the CTIA-ODE results are 0.245/0.267 on Weather and 0.396/0.390 on ETTm1, matching the corresponding Avg rows in Table 1. The other entries in Table 13 have also been updated based on the corresponding experiment results.

The corresponding entries use the same data split, evaluation targets, preprocessing, input length, seed protocol, and model configuration. All results and conclusions are now consistent. Additionally, the source code and experiment logs for these appendix experiments are uploaded to the anonymous repository at https://anonymous.4open.science/r/CTIA-ODE-15A6/Appendix/readme.pdf.


W2/Q2: ODE ablation (revised)


Thank you for this constructive suggestion. We agree that the Pure Linear comparison alone does not isolate the benefit of ODE integration. We therefore added a controlled comparison in Appendix E.4 (lines 964-1004) against a shared-weight gated recurrent cross-attention module.

Experimental design. Both models use the same encoder, auxiliary-context construction, linear branch, refinement network, loss, and training protocol. Both use ORHL. Only the ODE module is replaced. The discrete model performs 16 recurrent updates per forward pass. This matches the 16 vector-field evaluations in the RK4 model. The parameter counts differ by 129, or about 0.025%. Equal update and evaluation counts do not imply equal computational cost. We therefore also measure inference latency.

Results. Both models are evaluated on Weather. The input length is L=96, and the forecast horizon is T=96. Both use seeds 2026, 2027, and 2028.

Table. ODE integration versus discrete recurrent attention on Weather (L=96L=96, T=96T=96). Forecasting errors are reported as mean ± sample standard deviation across three paired seeds. Latency is measured per batch of 256 samples and reported as mean ± sample standard deviation across the three seed-specific average latencies.

Model VariantTest MSE ↓Test MAE ↓ParametersLatency (ms/batch) ↓
Discrete Recurrent Attention0.15653 ± 0.000590.19461 ± 0.00088514,50830.50 ± 0.14
CTIA-ODE (Proposed)0.15496 ± 0.000660.19359 ± 0.00066514,63736.30 ± 0.08

CTIA-ODE reduces mean MSE by 1.01% and mean MAE by 0.53% relative to the recurrent-attention model. Its inference latency is 19.0% higher. These results support the benefit of ODE integration over the tested discrete replacement under a nearly identical parameter budget. The full protocol and results are provided in Appendix E.4 (lines 964-1004) and summarized in the main-text ablation section (lines 414-418).


W2/Q3: Gradient clipping norm5


Thank you for raising this concern. The full model and the MSE- and MAE-loss ablations all used the same global L2 gradient-clipping threshold of 5.0. We added a controlled experiment in Appendix G.2 (lines 1170-1213) comparing each loss with and without gradient clipping to distinguish the effect of ORHL from that of clipping.

Controlled comparison. The experiments use Weather, L=96, and seed 2026. We report MSE and MAE averaged across four forecast horizons: 96, 192, 336, and 720.

Table. Loss and gradient-clipping comparison on Weather. MSE and MAE are averaged across the four forecast horizons. Max gradient norm denotes the maximum global L2L_2 gradient norm before clipping. Clip rate is the percentage of training steps that trigger clipping.

LossClipped MSE ↓Clipped MAE ↓Unclipped MSE ↓Unclipped MAE ↓Max gradient normClip rate
MSE0.2500.2770.2500.2770.7810%
MAE0.2480.2700.2480.2700.3750%
ORHL0.2450.2670.2450.2672.0600%

Results. The maximum gradient norm stays below 5.0 for every loss, so clipping is never activated. Forecasting errors are identical at the reported precision before and after removing clipping. ORHL achieves the lowest average MSE and MAE in both cases. These results show that ORHL's accuracy advantage in this setting is not attributable to norm-5 gradient clipping. We have added this analysis and the supporting results to Appendix G.2 (lines 1170-1213).

Reviewer_gY9TOpenReview ↗

Follow-up on Experimental Consistency and Released Materials

Thank you for the additional experiments. The discrete-attention comparison is useful, and the clipping logs substantially address that concern. However, inspection of the released materials leaves three issues requiring clarification:

  1. Internal inconsistency in the ODE comparison log. For seed 2026, the reported MSEs are 0.153207680 (RK4) and 0.155905229 (recurrent), giving a difference of 0.002697549, whereas the PAIRED line reports 0.001697548. Similar discrepancies occur for the other seeds and MAE. Although the reported means and standard deviations match the individual RESULT entries, the paired differences do not. Please reconcile these records with the supplied script and provide the corresponding original logs.

  2. Changed dataset and configuration. The original look-back experiment concerned ETTh1 at T=720, whereas revised Table 13 reports ETTm1 averaged across horizons. Please explain this change and resolve the original ETTh1 discrepancy. Moreover, table8.py uses hidden dimension 16 and learning rate 2e-5, versus 128 and 5e-4 stated in the manuscript. Which configuration produced the main-table ETTm1 results?

  3. Missing source dependencies. The supplementary scripts load weather.py and ettm1.py, but neither is included in the archive. Please provide the exact versions used.

I retain my current rating pending clarification of these consistency and traceability issues.

Reviewer_bbmeOpenReview ↗
Rating
4(+2)
Confidence
3(+1)
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

CTIA-ODE combines a Neural ODE with cross-variable attention to a static exogenous key-value context. Historical observations are encoded into variable representations; evolving latent states generate queries during integration over [0, 1]. The terminal states undergo a two-iteration refinement block. A parallel Linear Residual Augmentation (LRA) path combines trend and seasonal projections with future time-feature effects. Training uses an Outlier-Robust Hybrid Loss (ORHL).

The evaluation predicts all variables simultaneously across eight standard forecasting datasets. One designated reference variable is excluded from the shared auxiliary context, while all variable trajectories query that context (Section 3.1 and Appendix A, pp. 4 and 14). Thus, the implemented evaluation should not be summarized as exclusively a single-target, many-to-one forecast.

Critical strengths
  1. A coherent forecasting architecture. The linear and nonlinear branches provide complementary mechanisms, and static exogenous keys and values can be reused across solver evaluations.
  2. Competitive reported results in several settings. Table 1 shows promising performance on Electricity and Weather. The paper is not uniformly best: for example, its horizon-averaged Traffic MSE is 0.441 versus 0.428 for iTransformer (p. 7).
  3. Relevant empirical analyses. Table 2 studies architectural and loss variants; Appendix E compares solvers and initialization. Figure 3(c) includes a Huber-loss comparison under contamination, and Appendix B evaluates masking and temporal shifts. These experiments should be credited explicitly (pp. 8-9 and 15-17).
  4. Useful implementation disclosure. The appendices provide architecture settings, baseline configurations, complexity expressions, and forward-pass pseudocode.
Critical weaknesses
  1. Reported results require reconciliation before the empirical conclusions can be assessed confidently. The discrepancies below occur under apparently corresponding dataset and horizon labels. Different configurations could explain them, but such differences are not identified clearly.

    SettingMain or reference resultOther reported result
    Weather, forecast horizon 720Table 1, p. 7: MSE/MAE 0.351/0.342; Table 6, p. 17, agreesTable 7, p. 17, default delta = 1 and lambda = 5: 0.251/0.283
    ETTh1, look-back 96, forecast horizon 720Table 1, p. 7: 0.526/0.504Table 8, p. 22: 0.452/0.460
  2. The specific contribution of continuous integration is insufficiently isolated. The solver comparison is useful, but different solvers entail different numbers of vector-field evaluations. A discrete residual or repeated-attention control matched for parameters, loss, and function evaluations would help determine whether the advantage comes from ODE integration or iterative refinement. The need for such a control is an empirical attribution concern, not a claim that Neural ODEs are invalid.

  3. The temporal-alignment motivation exceeds the directly demonstrated sampling capability. A fixed integration interval is a legitimate latent-dynamics design and need not correspond to physical forecast time. However, the encoder still assumes regular sampling, as Appendix L acknowledges. Masking and shifting on a regular grid provide evidence of perturbation robustness, not direct processing of arbitrary irregular timestamps. Comparisons should isolate the role of continuous querying from the role of a static variable-level context.

  4. The loss contribution needs clearer positioning. Equation (5) gives exactly ORHL(e) = Huber_delta(e) + lambda * abs(e), under the conventional Huber definition. Figure 3(c) already compares ordinary Huber, so the remaining question is how much the full composite objective contributes across architectures. Appendix G explicitly acknowledges that the bounded prediction gradient does not globally bound the vector field. Stability should therefore be presented as an empirical finding rather than an established dynamical guarantee.

  5. Resource comparisons do not fully isolate architectural efficiency. The model uses hidden dimension 128, while several baselines use 512. This is not inherently unfair for a system comparison, but equal-resource or width-sweep comparisons are needed to attribute efficiency specifically to the proposed mechanism (Appendices C and J, pp. 16 and 21).

Questions for authors
  1. Can the two pairs of differing results be reconciled with exact configurations, seeds, checkpoints, and a corrected table? Which values correspond to the default model?
  2. Does the ODE branch outperform repeated cross-attention or a shared-weight residual network at the same number of vector-field evaluations and under the same loss?
  3. Beyond the existing masking and shift experiments, what evidence isolates continuous integration as the source of alignment robustness? Please distinguish latent integration time from observed timestamps.
  4. How much of the advantage remains if competing architectures receive the same Huber-plus-absolute-error loss? The existing ordinary-Huber contamination comparison is informative but does not answer this cross-architecture question.
  5. What does the accuracy-latency-memory frontier look like when model width is varied for both CTIA-ODE and its strongest baselines?
5 comments

Thank you for taking the time to provide this constructive feedback! We respond below to the identified weaknesses (W) and questions (Q).


W1/Q1: Differing results across tables


Thank you for identifying these inconsistencies. We sincerely apologize for these discrepancies between Table 1 and the appendix tables. The appendix reported results from an earlier version of the code. We did not update those results when we revised Table 1. We reran the relevant experiments and corrected the affected tables in the revised manuscript.

The specific corrections and reporting settings are as follows:

Table 7. The revised result for the default setting δ = 1 and λ = 5 is 0.245 MSE and 0.267 MAE. These values are averaged across T ∈ {96, 192, 336, 720} and match the Weather Avg row in Table 1. We have clarified this averaging protocol in the text.

Table 8 (now Table 13). The revised table reports MSE and MAE averaged across forecast horizons T ∈ {96, 192, 336, 720}. At L=96, the CTIA-ODE results are 0.245/0.267 on Weather and 0.396/0.390 on ETTm1, matching the corresponding Avg rows in Table 1. The other entries in Table 13 have also been updated based on the corresponding experiment results.

All results and conclusions are now consistent. Additionally, the source code and experiment logs for these appendix experiments are uploaded to the anonymous repository at https://anonymous.4open.science/r/CTIA-ODE-15A6/Appendix/readme.pdf.


W2/Q2: ODE ablation


Thank you for this question. We added a controlled comparison in Appendix E.4 (lines 964-1004) against a shared-weight gated recurrent cross-attention module. On Weather at L=96 and T=96, CTIA-ODE achieves lower mean forecasting errors than the tested recurrent-attention replacement.

Experimental design. We replace the ODE module with a discrete gated recurrent cross-attention module, RecurrentAttention. Both models use the same encoder, auxiliary-context construction, linear branch, refinement network, and training protocol. Both use ORHL. Only the ODE module is replaced. The discrete model performs 16 recurrent-attention updates per forward pass. This matches the 16 vector-field evaluations in the RK4 model. The parameter counts differ by 129, or about 0.025%. Equal update and evaluation counts do not imply equal computational cost. We therefore also measure inference latency.

Results. Both models are evaluated on Weather. The input length is L=96, and the forecast horizon is T=96. Both use seeds 2026, 2027, and 2028.

Table. ODE integration versus discrete recurrent attention on Weather (L=96L=96, T=96T=96). Forecasting errors are reported as mean ± sample standard deviation across three paired seeds. Latency is measured per batch of 256 samples and reported as mean ± sample standard deviation across the three seed-specific average latencies.

Model VariantTest MSE ↓Test MAE ↓ParametersLatency (ms/batch) ↓
Discrete Recurrent Attention0.15653 ± 0.000590.19461 ± 0.00088514,50830.50 ± 0.14
CTIA-ODE (Proposed)0.15496 ± 0.000660.19359 ± 0.00066514,63736.30 ± 0.08

CTIA-ODE reduces mean MSE by 1.01% and mean MAE by 0.53% relative to the recurrent-attention model. Its inference latency is 19.0% higher. These results support an accuracy advantage over the tested discrete replacement in this setting. The full protocol and results are provided in Appendix E.4 (lines 964-1004) and summarized in the main-text ablation section (lines 414-418).


W3/Q3: Alignment robustness attribution


Thank you for pointing this out. We explain the role of latent integration time and the evidence supporting alignment robustness below.

Latent integration time and observed timestamps. In CTIA-ODE, t ∈ [0, 1] is a latent integration coordinate. It describes how the encoded historical representation evolves inside the model. Observed timestamps indicate when measurements were recorded; t does not correspond to these timestamps or the elapsed time between observations. Although the ODE solver passes t to the vector field, the vector field does not use t explicitly. During ODE integration, the latent states query the static auxiliary context through attention. The prediction head then maps the terminal states to the full forecast horizon.

Continuous integration and alignment robustness. Discrete recurrent-attention models update their states in separate steps. These discrete jumps amplify phase errors under temporal misalignment or lag perturbations. ODE integration instead follows a smooth vector field, which dampens gradient spikes caused by temporal shifts. The discrete recurrent-attention comparison in Appendix E.4 (lines 964-1004) further supports the effectiveness of this mechanism. Both models use the same loss, and the recurrent update count matches the ODE evaluation count. The ODE model achieves lower mean forecasting errors across three seeds. Together with the masking and shift experiments, this comparison supports continuous integration as a contributor to alignment robustness.

We have revised the manuscript to clarify the role of latent integration time and the scope of our robustness claims.


W4/Q4: Cross architecture loss


Thank you for this suggestion. We added a cross-architecture loss comparison in Appendix E.6 (lines 1019-1063).

Experimental design. We train DLinear and iTransformer using either MSE or ORHL. The ORHL objective is Huber⁡δ=1(e)+5∣e∣\operatorname{Huber}_{\delta=1}(e)+5|e|. All experiments use Weather, L=96, T=96, and seed 2026. For each architecture, the two runs start from the same weights and use the same data-loading seeds and optimizer settings. We select each checkpoint by the lowest validation MSE. Only the training loss changes.

Results. The table reports errors truncated to three decimal places. Improvements are calculated from untruncated values relative to MSE training for each architecture. The CTIA-ODE row reports its previously obtained result for this setting.

Table. Cross-architecture loss comparison on Weather. L=96L=96, T=96T=96, and seed=2026.

ArchitectureTraining LossTest MSE ↓Test MAE ↓Improvement (MSE / MAE)
DLinearMSE0.1960.255—
DLinearORHL0.1900.2423.06% / 5.10%
iTransformerMSE0.1740.214—
iTransformerORHL0.1660.2014.60% / 6.07%
CTIA-ODEORHL0.1530.191—

For DLinear, ORHL reduces MSE by 3.06% and MAE by 5.10%. For iTransformer, the reductions are 4.60% and 6.07%. CTIA-ODE still achieves 7.83% lower MSE and 4.98% lower MAE than iTransformer trained using ORHL. Giving the competing models the same loss narrows the gap but does not eliminate the observed advantage of CTIA-ODE. The full protocol and results are provided in Appendix E.6 (lines 1019-1063) and summarized in the main-text ablation section (lines 390-412).


W5/Q5: Width efficiency comparison


Thank you for this suggestion. We added an experiment in Appendix E.7 (lines 1064-1093) comparing CTIA-ODE and TimeXer at matched hidden widths. The original comparison used different hidden widths, so it did not isolate the effect of width.

Experimental design. We compare CTIA-ODE and TimeXer at hidden widths of 128 and 512. All experiments use Weather, L=96, T=96, and seed 2026. All four configurations use ORHL and the same optimizer settings, batch size, and learning-rate schedule. We select each checkpoint by the lowest validation MSE.

Measurement protocol. We measure inference on an NVIDIA GeForce GTX 1650 in FP32. The batch size is 256. The models run in evaluation mode, and gradient computation is disabled. We run 10 warm-up passes and then time 30 forward passes. CUDA is synchronized before and after the timed region.

Table. Width comparison on Weather. L=96L=96, T=96T=96, and seed=2026. All configurations use ORHL. Test errors are truncated to three decimal places. Latency and peak allocated inference memory are measured per batch of 256 samples.

ModelWidthTest MSE ↓Test MAE ↓ParametersLatency (ms/batch) ↓Peak Memory (MiB) ↓
TimeXer1280.1650.214368,09624.46224.04
TimeXer5120.1650.2144,617,824171.06882.00
CTIA-ODE1280.1530.191514,63738.2467.16
CTIA-ODE5120.1580.1967,310,669220.97258.19

Results. At both widths, CTIA-ODE has lower forecasting errors and peak inference memory than TimeXer. However, it has higher latency and more parameters. Increasing the width to 512 raises resource requirements for both models. TimeXer's errors are unchanged at the reported precision, while CTIA-ODE's errors increase. CTIA-ODE at width 128 has lower forecasting errors, latency, and peak inference memory than its width-512 variant.

Reviewer_Cb6cOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CTIA-ODE proposes a multivariate long-term forecasting framework that embeds asymmetric inter-variable attention into a latent ordinary differential equation. One variable is designated as the primary channel, while the remaining M minus 1 variables form a static key-value context that is continuously queried by evolving latent states during ODE integration. The framework also combines a Stable Refinement block, a Linear Residual Augmentation module for trend and seasonal forecasting, and an Outlier-Robust Hybrid Loss for handling large residuals. It is evaluated on eight real-world datasets across four forecasting horizons and compared with nine representative baselines using MSE and MAE. Additional experiments examine component ablations, missing and misaligned auxiliary inputs, solver choices, loss sensitivity, input-window length, initialization, and computational efficiency. The reported results indicate competitive accuracy, especially for long-horizon forecasting, together with low memory usage and inference latency.

Critical strengths
  1. The CTIA mechanism is clearly formulated and closely connected to the stated motivation. The evolving latent state produces dynamic queries while the auxiliary variables form a static key-value context. This provides a concrete and mathematically specified asymmetric fusion mechanism that can be analyzed and reproduced.
  2. The empirical evaluation is broad and includes useful component analyses. The paper evaluates eight datasets, multiple horizons, nine baselines, ablations of the ODE, linear branch, CTIA, and loss function, as well as solver and robustness analyses. These results provide preliminary evidence that the complete architecture can be practically effective under the reported settings.
Critical weaknesses
  1. The central continuous-time and exogenous-variable claims are broader than the evaluated setting supports. The ODE evolves a hidden representation over a normalized interval from 0 to 1. It does not consume physical timestamps or time intervals. The encoder remains a fixed-step sequence encoder, and the paper itself acknowledges regular-sampling dependence in Appendix L. Moreover, the experiments use standard multivariate datasets in which all channels are jointly forecasted. The remaining M minus 1 channels are treated as exogenous by architectural assignment, rather than being genuinely external covariates with a separate sampling process or availability constraint. Appendix B tests zero masking and fixed temporal shifting on a regular grid, but does not establish support for genuinely irregular sampling or timestamp-aware forecasting. The headline claims should therefore be narrowed or supported by experiments with real timestamps and true external covariates. 2.The reported results contain internal inconsistencies and lack sufficient controlled evidence. Under apparently comparable settings, Table 1, Table 6, and Table 8 report Weather results of 0.351/0.342 at a prediction horizon of 720, whereas Table 7 reports 0.251/0.283 for the stated default delta equal to 1 and lambda equal to 5. For ETTh1 with a look-back length of 96 and a prediction horizon of 720, Table 1 reports 0.526/0.504, while Table 8 reports 0.452/0.460. ETTm2 also differs between the Table 1 result at a prediction horizon of 96 and the Original setting in Table 4. In addition, Table 1 contains eight datasets times four horizons times two metrics, which equals 64 metric entries, but the text describes baseline counts such as 15 out of 80. The paper reports no standard deviations or repeated-run results, and the efficiency comparison uses substantially different hidden dimensions and unspecified measurement conditions. These issues make it difficult to determine whether the reported gains are statistically stable and whether the efficiency conclusions are fairly controlled.
Questions for authors
  1. Please reconcile the numerical discrepancies across Tables 1, 4, 6, 7, and 8. Were the data splits, normalization procedures, evaluated channels, training budgets, checkpoint-selection rules, and random seeds identical? If the settings differed, please state the differences explicitly. If any values are typographical errors, please correct them and explain whether the corrections affect the main conclusions.
  2. Please define the primary-variable mechanism precisely. Which primary index is used for each dataset? What representations are modified by AsymmetricPrimaryFusion? Why are all M variables forecasted while only M minus 1 variables form the context? In Appendix H, H_primary is extracted but does not appear to be used afterward in the shown pseudocode. Please clarify the exact tensor flow and provide a sensitivity analysis for the primary-index choice.
  3. What does the ODE time variable represent? Is the interval from 0 to 1 only a latent integration coordinate, or is it intended to represent physical time? If it is only a latent coordinate, please revise the terminology and claims about continuous-time forecasting. If the model is intended to handle temporal misalignment, please explain how timestamps or time differences enter the computation and why the current fixed-step encoder is sufficient.
  4. Please report the complete statistical and computational protocol. Please provide the number of random seeds, mean and standard deviation or confidence intervals, checkpoint-selection procedure, parameter counts, FLOPs, actual function evaluations, inference batch size, numerical precision, and latency-measurement protocol. Please also correct the denominator and winner-count description in Table 1.
  5. Please clarify the novelty and evaluation of ORHL. Under the standard Huber definition, Equation 5 is equivalent to Huber loss with threshold delta plus lambda times the absolute residual. What is the specific methodological contribution beyond this combination? In Figure 3(c), were the Huber baseline and ORHL trained with identical tuning procedures? Please specify the heavy-tailed contamination distribution, magnitude, and repetition protocol.
3 comments

Thank you for taking the time to provide this constructive feedback! We respond below to the identified weaknesses (W) and questions (Q).


W2/Q1: Numerical discrepancies across tables


Thank you for identifying these inconsistencies. We sincerely apologize for these discrepancies between Table 1 and Tables 4, 6, 7, and 8 in the appendix. The appendix reported results from an earlier version of the code. We did not update those results when we revised Table 1. We reran the relevant experiments and corrected the affected tables in the revised manuscript.

The specific corrections and reporting settings are as follows:

Table 4. The corrected Original result on ETTm2 at L=96 and T=96 is 0.175 MSE and 0.253 MAE, matching the corresponding entry in Table 1. The other experimental results in Table 4 have also been updated based on the corresponding experiment results.

Table 6. The RK4 result on Weather at L=96 and T=720 is 0.351 MSE and 0.342 MAE, matching Table 1.

Table 7. The revised result for the default setting δ = 1 and λ = 5 is 0.245 MSE and 0.267 MAE. These values are averaged across T ∈ {96, 192, 336, 720} and match the Weather Avg row in Table 1. We have clarified this averaging protocol in the text.

Table 8 (now Table 13). The revised table reports MSE and MAE averaged across forecast horizons T ∈ {96, 192, 336, 720}. At L=96, the CTIA-ODE results are 0.245/0.267 on Weather and 0.396/0.390 on ETTm1, matching the corresponding Avg rows in Table 1. The other entries in Table 13 have also been updated based on the corresponding experiment results.

All results and conclusions are now consistent. Additionally, the source code and experiment logs for these appendix experiments are uploaded to the anonymous repository at https://anonymous.4open.science/r/CTIA-ODE-15A6/Appendix/readme.pdf.


W1/Q2: Primary-variable mechanism


Thank you for these questions.

Primary-variable selection. We select the final non-date feature column as the primary reference variable for all datasets. Its index is primary_idx = M - 1. The remaining M-1 variables form the auxiliary context. This is a fixed indexing convention. We have clarified this rule in Section 3.1 (lines 193-204) and Appendix A (lines 791-798).

Representations modified by AsymmetricPrimaryFusion. The encoder produces the variable representations E∈RM×DE \in \mathbb{R}^{M \times D} defined in the main text. In the code and Appendix H pseudocode, these representations are stored as HH. Inside AsymmetricPrimaryFusion, the primary token generates a query, and the auxiliary tokens generate keys and values. The attention output passes through a gate and is added to the primary token, followed by layer normalization. The updated primary token is written back into HH. Only the primary token is modified in this module; the auxiliary tokens remain unchanged.

Why forecast all M variables while using M-1 for context? We forecast all M variables because the full tensor HH enters the ODE and the prediction head. After fusion, the representations of the M-1 auxiliary variables are projected into static keys KexoK_{exo} and values VexoV_{exo}. All M variable states query this shared context during integration.

The unused H_primary slice in Appendix H. H_primary is used inside AsymmetricPrimaryFusion to update the primary representation. However, in the part shown in Appendix H, the H_primary slice was not utilized. We have removed that line from the revised pseudocode.

Primary-index sensitivity. We use primary_idx = M - 1 for all datasets. We have not tested alternative primary indices, so sensitivity to this choice has not yet been evaluated. Our results therefore demonstrate performance under the fixed final-channel convention.


W1/Q3: ODE time variable representation


Thank you for your comment.

The ODE time variable representation. In CTIA-ODE, t ∈ [0, 1] is a latent integration coordinate, not physical time. It just describes how the encoded historical representation evolves inside the model. Although the ODE solver passes t to the vector field, the vector field does not use t explicitly. Its output depends on the current latent state and the static auxiliary context.

Continuous-time forecasting. We have clarified in Section 3.2 (lines 214-216) that “continuous-time” refers to the evolution of latent states over t ∈ [0, 1]. During ODE integration, these states query the static auxiliary context through attention. The prediction head then maps the terminal states to the full forecast horizon.

Time information and the encoder. The known future time features T_f enter through Temporal Prior Injection and contribute to the final prediction. For the regularly sampled inputs considered here, fixed positions in the historical window represent consistent time lags, which is why the current fixed-step encoder is suitable for this setting. Its limitations in handling irregularly sampled data are discussed in Appendix L (lines 1363-1371).


W2/Q4: Parameters and winner-count


Thank you for asking us to clarify these points.

Statistical protocol and checkpoint selection. Table 1 reports results from seed 2026. In the newly added experiment in Appendix E.4 (lines 964-1004), we compare CTIA-ODE against a discrete recurrent-attention model. Both models use seeds 2026, 2027, and 2028. We report the mean and sample standard deviation across these three seeds. Each model is trained for up to 100 epochs. Training stops if validation MSE does not improve for 12 epochs. The checkpoint that achieves the lowest validation MSE is selected for test evaluation.

Parameters and computation. Both models use the same encoder, auxiliary-context construction, linear branch, refinement network, loss, and training protocol. Only the ODE module is replaced. CTIA-ODE has 514,637 trainable parameters. The recurrent-attention model has 514,508. The difference is 129 parameters, or about 0.025%. RK4 uses five grid points over [0,1]. These define four integration intervals and require 16 vector-field evaluations per forward pass. The discrete model performs 16 recurrent-attention updates. The newly added experiment in Appendix E.7 also reports parameter counts, inference latency, and peak allocated inference memory at hidden widths of 128 and 512.

Latency measurement. We measure inference latency on an NVIDIA GTX 1650 in FP32. Each batch contains 256 samples. The models run in evaluation mode, and gradient computation is disabled. Inputs are already on the GPU. For each trained model, we run 10 warm-up passes and time 30 forward passes. We synchronize CUDA before and after the timed region. Input transfer and metric computation are excluded. For each seed, we average the latency of the 30 timed passes. We then report the mean and sample standard deviation across the three seed-specific averages.

Table. ODE integration versus discrete recurrent attention on Weather. L=96L=96 and T=96T=96. Values are mean ± sample standard deviation across three paired seeds (2026, 2027, 2028). Latency is measured per batch of 256 samples.

Model VariantTest MSE ↓Test MAE ↓ParametersLatency (ms/batch) ↓
Discrete Recurrent Attention0.15653 ± 0.000590.19461 ± 0.00088514,50830.50 ± 0.14
CTIA-ODE (Proposed)0.15496 ± 0.000660.19359 ± 0.00066514,63736.30 ± 0.08

CTIA-ODE reduces mean MSE by 1.01% and mean MAE by 0.53% relative to the discrete model. Its inference latency is 19.0% higher.

Table 1 counting rule. Table 1 contains 64 horizon-level entries across eight datasets, four horizons, and two metrics. It also contains 16 dataset-average entries (lines 362-363). We count ties at the reported precision as joint first-place results. CTIA-ODE ranks first in 33 of the 64 horizon-level entries and 11 of the 16 average entries, giving 44 of 80 entries overall. The average entries summarize the horizon-level results and are not independent forecasting tasks.


Q5: Novelty and evaluation of ORHL


Thank you for these questions.

Loss formulation. We agree that, under the standard Huber definition, Equation 5 is algebraically equivalent to LORHL(e)=Huberδ(e)+λ∣e∣L_{\mathrm{ORHL}}(e) = \mathrm{Huber}_{\delta}(e) + \lambda \lvert e\rvert ORHL is not an entirely new loss family, but it is not simply a constant rescaling of Huber loss. We use it in CTIA-ODE to combine the Huber penalty and an additional absolute-error penalty. The added term changes the penalty and gradient as a function of the residual.

Scaled Huber comparison. We added an experiment in Appendix E.5 (lines 1006–1035) to test whether scaling Huber loss can match the reported ORHL result. We train five configurations of c⋅Huber⁡δ=1c \cdot \operatorname{Huber}_{\delta=1}, where c ∈ {1, 2, 3, 4, 5}. The dataset is Weather, the input length is L=96, the forecast horizon is T=720, and the random seed is 2026. All five runs start from the same weights and use the same data-loading seeds and training hyperparameters. For each run, we select the checkpoint that achieves the lowest validation MSE. Test MSE ranges from 0.354755 to 0.355099, and test MAE ranges from 0.348143 to 0.348371. Gradient clipping is never activated in these runs. The reported ORHL result is approximately 0.351 MSE and 0.342 MAE.

Training protocol for Figure 3cc. ORHL and Huber baseline used the same hyperparameter search space for learning rate, weight decay, batch size, and scheduler patience.

Contamination protocol. The experiment uses ETTm1. The input length is L=96, and the forecast horizon is T=720. We first standardize each variable using statistics fitted on the clean training split. We then corrupt only the training labels. Historical inputs, validation data, and test data remain clean. Each eligible training-label entry is selected independently at contamination rate η ∈ {0, 0.05, 0.10, 0.15, 0.20}. We add 5z to each selected entry. Here, z follows a Student-t distribution with three degrees of freedom. The factor 5 sets the noise scale in standardized units. All loss objectives share the same corruption masks and noise values. The masks are nested across contamination rates, so entries selected at a lower rate remain selected at higher rates. Each loss–contamination configuration is trained once. The training seed is 2026, and the contamination seed is 52026. All loss objectives start from the same model weights. These results therefore describe one training seed and one contamination realization.

We have added these protocol details to the revised manuscript (lines 436-440). Additionally, we also added a cross-architecture loss comparison in Appendix E.6 (lines 1019-1063).

Experimental design. We train DLinear and iTransformer using either MSE or ORHL. The ORHL objective is Huber⁡δ=1(e)+5∣e∣\operatorname{Huber}_{\delta=1}(e)+5|e|. All experiments use Weather, L=96, T=96, and seed 2026. For each architecture, the two runs start from the same weights and use the same data-loading seeds and optimizer settings. We select each checkpoint by the lowest validation MSE. Only the training loss changes.

Results. The table reports errors truncated to three decimal places. Improvements are calculated from untruncated values relative to MSE training for each architecture. The CTIA-ODE row reports its previously obtained result for this setting.

Table. Cross-architecture loss comparison on Weather. L=96L=96, T=96T=96, and seed=2026.

ArchitectureTraining LossTest MSE ↓Test MAE ↓Improvement (MSE / MAE)
DLinearMSE0.1960.255—
DLinearORHL0.1900.2423.06% / 5.10%
iTransformerMSE0.1740.214—
iTransformerORHL0.1660.2014.60% / 6.07%
CTIA-ODEORHL0.1530.191—

For DLinear, ORHL reduces MSE by 3.06% and MAE by 5.10%. For iTransformer, the reductions are 4.60% and 6.07%. CTIA-ODE still achieves 7.83% lower MSE and 4.98% lower MAE than iTransformer trained using ORHL. Giving the competing models the same loss narrows the gap but does not eliminate the observed advantage of CTIA-ODE. The full protocol and results are provided in Appendix E.6 (lines 1019-1063) and summarized in the main-text ablation section (lines 390-412).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.