acceptodds
Under review as a conference paper at ICLR 2027

Forecasts Are Paths: Curvature-Matched Hyvärinen Residuals and Overlap-Constrained Ordinal Flow

Abstract

Long-horizon forecasters produce trajectories, yet they are usually trained as if the horizon were a bag of independent coordinates. Pointwise squared error measures amplitude, but it neither controls how forecast errors evolve nor distinguishes trajectories whose local peaks, valleys, and turning order differ. We introduce HyOT, a single-pass loss that scores a forecast in two complementary geometries. In the cardinal geometry, a curvature-matched Gaussian–Student Hyvärinen score acts on both residual levels and residual innovations, combining the local efficiency of squared error with controlled tail influence. In the ordinal geometry, each five-point window induces a differentiable law over all \(5! = 120\) rankings. Because adjacent windows share four observations, every hard transition has an exact source–insertion representation over a \(120 \times 5\) transition alphabet. A factorized differentiable soft descriptor and entropy-gated Hellinger score align the predicted and observed ordinal flows. The resulting objective directly optimizes trajectory geometry during training while preserving the original inference procedure. Experiments across five forecasting architectures, six datasets, and four horizons show that optimizing the path, rather than isolated points, improves aggregate MSE and MAE. A representative trajectory-structure evaluation further shows that the MSE gain can be accompanied by lower Innovation MAE and Ordinal Hellinger and higher Local Rank Agreement.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Long-Horizon Forecasting, Forecasting Loss, Trajectory-Level Supervision, Hyvärinen Score, Ordinal Pattern Learning, Residual Modeling, Structured Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_WccJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HyOT, a trajectory-aware loss for long-horizon time-series forecasting. It keeps the existing forecasting models unchanged and replaces MSE with a loss combining a cardinal branch for residual information and an ordinal branch for local ranking transitions. Experiments on five models, six datasets, and four forecasting horizons show overall improvements over MSE and several existing structural losses.

Critical strengths

i) The paper provides a well-motivated trajectory-aware alternative to pointwise MSE. It explicitly considers temporal evolution and local ordering, which are not directly captured by standard pointwise MSE. ii) The proposed loss includes nontrivial mathematical design in both the cardinal and ordinal branches. The paper provides theoretical justification for the residual formulation and introduces an overlap-constrained ordinal transition representation. iii) The experiments cover multiple backbones, datasets, and forecasting horizons. The results suggest that the proposed loss is broadly applicable across different forecasting models and settings.

Critical weaknesses

i) The framework-level novelty is somewhat limited. HyOT does not introduce a new forecasting architecture, and the final objective is a weighted sum of two independently designed loss branches. ii) The novelty relative to prior work is not sufficiently clarified. Several ingredients, such as Hyvärinen scoring, permutation patterns, and Plackett–Luce ranking, are established concepts, so the paper should more clearly explain what is fundamentally new. iii) The validation of the trajectory-specific claim is relatively limited. The dedicated trajectory metrics are reported only in a representative setting, rather than across multiple datasets, backbones, and horizons. iv) The statistical robustness of the reported gains is unclear. The paper does not report repeated-run statistics such as standard deviations or confidence intervals, which makes it difficult to assess the stability of the improvements

Questions for authors

i) Since the final objective is a weighted sum of two independently designed loss branches, it would be helpful to clarify what makes the overall framework fundamentally different from simpler multi-loss formulations. Can the authors clarify the framework-level novelty of HyOT? ii) Several components of HyOT, including Hyvärinen scoring, permutation patterns, and Plackett–Luce ranking, are based on established concepts. It would therefore be helpful to clarify how the proposed formulation differs from the most closely related prior methods. Can the authors better distinguish the novelty of HyOT from prior work? iii) The trajectory-specific metrics are evaluated only in a limited representative setting. Extending this analysis to more datasets, backbones, and forecasting horizons would provide stronger support for the main trajectory-aware claim. Can the authors provide a broader trajectory-specific evaluation? iv) The reported results are mainly presented as single point estimates without repeated-run statistics. Reporting results over multiple random seeds with standard deviations or confidence intervals would help assess the stability of the improvements. Can the authors provide such statistical results?

1 comment

Dear Reviewer,

Thank you again for your valuable comments and feedback. As the rebuttal deadline is approaching, we would like to kindly follow up on our responses, in which we have addressed your questions and concerns.

We would greatly appreciate it if you could revisit our responses and consider whether they warrant an upward revision of your score. If you have any remaining questions or concerns, please do not hesitate to let us know. We would be more than happy to provide further clarification.

Thank you for your time and consideration.

Best regards, The Authors

Reviewer_z2tCOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HyOT is a training objective for long-horizon forecasting that combines curvature-matched Gaussian–Student Hyvärinen penalties on residual levels and first differences with Hellinger matching of overlapping ordinal-window descriptors. It leaves the inference architecture unchanged. Experiments cover five backbones, six datasets, and four horizons, reporting average relative MSE and MAE reductions of 2.12% and 2.80% over MSE training.

Critical strengths
  1. The residual construction is carefully specified. Local curvature matching, the admissible range preserving scalar gradient direction, and trace normalization have explicit derivations. Applying the penalty to residual differences also provides a clear mechanism for supervising temporal behavior without directly penalizing genuine target variation.

  2. The 120 matched settings and component ablations provide promising empirical support. Comparisons include several existing structural losses. The trajectory evaluation additionally reports improved local rank agreement.

Critical weaknesses
  1. The specific benefit of the transition formulation is not fully isolated. Equations 19–21 combine source and insertion marginals without retaining their statistical dependence. Although explicitly presented as a factorized descriptor, it is not compared with an additive loss using the same two marginals. Direct trajectory metrics are also limited to one configuration.
Questions for authors

Does the factorized Hellinger objective outperform an additive source-ranking and insertion-ranking loss under matched supervision and tuning? Please extend trajectory-metric comparisons beyond the single setting in Table 5.

1 comment

Dear Reviewer,

Thank you again for your valuable comments and feedback. As the rebuttal deadline is approaching, we would like to kindly follow up on our responses, in which we have addressed your questions and concerns.

We would greatly appreciate it if you could revisit our responses and consider whether they warrant an upward revision of your score. If you have any remaining questions or concerns, please do not hesitate to let us know. We would be more than happy to provide further clarification.

Thank you for your time and consideration.

Best regards, The Authors

Reviewer_TfyXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes HyOT, a novel loss function for long-term time series forecasting that treats predictions as structured trajectories rather than independent points. Itcombines curvature-matched Gaussian–Student Hyvarinen residual scoring with overlap-constrained ordinal flow to capture both numerical accuracy and local temporal ordering. The proposed loss can be directly integrated into existing forecasting models without modifying their architectures or inference procedures. Experiments across five forecasting models and six datasets demonstrate improvements in forecasting accuracy over conventional MSE training.

Critical strengths

The paper is well motivated, and introduces an interesting perspective on time series forecasting by explicitly considering trajectory geometry during optimization. The proposed HyOT loss combines residual dynamics and ordinal structure in a unified framework, supported by extensive theoretical derivations. Another strength is its model-agnostic design, which allows it to be incorporated into existing forecasting architectures without additional inference overhead. The authors also conduct experiments across multiple forecasting models and datasets, along with component ablations and hyperparameter sensitivity analyses, to validate the proposed HYOT.

Critical weaknesses
  1. In the comparisons with existing loss function designs for TSF, the authors do not explicitly indicate the publication years of the compared methods. I noticed that, except for PS Loss, the other methods were proposed approximately 7–9 years ago. Such comparisons are insufficient to convincingly demonstrate the competitiveness of the proposed method. I believe that at least two recent works on loss function design for TSF, published within the past 2–3 years, should be included, such as DBLoss and RI-Loss. Furthermore, the Related Work section should provide a more comprehensive discussion of recent advances in loss function design for TSF.
  2. The paper lacks a clear motivational figure illustrating the limitations of conventional L2 loss in time series forecasting. This makes it difficult for readers to quickly understand the specific problems the authors aim to address and the intuition behind their proposed solution. I suggest including a simple illustrative example to highlight the limitations of point-wise L2 supervision and demonstrate how the proposed method addresses them.
  3. Although the paper provides extensive theoretical proofs, it lacks sufficient visualizations or quantitative analyses to demonstrate the specific role of HyOT during training. For example, it would be helpful to show whether the predicted trajectories become closer to the ground truth at certain training iterations. Furthermore, the authors do not investigate whether Lres and Lord may introduce gradient conflicts that could negatively affect optimization.
  4. In Table 1, HyOT performs worse than the conventional L2 loss on certain forecasting tasks. Have the authors investigated the reasons behind these performance degradations? Furthermore, Table 1 mainly presents within-model comparisons. Could the authors also provide cross-model comparisons to demonstrate whether HyOT can help conventional forecasting models achieve SOTA performance?
Questions for authors
  1. Please include additional experimental comparisons with recent innovative loss functions for TSF.

  2. Please provide a motivational figure to better illustrate the limitations of conventional L2 loss and the key intuition behind the proposed method.

  3. Please provide additional analyses of training dynamics and address the concerns regarding potential gradient conflicts between the two loss terms.

  4. Please provide cross-model comparisons to demonstrate whether HyOT can help conventional forecasting models achieve state-of-the-art performance.

I will adjust my score based on the author's response.

2 comments

Dear Reviewer,

Thank you again for your valuable comments and feedback. As the rebuttal deadline is approaching, we would like to kindly follow up on our responses, in which we have addressed your questions and concerns.

We would greatly appreciate it if you could revisit our responses and consider whether they warrant an upward revision of your score. If you have any remaining questions or concerns, please do not hesitate to let us know. We would be more than happy to provide further clarification.

Thank you for your time and consideration.

Best regards, The Authors

Reviewer_TfyXOpenReview ↗

Response to authors

Thank you very much to the authors for their responses. I will maintain my recommendation.

Reviewer_sVgkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposed a new forecasting objective that combines cardinal geometry through curvature matched gaussian-student residual loss and ordinal geometry using overlapping windows. The objective is model-agnostic and can be applied with existing forecasting architectures. The empirical evidence showed improvement over MAE and MSE.

Critical strengths

The paper tackles an important problem in preserving the temporal structure, while ensuring pointwise accuracy. The theoretical results appear correct, and the ablations help shed into light the importance of the proposed components.

Critical weaknesses
  1. The writing of the paper seems convoluted and hard to follow at times. For e.g. the Introduction starts with PatchTST, and using patches, with no idea what patches are. It's one of many examples, but there is a lot of scope for writing improvement in the paper.
  2. The paper does not report standard deviations/confidence intervals, which makes it hard to assess the reported gains, given that many improvements are very small.
  3. Table 2 shows competing structural losses in only 6 settings, whereas there are 120 settings for MSE. There are no details about on baseline hyperparameters, or tuning procedures. At the same time, while ablations are there, they do not establish whether full construction can outperform simpler alternatives, such as using MSE with first-difference error, a robust loss with innovations, or simpler ordinal penalties, which makes it hard to judge the advantage over alternative training objectives.
  4. The theoretical results, while correct, they do not establish any statistical efficiency for the composite objective.
  5. If I understand correctly, Table 5 only evaluated the trajectory structure. in one model dataset-horizon setting and against only MSE training. More comparisons would better help support the paper's claim.
  6. Linear time complexity statement hides the cost of permutation, but it would be helpful to see training time etc to quantify the overall overhead.
Questions for authors

See Weaknesses above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.