acceptodds
Under review as a conference paper at ICLR 2027

RAGE: Reference-Guided Data Resampling via Adaptive Gradient Estimation for Time Series Foundation Models

Abstract

Data allocation can improve the downstream performance of time-series foundation models by prioritizing source windows relevant to a target task. However, time-series trajectories are often delimited by data acquisition rather than semantic boundaries and can contain heterogeneous local training signals. Averaging gradients over an entire trajectory can obscure this variation, whereas computing a gradient for every overlapping window is costly. This creates a challenge of retaining useful local information for reference-guided allocation under a limited gradient budget. We propose RAGE, a reference-guided resampling method that estimates and scores local gradient shapes. RAGE places sparse gradient anchors using coverage and refinement guided by interpolation risk and direction cancellation. Neighboring anchors define linear gradient shapes on a common unit interval, encoded by jointly energy-normalized midpoint and slope components. RAGE aggregates signed source–reference shape similarities with trajectory-balanced reference weights and maps the resulting block-score ranks to source-window sampling probabilities. We evaluate RAGE on Chronos-Base, TimesFM 2.5, and TimerXL across seven forecasting tasks and two seeds, comparing against selection baselines and component ablations. Under a nominal 1.5% window-gradient budget, RAGE reduces MASE by 2.03% on average relative to random resampling and by 2.37% relative to LESS-TS. RAGE achieves the highest mean improvement among the evaluated methods and ablations overall and on each model.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Foundation Models, Data Resampling, Gradient-Based Data Valuation, Adaptive Gradient Estimation, Reference-Guided Learning, Efficient Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EvBnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RAGE resamples source training windows for time series foundation model adaptation by matching local gradient shapes, reconstructed from sparse adaptive anchors, to target reference data.

Critical strengths
  1. The paper addresses a meaningful limitation of trajectory-level selection: averaging gradients can obscure locally useful training windows.
  2. The midpoint–slope representation captures local gradient variation using existing anchor gradients, without additional backward passes.
  3. Experiments across three foundation models and seven tasks, with matched training exposure and component ablations, support the benefit of local shape matching.
Critical weaknesses
  1. The ablations do not isolate adaptive anchor placement from uniform placement, leaving a central contribution insufficiently validated.
  2. The connection between normalized gradient-shape similarity and beneficial adaptation remains heuristic, particularly when gradients are computed only at the initial checkpoint.
  3. The practical advantage remains unclear because comparisons match training exposure rather than total computation and omit direct adaptation on the available target reference histories.
Questions for authors
  1. Does adaptive anchor placement outperform uniformly spaced anchors under the same gradient budget, and what does each refinement stage contribute?
  2. Why should matching gradient slopes across independently rescaled blocks predict useful training updates, and does this relationship persist during adaptation?
  3. How does RAGE compare with direct fine-tuning on the target reference histories, especially when those histories are short or contain regime changes?
  4. Does RAGE retain an advantage over random sampling when gradient preparation and post-training share the same total compute budget?
  5. Does the advantage over LESS-TS persist when optimizer-aware gradient features and multiple checkpoints, omitted in the current adaptation, are retained?
5 comments
AuthorsOpenReview ↗

Response to Reviewer EvBn

Response to Reviewer EvBn

We sincerely thank the reviewer for the careful evaluation of our work and for recognizing the research problem, local gradient-shape representation, and cross-model experimental validation. We particularly appreciate the specific suggestions regarding methodological explanations, experimental fairness, and practical computational costs, which will help us further improve the rigor of the paper.

W1, Q1: Adaptive Anchor Placement and Ablation

We thank the reviewer for highlighting the value of a direct comparison between adaptive and uniform anchor placement, as well as more detailed validation of the contributions of individual CRB stages.

Methodological Explanation: CRB is designed to address the practical challenge of gradient reconstruction under a limited budget, where high-dimensional gradients at unobserved locations are difficult to predict accurately. Rather than requiring globally optimal reconstruction, CRB uses available gradient information to identify and reduce currently observable local reconstruction risks as much as possible.

Specifically, Coverage ensures basic coverage; Risk refinement prioritizes intervals with potentially larger reconstruction errors based on observed interpolation residuals; and Cancellation refinement focuses on intervals with opposing endpoint gradient directions, where local information may be obscured by averaging. These three stages target different risks rather than attempting to accurately predict all unknown gradients.

Revision: We are adding a direct comparison between Uniform and CRB under the same gradient budget, while keeping the remaining Shape+Linear settings fixed, to isolate the contribution of adaptive anchor placement. We will also further explain the design rationale of the three stages in the revised manuscript and clarify which stage-specific contributions have not yet been independently validated.

W2, Q2: Gradient-Shape Similarity and Checkpoint Dependence

We thank the reviewer for raising important questions regarding the theoretical basis of gradient-shape matching and the choice of checkpoint.

Explanation: Conventional gradient matching uses gradient inner products to approximate the effect of a single update on the target loss. RAGE extends this idea to locally reconstructed linear gradient functions, whose integrated inner product satisfies:

$ \int_{-1/2}^{1/2}g_s(u)^\top g_r(u),du=\bar g_s^\top\bar g_r+\frac{1}{12}d_s^\top d_r. $

The two terms measure the alignment of mean gradients and local gradient variations, respectively. Therefore, even when the mean gradients are identical, Shape Matching can still distinguish different local variations. Joint energy normalization further makes the similarity equivalent to the normalized inner product of the reconstructed gradient functions, providing a clear mathematical interpretation rather than a strict guarantee of training benefits.

Regarding the checkpoint, we use gradients at the initial checkpoint for one-time data valuation to avoid the additional cost of repeatedly computing gradients during training. While gradient relationships may change during adaptation, the use of initial gradients represents a deliberate trade-off within the current fixed-budget method.

Revision: We will add the above mathematical derivation to the revised manuscript, clarify the connection and distinction between Shape Matching and first-order gradient matching, and further explain the rationale and applicability of using the initial checkpoint.

We sincerely thank the reviewer for this suggestion, which helps us further improve the theoretical explanation of our method.

AuthorsOpenReview ↗

Response to Reviewer EvBn (Continued: W3–W5)

Response to Reviewer EvBn (Continued: W3–W5)

This is a continuation of our previous response, addressing the remaining concerns W3–W5.

W3, Q3: Direct Fine-tuning on Reference Data

We thank the reviewer for suggesting a comparison with direct fine-tuning on the target reference data. This would be a valuable additional baseline for evaluating the practical value of reference-guided data selection.

RAGE uses target reference information to select training windows from a larger source pool, whereas direct training on the reference data adapts the model using limited target data. Both approaches have different advantages and merit direct comparison, particularly when reference histories are short or contain distribution shifts.

Additional Experiments: We plan to add a Ref-trained (RefT) baseline that directly fine-tunes on the target reference histories, while maintaining consistent model initialization and training budgets as much as possible. We will update the experimental results once completed to more clearly evaluate the performance and applicability of RAGE relative to direct target fine-tuning.

W4, Q4: End-to-End Compute Budget

We thank the reviewer for raising the important issue of fairness under an end-to-end compute budget. Including gradient-preparation costs in the total budget provides an additional perspective for evaluating the practical computational efficiency of data selection methods.

Explanation: The primary goal of our work is to compare the effectiveness of different resampling strategies under fixed post-training exposure, thereby isolating the effect of data allocation itself, rather than demonstrating that RAGE achieves higher efficiency under an equal end-to-end compute budget.

Meanwhile, we have separately reported the computational costs of gradient preparation, data selection, and training in Appendix I of the original manuscript, and explicitly stated that these results do not directly establish end-to-end training speedup.

Converting data-valuation costs into additional training budgets and comparing against random sampling under equal total computation would provide valuable complementary evidence. However, this requires a unified method for measuring computational costs and rerunning the corresponding experiments. Due to current computational-resource and revision-time constraints, we are temporarily unable to complete this full comparison.

Revision: We will more clearly distinguish between data-selection benefits under a fixed training budget and computational efficiency under a fixed end-to-end budget in the revised manuscript, and identify the latter as a direction for future research.

We thank the reviewer for helping us more accurately define the conclusions supported by our current experiments.

W5, Q5: Fairness of the LESS-TS Baseline

We thank the reviewer for highlighting the differences between our current LESS-TS implementation and the original LESS method. Our implementation is a time-series adaptation designed for a unified experimental setting, rather than a complete reproduction of the original LESS pipeline. In particular, optimizer-aware gradient features and multiple checkpoints are not included, and these differences may affect the comparison results.

Explanation: We initially adopted this simplified implementation mainly to maintain consistent training procedures and gradient-computation budgets across different time-series foundation models. However, fully transferring the original LESS pipeline to time-series models also involves optimizer-aware gradients, checkpoint selection, model training paradigms, and differences between the two data types in training-sample construction and evaluation. Therefore, how to construct a time-series adaptation that is strictly equivalent to the original LESS still requires further investigation.

Revision: In the revised manuscript, we will explicitly describe this baseline as a LESS-inspired gradient matching baseline and systematically revise the related naming, implementation details, and comparison conclusions. We will clearly identify its differences from the original LESS to avoid equating the performance of the current simplified implementation with that of the complete LESS method.

We sincerely appreciate the reviewer's rigorous attention to baseline fairness. This feedback helps us avoid overly strong comparative claims and more accurately define the scope of our current experiments.

AuthorsOpenReview ↗

Update on the Revised Manuscript

Update on the Revised Manuscript

We sincerely thank the reviewer again for the detailed and constructive feedback. We have uploaded a revised manuscript, with changes highlighted and annotated using the corresponding reviewer IDs to facilitate review. We would greatly appreciate it if you could take another look at the updated version.

We have responded to your concerns as follows:

W1, Q1 — Adaptive Anchor Placement (Sections 3.2, 4.2; Appendix B, Tables 6–7): We added controlled experiments comparing CRB with Uniform placement under identical gradient budgets and Shape+Linear scoring. CRB-Shape outperforms Uniform-Shape in 7/10 model–task comparisons and achieves a better overall average rank (1.8 vs. 3.0). Section 3.2 further clarifies the distinct roles of Coverage, Risk refinement, and Cancellation refinement. While we have not fully isolated the empirical contribution of each stage, doing so under strictly matched total gradient budgets would require additional controlled configurations and training runs beyond the current revision-time and computational resources.

W2, Q2 — Gradient-Shape Similarity (Section 3.3; Appendix C, Eqs. 9–12): We expanded the mathematical explanation of Shape Matching. Specifically, Eq. (10) shows that the integrated inner product of reconstructed gradient functions decomposes into midpoint alignment and signed slope alignment. Eq. (12) establishes that the normalized shape similarity corresponds exactly to their cosine similarity under the L2 inner product. This provides a mathematical interpretation of Shape Matching but not a guarantee of improved training updates.

Regarding checkpoint dependence, multi-checkpoint validation under our current experimental setting would require training and maintaining multiple model checkpoints, computing and storing gradient features for each, and substantially increasing the associated computational, storage, and indexing costs. A rigorous evaluation of this scale could not be completed within the rebuttal period. We therefore retain initial-checkpoint valuation as a deliberate efficiency trade-off and clarify its limitations.

W3, Q3 — Direct Reference Fine-tuning (Section 4.2; Appendix B, Tables 6–7): We added a RefT (Reference-trained) baseline that directly fine-tunes on the available target reference histories. CRB-Shape outperforms RefT in all 10 model–task comparisons, with an overall average rank of 1.8 versus 5.1. These results support the benefits of reference-guided source resampling in the evaluated settings. Dedicated analyses of short reference histories and regime changes remain outside the current experimental scope.

W3, Q4 — End-to-End Compute Budget (Section 4.2, Table 2; Appendix J): We further clarified the accounting of gradient preparation, selection, and post-training costs. We explicitly distinguish performance under matched training exposure from efficiency under matched total computation. Since a controlled equal-total-compute experiment has not been completed, we acknowledge this limitation and do not claim an end-to-end computational advantage.

Q5 — LESS Baseline Fairness (Section 4.1; Appendix I): We renamed the baseline LESS-GM (LESS-inspired Gradient Matching) and clarified its implementation and differences from the original LESS, particularly the omission of optimizer-aware gradient features and multiple checkpoints. Importantly, the forecasting models, time-series window construction, and data-resampling problem studied here differ substantially from the original setting of LESS. Consequently, there is no straightforward or established time-series counterpart that can be regarded as a faithful implementation of the original LESS pipeline. We therefore position LESS-GM as a simple gradient-matching baseline inspired by the core idea of LESS, rather than a direct adaptation or complete reproduction. We have revised the naming and comparison claims accordingly to avoid implying superiority over the full original method.

We sincerely appreciate your suggestions, which have helped us improve the theoretical explanations, experimental comparisons, and transparency of our work. We hope these revisions respond to your main concerns and would be grateful for your reconsideration.

Thank you again for your time and thoughtful feedback.

Reviewer_nSfqOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RAGE addresses allocating a fixed post-training draw budget across legal source forecast windows for TSFMs. Motivation: time-series trajectories lack semantic boundaries, so averaging gradients across a trajectory hides local variation, yet scoring every overlapping window is too costly. RAGE: (1) CRB, a coverage→interpolation-risk→endpoint-cancellation three-stage adaptive anchor placement spending ≈1.5% of legal starts; (2) reconstruct each anchor interval as a linear gradient "shape" encoding energy-normalized midpoint and slope; (3) score source blocks against reference blocks via a Linear signed shape-inner-product aggregation with trajectory-balanced reference weights; (4) map block scores to source-window sampling probabilities via a fixed-temperature exponential CDF on ranks, with weight-sharing by block size. Validation: Chronos-Base, TimesFM-2.5, TimerXL; seven tasks; two seeds; seven baselines (Rand, Orig, LESS-TS, TSDS, LP-Mirror, G-Dot, GraNd-Static); claims 2.03% mean MASE reduction over Rand (28/42 paired wins), 2.37% over LESS-TS, first on all three model aggregates.

Critical strengths
  1. The mathematical development is sound and carefully verified — the residual R_I = h_I^5 ĉ_I²/120 (Eq.5) via the β(3,3) moment, the energy factor 1/√12, the inner-product identity (Appendix C), the cancellation-score κ_I (Eq.6) handling all degenerate cases, and the rank-temperature result all check out, giving a clean, well-grounded gradient-shape motivation. This is consequential because correct, non-trivial math underpinning the scoring rule is a genuine technical contribution that, combined with a complete ablation, would support acceptance.
  2. Exceptionally honest cost reporting and a thorough ablation suite: Appendix I conservatively charges the 65.8s selection time to RAGE, Table 7 explicitly disclaims a training speedup, and the 23-config ablation (3 granularities × 4 aggregations × allocations, Tables 4–5) substantiates the Linear-aggregation conclusion — rigor beyond most resampling papers. This is consequential because it shows the authors honestly test where their method does not win, which is the main reason I rate soundness at 2 (fixable) rather than lower.
Critical weaknesses
  1. The central algorithmic novelty — adaptive CRB anchor placement — is never directly validated: there is no ablation of Shape+Linear with CRB replaced by uniformly spaced anchors at the same budget K, the apples-to-apples comparison isolating adaptivity's value. Given the strongest ablation gap is Traj-Linear (−1.39%) vs Shape-Linear (+2.03%), only ~3.4 points plausibly from local granularity rather than adaptive placement, the contribution of CRB itself is undemonstrated. This caps the score because the paper sells CRB as the novelty, yet a cheaper uniform variant may match it — the headline gain could come entirely from the shape representation rather than from the adaptive placement.
Questions for authors
  1. Please run Shape+Linear with uniformly placed anchors at the same K vs. CRB at K under identical scoring — does adaptive placement contribute beyond the shape representation? This materially determines whether the central novelty (CRB) holds.
3 comments
AuthorsOpenReview ↗

Response to Reviewer nSfq

Response to Reviewer nSfq

We sincerely thank the reviewer for the careful evaluation of our work, particularly for recognizing the mathematical derivations, experimental ablations, and computational-cost reporting. We also greatly appreciate the reviewer's suggestion to further strengthen the experimental validation of adaptive anchor placement. This provides an important opportunity to more clearly demonstrate the core contributions of RAGE.

W1, Q1: Contribution of Adaptive Anchor Placement

The original ablations compared different gradient representations and scoring methods, while a direct comparison between CRB and uniform anchor placement under the same gradient budget would provide more targeted evidence of the independent contribution of adaptive anchor placement.

Revision: To directly address this concern, we are adding experiments comparing Uniform + Shape+Linear with CRB + Shape+Linear under the same gradient budget (K). Both configurations will use identical gradient representations, scoring methods, and training settings, with anchor placement being the only difference.

This comparison will directly examine whether adaptive anchor placement provides additional benefits beyond local gradient-shape representation and help us more accurately distinguish the contributions of CRB and Shape Matching.

We will update the results as soon as the experiments are completed and explicitly report this direct comparison and its conclusions in the revised manuscript.

We sincerely thank the reviewer for this important suggestion, which enables us to validate the core methodological design more rigorously and directly.

AuthorsOpenReview ↗

Update on the Revised Manuscript

Update on the Revised Manuscript

We sincerely thank the reviewer again for the careful evaluation and constructive suggestions. We have uploaded a revised manuscript, with changes highlighted and annotated using the corresponding reviewer IDs. We would greatly appreciate it if you could review the updated version.

W1, Q1 — Independent Contributions of Adaptive Anchor Placement and Gradient-Shape Matching (Section 4.2; Appendix B, Tables 6–7):

To directly address your central concern, we conducted new controlled experiments comparing Uniform vs. CRB anchor placement and Mean vs. Shape matching under matched gradient budgets, with identical scoring and training settings.

The results show that:

  • Contribution of adaptive placement: With Shape matching fixed, CRB outperforms Uniform in 7/10 model–task comparisons; with Mean matching fixed, CRB outperforms Uniform in 9/10 comparisons.
  • Contribution of shape matching: With anchor placement fixed, Shape outperforms Mean in 14/20 comparisons.
  • Combined effectiveness: CRB-Shape achieves the best overall average rank (1.8), compared with 3.0 for Uniform-Shape.

These controlled comparisons support the independent contributions of both adaptive anchor placement and gradient-shape matching, while the combined CRB-Shape configuration achieves the strongest overall performance. Thus, the observed benefits cannot be attributed solely to the shape representation.

We sincerely appreciate your suggestion, which helped us strengthen the experimental validation of both components. We hope these results address your main concern and would be grateful for your reconsideration of the revised manuscript.

AuthorsOpenReview ↗

Follow-up on the Revised Manuscript

We would like to gently follow up on our revised manuscript. We sincerely appreciate your insightful feedback, particularly your concern regarding the independent contribution of adaptive anchor placement, which motivated us to conduct additional controlled experiments.

Regarding your main concern, we have completed the experiments directly addressing your question:

  • Direct comparison under matched settings: We conducted the requested comparison between CRB and Uniform anchor placement at the same gradient budget K, with identical Shape scoring.
  • Experimental results: Across five supplementary tasks and two models, CRB-Shape outperforms Uniform-Shape in 7/10 model–task comparisons, achieving a better overall average rank (1.8 vs. 3.0). These results provide direct empirical evidence supporting the independent contribution of adaptive anchor placement beyond gradient-shape representation.

Beyond addressing your main concern, we have also conducted additional analyses and made further revisions to improve the overall rigor, clarity, and completeness of the paper:

  • Complementary component ablations: We further compared Mean and Shape matching under matched gradient budgets. Shape outperforms Mean in 14/20 comparisons, while CRB also outperforms Uniform in 9/10 comparisons when Mean matching is fixed. These results provide additional evidence for the respective contributions of the two components.
  • Methodological clarification: We further clarified the complementary roles of adaptive anchor placement and gradient-shape matching, together with the rationale behind the three-stage CRB design.
  • Expanded experimental evaluation: We added supplementary backbone evaluations, including Chronos-2-small, and improved the presentation and interpretation of the experimental results.

We would be very grateful if you could take another look at our revised manuscript and reconsider your assessment in light of these additional experiments and improvements. We would also sincerely welcome any further comments or guidance you may have.

Thank you very much for your time and consideration.

Reviewer_rmckOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This paper proposes RAGE, a reference-guided data resampling method for time series foundation model adaptation. The authors argue that averaging gradients over entire trajectories can obscure locally useful training signals, whereas computing gradients for every overlapping forecasting window is computationally expensive. To address this issue, RAGE introduces a budgeted gradient estimation procedure that selects sparse gradient anchors using coverage, interpolation-error refinement, and gradient-cancellation criteria. It then linearly interpolates gradients between adjacent anchors, represents each block using jointly energy-normalized midpoint and slope components, and scores source blocks against reference blocks through weighted gradient-shape similarity. Finally, the resulting scores are converted into window-sampling probabilities using a rank-based allocation scheme.

Critical strengths

Relevant problem with practical value. The paper addresses the computational cost of gradient-based data selection for time series foundation model adaptation, where acquisition-defined trajectories may contain heterogeneous local patterns. Budgeted local gradient estimation is a reasonable direction to explore.

Critical weaknesses
  1. The paper is difficult to follow and the motivation is unclear. For example, the introduction reads like a collection of disconnected paragraphs rather than a coherent story. The problem is not clearly articulated and properly scoped before the paper jumps into complex formal definitions in Section 2.
  2. The methodological novelty is limited. The paper combines sparse adaptive gradient estimation, linear interpolation, midpoint–slope representations, reference-gradient matching, and rank-based resampling, but does not convincingly establish which component constitutes the central innovation or why the overall design is principled.
  3. The performance gains are not sufficiently significant. As shown in Table 1, the proposed RAGE is not the top-performing method in the majority of settings, which is insufficient to establish its effectiveness.
Questions for authors
  1. What is the fundamental methodological novelty of RAGE over existing gradient-matching and adaptive data-selection methods? Can the authors isolate a genuinely new principle beyond combining adaptive gradient interpolation, local similarity scoring, and rank-based resampling?
  2. Is there any reasons to choose Chronos, TimesFM 2.5 and TimerXL as the backbone models? Could the effectiveness be validated on more recent models such as Chronos 2 and TimesFM 3?
4 comments
AuthorsOpenReview ↗

Response to Reviewer rmck

Response to Reviewer rmck

We thank the reviewer for the constructive feedback, which has helped us improve the motivation, methodological explanations, and experimental validation.

W1: Writing and Motivation

We appreciate the reviewer's suggestion to clarify the motivation and its connection to existing methods before introducing formal definitions.

Revision: We have comprehensively revised the Introduction, improving the connections among the background, limitations of existing methods, and RAGE's motivation. We have also added intuitive explanations before the formal problem formulation and clarified key symbols and equations to improve readability. These changes will be highlighted in the revised manuscript.

W2, Q1: Methodological Novelty and Component Contributions

We appreciate the suggestion to clarify the central novelty and individual component contributions.

Core Idea: RAGE addresses the challenge that local time-series windows have different training values, while computing every window's gradient is costly. Thus, our goal is to preserve and utilize local gradient variations as much as possible under a limited gradient budget, rather than averaging entire trajectories. RAGE introduces two interconnected designs:

(1) Adaptive Gradient Estimation (CRB): CRB prioritizes intervals with high interpolation-error risks or potential gradient cancellation using observed gradients. Rather than seeking globally optimal reconstruction from sparse observations, it reduces identifiable local reconstruction risks under a fixed budget.

(2) Local Gradient-Shape Matching: Shape Matching captures both mean gradient direction and variations between anchors. Derived from linearly reconstructed gradient functions, it preserves local information without additional backward passes.

These designs determine where to compute gradients and how to use them for local matching. Together, they form a budget-constrained local data resampling method for time series.

Revision: We are adding controlled experiments comparing Uniform vs. CRB and Mean vs. Shape Matching under the same gradient budget to isolate component contributions. Results and corresponding explanations will be included in the revised manuscript.

W3: Empirical Effectiveness

We appreciate the concern about RAGE not ranking first in most individual settings.

Explanation: RAGE targets improvements across different tasks and models rather than optimizing individual benchmarks. Although not always the best in each setting, the main experiments still demonstrate overall improvements across tasks and models:

  • Compared with random sampling, RAGE reduces average MASE by 2.03% and improves 28 out of 42 paired comparisons.
  • Average improvements are positive across all 7 forecasting tasks.
  • RAGE achieves the highest aggregated average improvement on Chronos-Base, TimesFM 2.5, and TimerXL.

Individual rankings do not fully capture overall effectiveness. These results support the overall effectiveness of RAGE across different tasks and models, while also indicating that its benefits vary across specific settings.

Revision: We are updating the supplementary experimental results and plan to evaluate newer foundation models. We will clarify the distinction between aggregate improvements and individual-setting performance.

Q2: Choice of Backbone Models

We appreciate the suggestion to evaluate newer models such as Chronos 2 and TimesFM 3.

Model Selection: We chose Chronos-Base, TimesFM 2.5, and TimerXL to:

(1) Cover different architectures and assess cross-architecture applicability.

(2) Control forecasting-window settings. Specifically, all three models use a context length plus prediction length of 576, maintaining a consistent total training-window length and reducing experimental differences caused by different window sizes.

Revision: We are preparing to add experiments on Chronos 2 and will update the results. For TimesFM 3, comparable evaluation is currently constrained by computational resources and revision time, but remains a direction for future work.

AuthorsOpenReview ↗

Update on the Revised Manuscript

Update on the Revised Manuscript

We sincerely thank the reviewer again for the constructive feedback. We have uploaded a revised manuscript, with changes highlighted and annotated using the corresponding reviewer IDs to facilitate review. We would greatly appreciate it if you could take another look at the revision.

We have responded to your comments as follows:

W1 — Writing and Motivation (Sections 1–2): Please refer to the substantially revised Introduction and Problem Setup.

W2, Q1 — Methodological Novelty (Sections 3.2–3.3, 4.2; Appendix B, Tables 6–7): We have clarified the complementary roles of adaptive anchor placement (where to compute gradients) and local gradient-shape matching (how to utilize them). New controlled experiments independently compare Uniform vs. CRB and Mean vs. Shape under matched gradient budgets. CRB outperforms Uniform in 9/10 comparisons using Mean and 7/10 using Shape, while Shape outperforms Mean in 14/20 comparisons. These results provide empirical evidence for the individual contributions of both components.

W3 — Empirical Effectiveness (Section 4.2, Table 1): We have clarified the distinction between aggregate improvements and individual-task results. RAGE achieves a 2.03% average MASE reduction over random sampling, with improvements in 28/42 paired comparisons and positive average gains across all seven tasks. We also explicitly discuss settings where alternative methods perform better.

Q2 — Backbone Models (Section 4.1; Appendix B, Tables 6–7): We have clarified the backbone-selection rationale and added supplementary experiments on Chronos-2-small, alongside TimerXL 84M. CRB-Shape achieves the best average rank across these evaluations (1.8 overall). TimesFM 3.0 remains outside the current experimental scope due to computational and revision-time constraints.

We hope these revisions provide a clearer presentation and stronger experimental support for our contributions. We would be grateful if you could reconsider the revised manuscript.

Thank you again for your time and valuable feedback.

Thank you for the response and revisions. Most of my concerns are resolved. The writing in the introduction becomes much clearer and the overall quality is better. However, the experimental performance of RAGE is still not impressive. Taking everything into consideration, I decided to raise my score to 2.

Thank you very much for your further evaluation and for reconsidering your score. We sincerely appreciate your recognition of the improvements in the revised manuscript and your acknowledgment of the efforts made to address your concerns. We also understand your remaining reservations regarding the experimental performance, and will continue to take them into consideration.

Thank you again for your time and constructive feedback.

Comments on the paper

2 comments

Dear Reviewers,

Please read authors' rebuttals and confirm if your concerns are addressed well (if you have not done so). You can double check if all concern in your reviews are addressed or not, which will be the key things we need to discuss later.

Best regards,

Your AC

AuthorsOpenReview ↗

Overall Response to All Reviewers

Overall Response to All Reviewers

We sincerely thank all reviewers for their careful evaluation and constructive feedback. Their comments helped us further validate our methodology and strengthen the theoretical analysis, experiments, and presentation of RAGE. In response, we have made systematic revisions and improvements to the manuscript, summarized as follows:

  1. Further Analysis of Main Experimental Results. We have further consolidated and analyzed paired results across three foundation models (Chronos-Base, TimesFM 2.5, and TimerXL), seven tasks, and two random seeds. RAGE achieves 2.03% average and 1.32% median MASE improvements over Rand, winning in 28/42 comparisons. Against LESS-GM, it achieves a 2.37% average improvement and wins in 31/42 comparisons. (Section 4.2 and Appendices)
  2. Supplementary Controlled Experiments on Adaptive Anchor Placement. To validate CRB, we added controlled experiments on five additional forecasting tasks. Besides TimerXL 84M, we introduced Chronos-2-small as an additional, more recent time-series foundation model. Under equal gradient budgets, CRB outperforms Uniform in 7/10 comparisons with Shape fixed and 9/10 with Mean fixed, demonstrating the effectiveness of adaptive anchor selection. (Appendix B)
  3. Experimental Validation of Gradient Shape Representation and Data Allocation. We compare Shape and Mean representations in the supplementary experiments. Shape outperforms Mean in 14/20 comparisons, with CRB-Shape achieving the best overall average rank of 1.8. We further examine Shape, Block, and Trajectory representations, four aggregation strategies (Linear, Exp, ZeroClip, and Max), and two uniform allocation strategies (Block-Uni and Win-Uni). The results support the benefits of preserving local gradient information and score-based data allocation. (Section 4.2 and Appendix B)
  4. Theoretical Motivation and Applicability of Gradient Shape Matching. We strengthened the theoretical explanation by deriving the exact relationship between the integral inner product of reconstructed gradients and their midpoint–slope representations. We clarify that slope captures additional within-block alignment information, while first-order expected target-loss changes depend on mean gradients. We also discuss the distinction between shape similarity and actual single-step loss improvement. (Appendix C)
  5. Mathematical Derivations and Algorithmic Details. We supplemented the technical details of CRB, including gradient interpolation, CountSketch projection, and quadratic-approximation-based reconstruction risk. We also provide complete RAGE pseudocode and the strict-budget adaptive anchor selection algorithm, clarifying its coverage, risk-refinement, and cancellation stages. (Sections 3.2–3.5 and Appendices D–G)
  6. Baseline Configurations and Reproducibility. We refined the definitions, time-series adaptations, scoring procedures, and configurations of all baselines. In particular, we renamed LESS-TS as LESS-GM (LESS-inspired Gradient Matching) and clarified that it is inspired by, rather than a direct reproduction of, the original LESS method. We also document checkpoint identifiers, dataset configurations, model lengths, training hyperparameters, random seeds, and gradient projection details. (Section 4.1 and Appendices H–I)
  7. Computational Cost and Resource Analysis. We refined the computational cost analysis by separating shared gradient preparation, data selection, post-training, and output storage. RAGE's conservative selection time is 65.8 seconds, versus 131.8 seconds for TSDS and 178.9 seconds for LP-Mirror; its average output size is 2.54 MiB. Importantly, we distinguish between fixed post-training data exposure and equal total computational budgets, clarify the resource accounting, and discuss measurement limitations, including GPU time, FLOPs, peak memory, and cold-start latency. Strict equal-compute comparisons are left for future work. (Section 4.2, Table 2, and Appendix J)

The revised manuscript further clarifies RAGE's motivation, methodology, and contributions through additional experiments, theoretical derivations, implementation details, and resource analysis. Different text colors and margin tags highlight the revisions for easy reference.

We sincerely thank all reviewers again for their constructive feedback. These suggestions have helped us further strengthen our work and communicate the contributions, effectiveness, and applicability of RAGE more clearly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.