acceptodds
Withdrawn from ICLR 2027

Factorized Initialization for Few-Step Adaptation in Time-Series Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Few-step adaptation in time-series forecasting leaves little optimization time to repair a poor start. We ask whether reusable transfer and current-support evidence should be encoded by one complete conditional update or by two explicit initialization paths. Statistical Sequence-Conditioned Initialization (SSCI) implements the latter: a learned shared displacement provides the reusable start, and a bounded support-conditioned correction specializes it before a fixed adaptation procedure. Shared-5, our reference, adapts the pretrained shared base for five support updates. Under a matched complete-update projection, factorization improves Shared-5-relative query gain over a complete conditional generator by 3.88 percentage points (pp) on External-7 and 4.59 pp on a temporal-split Monash replication. Within one jointly trained checkpoint, enabling the correction improves all six held-out real sources, with a source-equal mean of 3.44 pp. Statistics alone provide a competitive conditioning signal on External-7; the real-suite branch effect persists from zero to 20 support updates and recurs across three host architectures. The results show that explicitly separating shared transfer from support-conditioned specialization can improve the starting point supplied to a constrained optimizer.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
few-step adaptation, time-series forecasting, meta-learning, conditional initialization, task-conditioned parameter generation, sequence statistics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kithOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper examines how to parameterize the initialization for few-step adaptation in time-series forecasting. It proposes SSCI, which represents the starting parameters as a shared displacement + a bounded residual conditioned on descriptors extracted from the support data. SSCI constructs a descriptor for the residual from multiple types of information in the given support data and introduces a two-stage training procedure to learn the base forecaster and the factorized initialization.

The paper evaluates SSCI on held-out real-world and synthetic datasets through comparisons with a complete conditional parameter generator, ablation studies of residual use and descriptor configurations, and evaluations across different adaptation budgets and forecasting architectures. The results show that SSCI outperforms a complete conditional parameter generator under a matched update budget on two held-out real-data evaluation protocols.

Critical strengths
  • Clear and Practical Research Question. The paper asks whether explicitly separating a reusable shared displacement from a support-conditioned residual can improve the initialization used for few-step forecasting adaptation. 


  • Experiments probing diverse aspects of SSCI. Beyond comparing SSCI with a complete conditional generator under a matched update budget, the paper presents a range of experiments under carefully controlled settings, particularly paired residual-on/off comparisons, descriptor ablations, and evaluations across adaptation budgets. These experiments help clarify where the reported gains come from.

Critical weaknesses
  • The main positive result is obtained against complete conditional generators designed for this study. However, SSCI does not consistently outperform FOMAML. Moreover, the improvement over the independently trained Global-only initializer is marginal. Thus, the experiments establish an advantage over the particular complete-generator baselines considered in this paper, but do not yet demonstrate a clear practical benefit of the proposed conditional factorization.

  • Except for FOMAML, most of the baselines are closely related variants or ablations of SSCI. Comparisons with a broader set of established meta-initialization and conditional adaptation methods are needed to demonstrate the practical value of SSCI beyond its own design variants.

  • The paper acknowledges that a complete conditional generator is expressive enough to represent both shared and conditional components, but provides limited theoretical or diagnostic analysis of why explicitly separating them should improve optimization.

  • Several central choices, including the descriptor composition and the set of parameter tensors to modify, are manually specified. This makes it difficult to determine how these choices should be made for a new dataset and how much of the reported improvement can be attributed specifically to the proposed factorization.

Furthermore, the overall readability of the paper is poor, and several aspects of the experimental design are difficult to follow.

Questions for authors

Please carefully address each of the critical weaknesses raised above. As parameter initialization for few-step adaptation in time-series forecasting is outside my primary area of expertise, clear and convincing responses to these concerns, particularly additional evidence and analysis regarding the comparative effectiveness of SSCI, could change my overall assessment.

Reviewer_93fYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Statistical Sequence-Conditioned Initialization (SSCI), a few-step adaptation framework for time-series forecasting. It combines a learned shared initialization with a bounded, descriptor-conditioned low-rank residual before gradient-based adaptation. Experiments on synthetic and real-world forecasting sources compare SSCI with shared initialization, conditional parameter generation, and FOMAML, examining the effectiveness of factorization, descriptor conditioning, and adaptation across different backbones.

Critical strengths
  1. Separating reusable shared parameters from descriptor-conditioned residuals provides a reasonable inductive bias for few-step adaptation under limited optimization budgets.

  2. The paper includes matched-capacity hypernetwork comparisons, paired residual ablations, descriptor interventions, multiple random seeds, and cross-backbone experiments, providing useful evidence for the proposed design.

Critical weaknesses
  1. SSCI does not significantly outperform independently trained Global-only initialization (+0.98 pp, 95% CI [−1.33, 3.31]). Moreover, FOMAML achieves better relative gains on External-7 and Monash, weakening the practical advantage of the proposed method.

  2. The framework primarily combines established ideas from meta-learning, hypernetworks, low-rank parameter updates, and statistical sequence descriptors. Its conceptual distinction from existing conditional meta-learning methods is not sufficiently compelling.

  3. The primary experiments use a relatively short 48-step input and 12-step prediction horizon. The conditional initialization also introduces approximately 50% additional end-to-end computation over the residual-off baseline, without consistently outperforming strong alternatives.

  4. Figure 1 contains excessively small text and densely packed visual elements, making the framework difficult to interpret. Furthermore, results from different checkpoints, normalization schemes, and evaluation protocols are scattered across the main text and appendix, complicating direct comparisons.

  5. Figure 1 is visually overloaded, with excessively small and partially illegible text. Its infographic-style layout appears insufficiently refined for a scientific paper, and a clearer, simplified architecture diagram would substantially improve readability.

Questions for authors

See weakness.

Reviewer_bVymOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to initialize a forecasting model so that it can adapt to a new time series with only a few gradient updates. The proposed method, SSCI, adds a learned shared parameter offset and a bounded task-specific residual to a frozen pretrained model. The residual uses a low-rank parameterization and is generated from support-set sequence statistics, probe responses, and window representations. Training first learns the base model, then learns the shared offset and conditional residual by minimizing query loss before and after adaptation on the support set. The method is trained on 21 synthetic tasks and evaluated on unseen synthetic generator families and real-world time series. With identical per-tensor norm limits on the total parameter correction, SSCI outperforms a generator that produces the entire correction by 3.88% and 4.59% on External-7 and a Monash temporal-split evaluation, respectively. These gains measure the reduction in query loss relative to the Shared-5 baseline. In the jointly trained model, enabling the residual branch improves this metric by an average of 3.44% across six real-world data sources. Descriptor replacement experiments on synthetic data further show that the generated correction depends on the dynamical information contained in the descriptor. The results support modeling the shared offset and conditional residual separately, although the advantage over an independently trained shared initialization remains uncertain.

Critical strengths

S1. The paper studies how support-set information can improve the initialization of a forecasting model when only a few gradient updates are allowed. Time series differ in their trends, seasonal patterns, and variability, which motivates adjusting a shared initialization to the observed sequence. The authors further investigate whether modeling shared and sequence-specific parameter adjustments separately is more effective than generating the entire adjustment through a single network. This gives the study a clear research question and a reasonable basis for the method and experimental comparisons. S2. The experiments provide useful evidence of how sequence features affect initialization. Beyond comparing forecasting performance, the paper tests whether the sequence information in the descriptor actually matters. With the forecasting task held fixed, generating the residual from a descriptor associated with a different dynamical regime substantially increases prediction error. This remains true even when the residual magnitude is matched. These findings provide direct support for using sequence features to adjust the initialization and help explain the method’s effectiveness.

Critical weaknesses

W1. The paper evaluates the conditional residual by comparing the full model with a version in which the residual is removed. However, both use the same jointly trained parameters, and the version without the residual is not independently optimized. This comparison therefore does not establish the residual’s practical benefit over a shared initialization. In Table 2, the mean gains of SSCI and the independently trained Global-only baseline are 15.36% and 14.38%, respectively. The 95% confidence interval for their difference includes zero, and SSCI performs worse on two data sources. Table 3 also shows slightly higher gains for FOMAML on External-7 and Monash. The results support the residual’s role within the jointly trained model, but do not establish a consistent advantage over an independently trained shared initialization. W2. The paper seeks to make the comparison fair by matching the parameter counts of the conditioning modules and the norm limits on the total parameter correction. However, the methods use different rank and basis configurations, and the shared offset has a separate regularization term. The main text should clarify how these settings correspond across methods and how regularization affects the results. The main comparison also uses a fixed budget of 50 episodes per training stage, leaving unclear whether the advantage persists with more training. The experiments support the design under the reported settings, but further controls would strengthen the attribution of the improvement to the explicit decomposition itself.

Questions for authors

Q1. In Table 2, SSCI’s average improvement over the independently trained Global-only baseline is small, and the confidence interval for the difference includes zero. Could you provide results for individual training seeds to clarify whether this improvement is consistent? Given the additional cost of computing the descriptors and residuals, does SSCI retain an advantage under the same total time budget? Q2. The synthetic experiments show that using a mismatched descriptor increases prediction error. On real-world data, how does forecasting performance change when the current sequence’s descriptor is replaced with a fixed descriptor or one from another sequence? This would help establish whether the method’s gains come from using features specific to the current sequence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.