acceptodds
Under review as a conference paper at ICLR 2027

SPAR: SPARSE PHASE-AWARE ADAPTATION OF REPRESENTATIONS FOR TIME SERIES FORECASTING WITH EXOGENOUS VARIABLES

Abstract

Exogenous variables provide complementary information for time series forecasting, but redundant or unstable cross-variable dependencies can impair predictions. Existing methods primarily model or select these dependencies, with limited ability to directly regulate their predictive effects in learned representations. Using sparse autoencoders (SAEs), we find that representation shifts induced by incorporating exogenous variables concentrate in a small set of sparse features, and that intervening on these features can substantially alter forecasting behavior. Motivated by this observation, we propose Sparse Phase-aware Adaptation of Representations (SPAR), a plug-in that leverages sparse adaptation shifts as an intervention interface to selectively regulate the predictive effects of exogenous variables. SPAR learns phase-aware corrections from forecasting error, enabling beneficial cross-variable effects to be preserved while mitigating harmful ones. Experiments on four backbones demonstrate consistent improvements, with average relative MSE reductions of 5.78%, 8.02%, 6.84%, and 5.74%, respectively, across seven datasets, highlighting SPAR's strong generalizability across diverse backbones and datasets. Code is available at: https://anonymous.4open.science/r/spar-Wzcc-Xzcz.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Exogenous Variables, Sparse Autoencoders

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_uXN9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SPAR, a post-hoc correction module for forecasting with other channels' histories as exogenous inputs. It trains an endogenous-only forecaster (O2O), adapts it to a multivariate forecaster (M2O), and encodes both pre-head states with a shared TopK sparse autoencoder. High-shift dictionary features are selected per target channel. Phase-conditioned signed gains modify these shifts along decoder directions, and the frozen M2O prediction head maps the edits to forecast increments. An output calibrator is fitted before the sparse correction, whose coefficients are estimated with ridge regression. Evaluation on four backbones, seven datasets, and four forecast horizons reports average relative MSE reductions of 5.74% to 8.02% over Adapted M2O.

Critical strengths
  1. Broad evidence of forecasting improvements. The evaluation covers four backbones, seven datasets, and four horizons. SPAR improves horizon-averaged MSE in 27 of 28 backbone-dataset combinations in Table 1, with average relative reductions of 5.74% to 8.02%. This provides useful evidence that the correction is effective across the tested architectures rather than being specific to one forecaster.

  2. Ablations support a contribution beyond output calibration. Table 3 separates sparse correction from calibration. On TimeXer across four datasets, sparse correction alone reduces MSE by 7.70% on average, whereas calibration alone yields 0.59%. Together with the feature-selection, coefficient, and Raw/PCA comparisons, these results show that the representation correction contributes materially to the reported gains.

Critical weaknesses
  1. The framing is stronger than the demonstrated mechanism and methodological advance. I find the prominence of phase-aware regulation in the title and narrative disproportionate to the demonstrated methodological advance. The mechanism conditions signed correction gains on sinusoidal features of predefined periodic positions at the forecast origin. This is a legitimate form of phase conditioning, but the framing can suggest a more substantive treatment of phase structure in the observed signals than is implemented. The static and mismatched-phase ablations support the utility of conditioning in the tested TimeXer setting, yet do not directly connect the learned gains to phase-dependent beneficial or harmful exogenous effects. Sparsity is also genuine, arising from an existing TopK SAE and a fixed budget of selected dictionary directions. With an affine prediction head, the correction is a structured ridge regression on products of activation differences and phase features (Eqs. 4 and 8; Appendix A.4). The paper should make the incremental methodological contribution clearer and bring the emphasis on phase awareness into proportion with the demonstrated mechanism and evidence.

More importantly, the interpretation as selective regulation of exogenous effects is not sufficiently established. The O2O-to-M2O transition changes both access to other channels and model parameters; Appendix A.1 explicitly acknowledges the latter. Their representation difference therefore mixes input-access effects with adaptation-induced parameter and representation changes. A shared SAE does not separate these sources. Successful residual fitting and rollback show that the differences are predictive and editable, but do not establish that the edits specifically preserve useful exogenous contributions and suppress harmful ones. This gap affects the interpretation and significance of the central contribution.

  1. The empirical controls do not sufficiently distinguish SPAR from ordinary residual adaptation. The overall benchmark coverage is substantial, and the existing calibration/sparse-correction ablations, output-difference gate, and Raw/PCA comparisons are useful. However, they do not fully isolate the value of the proposed SAE intervention interface against a comparably expressive residual adapter with the same O2O/M2O information, phase features, calibration, and tuning budget. The output-difference gate uses a scalar gain per channel, whereas SPAR has separate feature-specific gains, leaving a capacity difference. The Raw/PCA budget study covers only iTransformer on two datasets, and the detailed mechanism ablations cover only TimeXer. These controls leave uncertainty about how much of the gain depends on the proposed sparse interface.

The motivating claim about beneficial and harmful external information also warrants direct controlled tests with irrelevant, redundant, or noisy exogenous channels, or changing cross-variable relationships. Standard-benchmark improvements alone do not establish that behavior. The rollback evidence has a relevant exception: in DUET on Electricity, top-shift rollback is 36.4%, slightly below active-random selection at 37.2% (Figure 6). This is a diagnostic limitation, not a claim that SPAR fails to improve DUET forecasts. Stronger controlled evidence and validation on such exceptions would materially affect my assessment.

Questions for authors
  1. Critical mechanism clarification. Can the authors quantify how much of the selected O2O-M2O shift reflects exogenous access rather than continued parameter training, using an O2O continuation with a comparable training budget or an access ablation under the same frozen checkpoint? Would the feature rankings and correction gains remain similar?

  2. Critical comparison. How does SPAR compare with a similarly expressive linear or low-rank residual adapter given the same O2O/M2O information, phase conditioning, calibration, and tuning budget? This comparison would help establish the incremental value of the SAE-based correction interface.

  3. Critical validation of the motivating claim. Can controlled irrelevant, redundant, or noisy exogenous-input tests substantiate the claimed preservation and suppression of useful and harmful effects? Please also clarify whether Figure 1 uses the conditional decoded loss-gap statistic defined for Figures 5-7, report the eligible-channel counts, and explain the DUET rollback exception.

  4. Experimental protocol clarification. Please specify the actual phase periods and the precise construction of the mismatched-phase control, and explain why the Base results in Table 2 differ from the Adapted M2O results in Table 1. These details affect the interpretation of the reported comparisons.

7 comments
AuthorsOpenReview ↗

Response to Reviewer uXN9

Thanks very much for your thoughtful and constructive comments. Before addressing the individual concerns, we would like to clarify the central contribution of SPAR and two points raised in W1.

Our central finding is that representation shifts induced by incorporating cross-variable information are highly concentrated in a small subset of SAE features, rather than being uniformly distributed across the representation space. These high-shift features provide an effective intervention interface, through which forecast-supervised corrections can selectively regulate cross-variable effects and improve forecasting performance.

We emphasize that the sparsity we identify is distinct from the inherent activation sparsity imposed by the TopK SAE. The former concerns the concentration of O2O–M2O adaptation shifts across dictionary features, not simply the number of active features in an individual representation or the fixed intervention budget.

Regarding phase awareness, our method explicitly conditions each feature-specific signed correction gain on sinusoidal encodings of the forecast origin's position within predefined calendar cycles (Sec. 4.2, Eq. 4). It does not claim to learn a more elaborate phase structure from observed signals. We apologize that our original presentation may have inadvertently caused some misunderstanding regarding the scope and implementation of phase awareness. To clarify this, the revised manuscript provides the actual phase periods used for each dataset in Appendix A.6 and a detailed description of the mismatched-phase control design in Appendix C.1. The revised manuscript also explicitly specifies the cycle template and dataset-specific periods.

These revisions clarify the scope of our sparsity claim and the implementation of phase conditioning. We now address the broader concerns raised in W1 and W2 through the additional analyses and comparisons below. Specifically, Q1 and Q3 provide further evidence regarding the mechanism and selective regulation of cross-variable effects; Q2 establishes the incremental value of the SAE-based intervention interface through a matched-budget residual adapter comparison; and Q4 clarifies the phase-related experimental protocols. For ease of cross-reference, we organize our detailed responses according to your four questions, which together address both weaknesses.

AuthorsOpenReview ↗

Response to Reviewer uXN9

Q1. To address your concern in W1 that O2O–M2O shifts conflate cross-channel access effects with continued training, we conduct two controls: (i) matched-budget O2O continuation, retaining the O2O access mask with optimizer updates matched to the selected M2O checkpoint; and (ii) frozen-M2O access ablation, toggling cross-channel access at the same frozen M2O checkpoint.

Using the same frozen O2O-trained SAE, we decompose the original shift as: Δatotal=(AC−AO)+(AM,off−AC)+(AM,on−AM,off)\Delta a_{\mathrm{total}}=(A_C-A_O)+(A_{M,\mathrm{off}}-A_C)+(A_{M,\mathrm{on}}-A_{M,\mathrm{off}})

Here, AOA_O and ACA_C denote activations from the original and continued O2O models; AM,offA_{M,\mathrm{off}} and AM,onA_{M,\mathrm{on}} are from the same frozen M2O checkpoint with cross-channel access disabled and enabled. The three terms correspond to the continued-training, training-access, and inference-access shifts, respectively. The latter two components constitute the access-related shift (Appendix E.1).

(1) Shift source quantification. Shapley-style allocation of squared shift energy shows that access-related components account for the majority of the original shift, both on the SPAR-selected Top-5% features and across the full SAE dictionary.

Table R1-1. Shift-energy contributions (%).

BackboneContinuationAccess-related
iTransformer34.2365.77
TimeXer24.7275.28
CrossLinear30.3769.63
DUET38.9361.07

(2) Feature ranking stability. The original total-shift ranking closely matches the access-related ranking, but not the continuation ranking.

Table R1-2. Feature ranking agreement (Spearman correlation / Top-5% overlap).

BackboneTotal vs. AccessTotal vs. Continuation
iTransformer0.9919 / 83.39%0.3708 / 15.59%
TimeXer0.9973 / 91.24%0.2594 / 12.72%
CrossLinear0.9295 / 86.82%0.2457 / 17.34%
DUET0.9300 / 83.48%0.4102 / 23.56%

(3) Correction gains. We replace the original shift with source-specific shifts while keeping all other SPAR components and fitting protocols unchanged. Fixed support uses the original Top-5% features; re-selected support independently selects Top-5% features for each shift source.

Table R1-3. Relative test MSE reductions over the corresponding C-only baseline (%).

BackboneSupportSPAR-totalSPAR-accessSPAR-cont
iTransformerFixed11.0611.064.63
Re-selected11.0611.544.55
TimeXerFixed17.9217.925.25
Re-selected17.9217.986.07
CrossLinearFixed14.6014.985.09
Re-selected14.6015.005.57
DUETFixed9.469.712.19
Re-selected9.4610.422.93

Across all backbones, access-related shifts account for 61.07%–75.28% of shift energy, closely preserve the original feature rankings (Spearman: 0.9295–0.9973; Top-5% overlap: 83.39%–91.24%), and retain approximately 100%–110% of SPAR's original correction gains. In contrast, continuation-based correction retains only 23%–42%. These results show that SPAR's gains are strongly associated with cross-channel-access-related adaptation rather than generic continued training.

AuthorsOpenReview ↗

Response to Reviewer uXN9

Q2. This directly addresses your concern in W2 that the existing controls do not sufficiently distinguish SPAR's SAE-based intervention interface from ordinary residual adaptation.

To isolate the incremental value of the SAE-based correction interface, we implemented a phase-conditioned Linear Residual Adapter (LA) that performs corrections directly in the raw hidden representation space.

Specifically, LA selects the top-K raw hidden coordinates according to their O2O–M2O representation shifts and learns phase-conditioned signed correction coefficients along these coordinates. It uses the same O2O/M2O representations, phase features, auxiliary calibration, frozen M2O prediction head, and training/validation protocol as SPAR. Importantly, LA and SPAR have exactly the same number of trainable correction parameters (81,855 on Electricity).

As shown in the new comparison on Electricity, SPAR consistently outperforms LA across all four backbones in both MSE and MAE. The values denote relative MSE/MAE reductions for each backbone relative to its corresponding C-only baseline.

These results show that SPAR's improvements cannot be explained solely by additional correction parameters or generic linear residual fitting. More broadly, SPAR is not simply a linear corrector operating in a different feature basis. Its central design is a shift-guided sparse representation intervention mechanism: adaptation-induced shifts identify where to intervene, SAE decoder directions define structured intervention directions, and forecasting supervision learns phase-conditioned signed corrections through the frozen native prediction head. The matched comparison further demonstrates that this structured intervention provides benefits beyond directly correcting raw hidden coordinates. Together with our shift-concentration, predictive-association, selective-rollback, and phase-ablation results, these findings support the effectiveness of the proposed representation-level intervention framework.

BackboneMethodMSE Gain (%) ↑MAE Gain (%) ↑
iTransformerLinear Adapter5.702.97
SPAR10.089.60
TimeXerLinear Adapter11.449.47
SPAR17.9210.42
CrossLinearLinear Adapter9.503.03
SPAR11.9212.85
DUETLinear Adapter3.946.14
SPAR7.429.63
Reviewer_QLugOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

SPAR studies how to regulate the predictive effects of exogenous information through sparse representation corrections. An endogenous-only forecaster (O2O) is trained first; an endogenous-exogenous forecaster (M2O) is initialized from that checkpoint and subsequently adapted with cross-variable access. A sparse autoencoder (SAE), trained on O2O states, encodes both models' representations. SPAR selects features with large training-set activation differences and learns signed, phase-dependent corrections along their decoder directions. An auxiliary output calibrator is fitted before the sparse correction. The forecasters and SAE remain frozen during correction fitting.

Table 1 reports lower horizon-averaged MSE in 27 of 28 backbone-dataset combinations, relative to Adapted M2O. The reported mean relative reductions are 5.78%, 8.02%, 6.84%, and 5.74% for iTransformer, TimeXer, CrossLinear, and DUET, respectively. These are per-backbone averages of per-dataset relative reductions, rather than reductions in one pooled MSE. The inputs contain historical exogenous observations, not future exogenous measurements (pp. 6, 16, and 19).

Critical strengths

The main strengths of this paper are listed as follows:

  1. A useful representation-level intervention mechanism. Sparse activation differences provide an explicit interface between representation analysis and forecast-supervised correction. The method is more specific than simply adding another unrestricted prediction head.
  2. Relevant ablations. Table 3 separates sparse correction from calibration and tests candidate selection, channel specificity, static versus phase-dependent gains, signed versus attenuation-only edits, and output-difference gating. Figure 3 compares SAE, raw, and PCA coordinates under matched direction budgets (pp. 7-9).
  3. Evidence across several backbones. Improvements across four architectures support usefulness beyond one implementation. Comparisons use the frozen adapted checkpoint, so SPAR's incremental gains over Adapted M2O cannot be attributed simply to additional backbone training.
  4. Transparent optimization and interface assumptions. The correction preserves the SAE reconstruction residual. With fixed normalization statistics and an affine prediction mapping, its coefficients can be fitted by ridge regression. Appendix A specifies these assumptions and implementation details (pp. 13-15).
Critical weaknesses
  1. The mechanistic interpretation should remain specific to adaptation-induced differences. The O2O-M2O difference includes both parameter updates and restored cross-variable access. Appendix A.1 explicitly acknowledges this. It is a valid operational feature for correction, but does not isolate the instantaneous effect of exogenous inputs at fixed parameters. Further controls would strengthen the interpretation without being necessary to establish that the correction improves forecasts.
  2. Selective rollback is not uniformly stronger than active-random rollback. For DUET on Electricity, Figure 6 reports 36.4% decoded loss-gap reversal for the top-shift features versus 37.2% for active-random features at the same 51-feature budget. This is a local exception, not evidence that SPAR fails overall. Moreover, the Appendix B.1 statistic is conditional on an eligible channel subset and uses a common frozen M2O head; it is not an unconditional forecasting improvement or the fraction of harmful effects removed (pp. 17-18).
  3. The relevant remaining efficiency question is cost-matched predictive value. Figure 4 already reports latency and memory and explicitly acknowledges the retained O2O branch. The missing comparison is against a similarly resourced residual corrector receiving the same histories and O2O/M2O forecasts. The existing calibration and output-difference controls are useful but do not exhaust this comparison (pp. 7-9).
  4. Robustness under changing dependencies remains uncertain. The candidate set is fixed after training, and the phase basis uses specified periods. Equation (4) also includes a constant term and input-dependent activation differences, so the method should not be described as incapable of nonperiodic correction. The narrower open question is robustness when the relevant features or phase relationships change over time.
Questions for authors
  1. What explains the DUET rollback exception? Please report variability for the top-shift versus active-random comparison and clarify how the eligible channel subset affects its interpretation.
  2. How does SPAR compare with a matched-budget residual corrector using the same histories and both original forecasts, extending the existing calibration and output-difference-gating controls?
  3. How does performance change under temporal drift in cross-variable relationships or periodic effects? Would periodically updating the candidate set help?
4 comments
AuthorsOpenReview ↗

Response to Reviewer QLug

Thanks for your careful reading and appreciating the presentation and soundness of our work. We understand your main concerns lie in the rollback exception of DUET, the lack of comparisons with the linear residual corrector under the same budget, and the robustness analysis when drifts occur in cross-variable relationships or periodic patterns. Below, we will address your concerns point by point.

Q1. What explains the DUET rollback exception?

A1. Thanks for raising this important point. Our rollback metric is conditional on channels with a sufficiently positive decoded O2O–M2O loss gap, as defined in Appendix B.1. We repeated the top-shift versus active-random comparison using 1,000 random feature-set draws. Across three seeds, top-shift achieves 36.45 ± 4.04% reversal, slightly below active-random (37.27 ± 3.83%). The eligible channels account for only 25.86%–28.35% of all 321 channels. These results should be interpreted conditionally on channels benefiting from exogenous information.

We attribute this exception to two characteristics of DUET. First, its learned channel-routing mechanism exhibits substantial channel heterogeneity: only 27.10% of target channels meet the eligibility criterion based on a sufficiently positive decoded loss gap, compared with 99.27% for iTransformer and 98.13% for TimeXer. Second, SPAR's finding that cross-variable adaptation shifts concentrate in a small subset of SAE features still holds. Although top-5% features do not consistently outperform active-random, increasing the budget to top-10% yields 89.88% rollback reversal, substantially exceeding active-random (67.20%). Together, these results suggest that DUET's heterogeneous channel dependencies require a larger SAE feature budget to effectively capture beneficial adaptation effects.

Table R1. The variability for the top-shift versus active-random comparison and number of eligible channels.

SeedTop-shiftActive-RandomTotal channelsEligible channels
202435.47%33.43%32187
202540.89%37.29%32191
202632.98%41.09%32183

Q2. How does SPAR compare with a matched-budget residual corrector using the same histories and both original forecasts, extending the existing calibration and output-difference-gating controls?

A2. Thanks for your suggestions. We report the performance of a linear residual corrector matched to SPAR’s parameters, comparing it against both SPAR and the output-gating scheme.

Table R2. Comparison of SPAR with a linear residual corrector on Electricity. Relative MSE/MAE reductions (%) on Electricity against the corresponding C-only baseline.

BackboneMethodTrainable ParamsMSE Gain (%) ↑MAE Gain (%) ↑
iTransformerOutput Gate1,6054.986.77
Linear Corrector81,8555.702.97
SPAR81,85510.089.60
TimeXerOutput Gate1,60514.086.30
Linear Corrector81,85511.449.47
SPAR81,85517.9210.42
CrossLinearOutput Gate1,6053.735.07
Linear Corrector81,8559.503.03
SPAR81,85511.9212.85
DUETOutput Gate1,6053.523.40
Linear Corrector81,8553.946.14
SPAR81,8557.429.63

As shown in Table R-2, the performance gains brought by the linear residual corrector sharing the same parameters as SPAR are lower than those of SPAR across all four backbones. Notably, the MSE gain of the Linear Corrector on TimeXer is even lower than that of the Output Gate, which has far fewer parameters (Linear Corrector MSE Gain: 11.44%, Output Gate MSE Gain: 14.08%). This suggests that the SPAR outperforms the linear residual corrector in both prediction performance and efficiency.

AuthorsOpenReview ↗

Response to Reviewer QLug

Q3. How does performance change under temporal drift in cross-variable relationships or periodic effects? Would periodically updating the candidate set help?

A3. Thanks for raising the important question of temporal robustness and candidate-set adaptation. We conducted two controlled evaluations with iTransformer.

A3-1. Gradual degradation of cross-variable input reliability. On Electricity, we gradually degraded the reliability of 160 out of 321 exogenous channels by adding Gaussian noise, while preserving target histories and labels. Specifically, we controlled input reliability using a signal-retention coefficient ρ\rho, where the perturbed standardized input is x~=ρx+1−ρ2ϵ\widetilde{x}=\rho x+\sqrt{1-\rho^2}\epsilon. Thus, ρ=1\rho=1 represents clean inputs, while decreasing ρ\rho indicates increasingly noisy inputs. As shown in Table R3-1, Fixed SPAR maintained a positive overall relative MSE reduction of 1.589 ± 0.767% over the correspondingly perturbed Adapted M2O, compared with 3.092 ± 0.419% under clean inputs. This demonstrates that SPAR retains predictive benefits even as exogenous-input reliability gradually deteriorates. Furthermore, Refresh-SPAR improved the overall relative MSE reduction over coefficient-only Rolling-SPAR from 0.707% to 1.224%. This provides evidence for the potential benefit of periodically updating the candidate set in adapting to changing cross-variable input reliability.

Table R3-1. Performance under gradual exogenous-input reliability degradation.

SettingMean signal retention (ρ\rho)Fixed SPARRolling SPARRefresh SPAR
Clean reference1.00003.092 ± 0.419%----
Noisy (Overall)1.0 → 0.11.589 ± 0.767%0.707 ± 2.959%1.224 ± 1.502%

A3-2. Temporal drift in periodic effects. DoTime is a benchmark of synthetically generated multivariate time series with known structural mechanisms, providing controlled observational trajectories for studying temporal dependencies. Using its seasonal observational trajectories, we introduced phase shifts of π/4\pi/4 and π/2\pi/2, as well as a 20% increase in the effective period of an observed seasonal input, while keeping all target observations and target histories unchanged. As shown in Table R3-2, Fixed SPAR consistently exhibits smaller drift-induced MSE degradation than M2O under all three perturbations. This indicates that SPAR retains more stable predictive performance under temporal drift in periodic exogenous effects, even with a fixed candidate set.

Table R3-2. Robustness under periodic-effect drift.

DriftM2O degradationSPAR-Fixed degradationDifference
Phase shift (π/4\pi/4)0.0035400.0010320.002508
Phase shift (π/2\pi/2)0.0082260.0024370.005789
Period increase (+20%)0.0055260.0022170.003309
AuthorsOpenReview ↗

Response to Reviewer QLug

W1. The mechanistic interpretation should remain specific to adaptation-induced differences.

A4. Thanks for pointing this out! We disentangle continued-training effects from cross-channel access effects with two controls: (i) matched-budget O2O continuation, initialized from the same O2O checkpoint and retaining the O2O access mask, with optimizer updates matched to the corresponding selected M2O checkpoint for each seed; and (ii) frozen-M2O access ablation, toggling the access mask at the same frozen M2O checkpoint.

Using the same frozen O2O-trained SAE, we decompose the original shift into three components: Δatotal=(AC−AO)+(AM,off−AC)+(AM,on−AM,off)\Delta a_{\mathrm{total}}=(A_C-A_O)+(A_{M,\mathrm{off}}-A_C)+(A_{M,\mathrm{on}}-A_{M,\mathrm{off}})

Here, AOA_O and ACA_C denote the SAE activations of the original O2O and matched-budget O2O-continuation models, respectively. AM,offA_{M,\mathrm{off}} and AM,onA_{M,\mathrm{on}} are obtained from the same frozen M2O checkpoint with cross-channel access disabled and enabled, respectively. The three terms correspond to the continued-training, training-access, and inference-access shifts, respectively. The latter two components constitute the access-related shift. Detailed protocols are provided in the revised Appendix E.1.

(1) Shift source quantification. We use Shapley-style allocation of squared shift energy to account for interactions among components. As shown in Table R4-1, on the original SPAR-selected Top-5% Shift features, access-related components account for more total shift energy, compared with generic continuation. The same trend holds over the full SAE dictionary(Appendix E).

Table R4-1. The results of shift source quantification.

BackboneContinuationAccess-related
iTransformer34.23%65.77%
TimeXer24.72%75.28%
CrossLinear30.37%69.63%
DUET38.93%61.07%

(2) Feature ranking stability. As shown in Table R4-2, the original total-shift ranking aligns much more closely with the access-related ranking than with the continuation ranking.

Table R4-2. The results of feature ranking stability.

BackboneTotal vs Access<br>Spearman / Top-5% OverlapTotal vs Continuation<br>Spearman / Top-5% Overlap
iTransformer0.99190.9919 / 83.39%83.39\%0.37080.3708 / 15.59%15.59\%
TimeXer0.99730.9973 / 91.24%91.24\%0.25940.2594 / 12.72%12.72\%
CrossLinear0.92950.9295 / 86.82%86.82\%0.24570.2457 / 17.34%17.34\%
DUET0.93000.9300 / 83.48%83.48\%0.41020.4102 / 23.56%23.56\%

(3) Correction gains. We replace the original shift with source-specific shifts while keeping the remaining SPAR components and fitting protocol unchanged. Fixed-support uses the original Top-5% features; re-selected-support independently selects Top-5% features for each shift source. We report the relative test MSE reductions for the three variants under both support protocols:

Table R4-3. The results of correction gains. Values are relative MSE reductions (%) against the corresponding C-only baseline for each backbone.

BackboneSupportSPAR-total(%)SPAR-access(%)SPAR-cont(%)
iTransformerFixed11.0611.0611.0611.064.634.63
Re-selected11.0611.0611.5411.544.554.55
TimeXerFixed17.9217.9217.9217.925.255.25
Re-selected17.9217.9217.9817.986.076.07
CrossLinearFixed14.6014.6014.9814.985.095.09
Re-selected14.6014.6015.0015.005.575.57
DUETFixed9.469.469.719.712.192.19
Re-selected9.469.4610.4210.422.932.93

As shown in Table R4-3, SPAR-access consistently matches or yields numerically higher gains than SPAR-total across all four backbones under both protocols. In contrast, SPAR-cont yields substantially smaller gains.

In summary, access-related shifts account for the majority of the original shift energy, preserve the original feature rankings, and retain essentially all of SPAR's correction gains. In contrast, generic continuation produces substantially different rankings and much smaller correction gains. These results demonstrate that SPAR's gains are not primarily attributable to generic continued training, but are strongly associated with cross-channel-access-related adaptation.

We hope these replies address your concerns.

Thank you for your rebuttal. All of my concerns are addressed.

I will raise my score.


Update: I have raised my rating to

  • Rating: 4: Clear acceptance
  • Confidence: 4: I have substantial direct research experience on this topic.
Reviewer_p5RUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue that incorporating exogenous variables does not necessarily improve multivariate time series forecasting performance. It proposes SPAR, which uses a shared SAE feature space to identify representation changes induced by adapting from O2O to M2O, and employs phase-aware signed correction and auxiliary output calibration to improve forecasting performance.

Critical strengths
  1. Practical significance of the research problem: The paper investigates the potential adverse effects of exogenous variables from the perspective of internal model representations, offering a new direction for multivariate time series forecasting beyond input selection and backbone architecture design.
  2. Novelty in method design: The paper leverages a shared Top-K SAE feature space to compare representation differences between O2O and M2O, combined with phase-aware signed correction to adjust candidate features in a targeted manner.
  3. Broad experimental coverage: The paper conducts experiments on seven datasets and four backbone models. SPAR reduces the average MSE in 27 out of 28 backbone–dataset combinations, demonstrating strong overall effectiveness.
  4. Reasonably convincing comparative experiments and systematic ablation studies: The paper evaluates the effectiveness of sparse representation correction and its key design components through experiments that remove the calibration module, replace the candidate feature selection strategy, and simplify the phase-aware and signed coefficients.
Critical weaknesses
  1. Lack of simpler correction baselines: Further experiments are needed to determine whether SPAR outperforms a linear residual corrector with a comparable number of parameters, trained on frozen M2O hidden states. Such a comparison would help establish the necessity of SPAR's more complex design.

  2. Limited baseline coverage: The paper compares SPAR with a relatively limited number of baseline methods, raising questions about the general applicability of the proposed plug-in module. Evaluations against a broader range of relevant methods would help better establish SPAR's competitiveness and generalizability.

  3. Insufficient analysis of computational overhead and efficiency trade-offs: Although the paper reports inference latency and peak GPU memory usage under specific experimental configurations, the relationship between forecasting improvements and additional computational costs is not sufficiently analyzed. A more comprehensive assessment would help clarify the practical efficiency of SPAR.

Questions for authors
  1. Could the authors further verify whether SPAR outperforms a simple linear residual corrector with comparable parameter count, which is trained upon the frozen M2O hidden states? This would help justify the necessity of SPAR’s elaborate overall design.

  2. The two‑stage training pipeline of SPAR brings additional computational overhead. How do the authors analyze the trade‑off between forecasting performance gains and the extra computational cost introduced by SPAR?

  3. This work uses O2O representations as a baseline and computes Δa\Delta a as the difference between M2O and O2O representations, based on which SPAR’s correction terms are constructed. Since the O2O model does not use any exogenous variables at all, one concern arises: does this setup mean SPAR essentially only performs calibration against the endogenous baseline and fails to genuinely leverage information from exogenous variables? Could the authors clarify the following three points:

  1. Does SPAR still require exogenous variable inputs during inference?
  2. Is Δa\Delta a computed from raw time‑series data, or from the latent activations inside the model?
  3. What mechanism prevents SPAR from learning a solution that fully rolls back to O2O and discards all useful predictive signals brought by exogenous variables?

===Pure English block for ICLR input box===

  1. Could the authors further verify whether SPAR outperforms a simple linear residual corrector with comparable parameter count, which is trained upon the frozen M2O hidden states? This would help justify the necessity of SPAR’s elaborate overall design.

  2. The two‑stage training pipeline of SPAR brings additional computational overhead. How do the authors analyze the trade‑off between forecasting performance gains and the extra computational cost introduced by SPAR?

  3. This work uses O2O representations as a baseline and computes Δa\Delta a as the difference between M2O and O2O representations, based on which SPAR’s correction terms are constructed. Since the O2O model does not use any exogenous variables at all, one concern arises: does this setup mean SPAR essentially only performs calibration against the endogenous baseline and fails to genuinely leverage information from exogenous variables? Could the authors clarify the following three points:

  1. Does SPAR still require exogenous variable inputs during inference?
  2. Is Δa\Delta a computed from raw time‑series data, or from the latent activations inside the model?
  3. What mechanism prevents SPAR from learning a solution that fully rolls back to O2O and discards all useful predictive signals brought by exogenous variables?
2 comments
AuthorsOpenReview ↗

Response to Reviewer p5RU

We sincerely thank you for your thoughtful and constructive review. In response, we added a parameter-matched residual baseline, profiled the computational costs, and further clarified how SPAR leverages exogenous information. We address your concerns below.

Q1. Could SPAR outperform a simple linear residual corrector with a comparable parameter count, trained on frozen M2O hidden states?

A1. Thanks for your suggestions. We added a parameter-matched Linear Residual Adapter trained on frozen M2O hidden states. Both methods use the same representations, phase features, calibration, prediction head, data split, and tuning protocol, with 81,855 trainable parameters each.

Table R1. Comparison of SPAR with a linear residual adapter on Electricity. Relative MSE/MAE reductions (%) against the corresponding C-only baseline.

BackboneMethodMSE Gain (%) ↑MAE Gain (%) ↑
iTransformerLinear Adapter5.702.97
SPAR10.089.60
TimeXerLinear Adapter11.449.47
SPAR17.9210.42
CrossLinearLinear Adapter9.503.03
SPAR11.9212.85
DUETLinear Adapter3.946.14
SPAR7.429.63

As shown above, SPAR outperforms the matched adapter on both metrics across all four backbones, supporting the additional value of its shift-guided sparse correction.

Q2. How do the authors analyze the trade-off between forecasting performance gains and the additional computational cost introduced by SPAR?

A2. Thanks for your suggestions. We profiled all five training stages on iTransformer/Electricity. The complete SPAR pipeline takes 472.79 s (7.88 min; 0.131 GPU-hours), compared with 337.80 s (5.63 min; 0.094 GPU-hours) for Adapted M2O. The incremental SPAR cost (SAE, calibration, and correction fitting) is approximately 135 s (2.25 min; 0.037 GPU-hours), or 40% over Adapted M2O. SAE training accounts for 125.35 s, while calibration and correction fitting take only 9.63 s combined. In the same setting, SPAR reduces MSE from 0.1521 to 0.1336 (12.2%). Thus, SPAR trades approximately 40% more offline training time for a 12.2% MSE reduction. At inference, SPAR incurs a latency of 2.938 ms/window and peak memory usage of 20.85 MiB, compared with 0.564 ms/window and 11.41 MiB for M2O (Figure 4).

Q3. Since O2O does not use exogenous variables, does SPAR merely calibrate against an endogenous baseline rather than genuinely leveraging exogenous information?

A3. Thanks for raising this concern. SPAR does not merely calibrate against the endogenous-only baseline. We clarify the following three points.

Q3-1. Does SPAR still require exogenous inputs during inference?

A3-1. Yes. SPAR still requires historical exogenous observations at inference.

Q3-2. Is Δa computed from raw time-series data or latent activations?

A3-2. It is computed from latent SAE activations, not raw time-series values.

Q3-3. What prevents SPAR from fully rolling back to O2O and discarding useful exogenous information?

A3-3. SPAR does not impose an explicit constraint against fully restoring selected SAE coefficients to their O2O values. Instead, corrections are learned from forecasting errors, rather than by minimizing the distance between O2O and M2O representations. Importantly, SPAR edits only selected SAE decoder directions, while leaving unselected coefficient coordinates and the original SAE reconstruction residual unchanged. Thus, restoring selected coefficients does not replace the entire M2O representation with its O2O counterpart.

Specifically, SPAR applies a signed correction along the selected decoder directions:

\alpha\sum_{j\in S_c}D_j\Delta a_{c,j}g_{c,j}(t), \quad g_{c,j}(t)=\phi(t)^\top\theta_{c,j}.$$ The effective coefficient $\alpha g_{c,j}(t)$ determines how each selected shift is regulated: 0 retains the M2O shift; values in $(-1,0)$ attenuate it; -1 restores the selected coefficient to its O2O value; and positive values amplify it. These coefficients are learned by ridge regression against forecasting residuals, with validation selecting the regularization and overall shrinkage, including $\alpha=0$ (no sparse correction). Therefore, SPAR does not guarantee that every useful exogenous contribution is retained; rather, it uses forecasting supervision to determine which selected representation shifts should be adjusted and by how much. We hope these replies address your concerns.
AuthorsOpenReview ↗

Your invaluable input is needed

Dear Reviewers p5RU,

We have carefully prepared responses to address your comments and questions. We would greatly appreciate it if you could take another look at our responses and let us know if you have any further concerns or questions. Your feedback is invaluable to us, and we sincerely appreciate your time and consideration.

Thank you again!

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

We sincerely appreciate the tremendous efforts of the Program Chairs, Senior Area Chairs, and especially Area Chairs in coordinating the review process. We also extend our sincere thanks to all Reviewers for their constructive and detailed reviews.

We provide the following summary of the discussion phase to assist the Area Chair in tracking the progress of our rebuttal.

Main contributions. We propose SPAR, a representation-level intervention framework for time series forecasting with exogenous variables, addressing the challenge that exogenous variables can introduce both beneficial and harmful predictive effects. SPAR identifies sparse O2O–M2O representation shifts using a shared TopK SAE and selectively regulates them through phase-conditioned signed corrections. Experiments across seven datasets and four backbones demonstrate consistent improvements, with average relative MSE reductions of 5.78%, 8.02%, 6.84%, and 5.74% for the four backbones, respectively.

Strengths acknowledged by reviewers. All three reviewers recognized the practical significance of the problem, the novelty of the representation-level intervention mechanism, the broad experimental coverage, and the systematic ablation studies.

Additional experiments. The primary concerns centered on mechanism validation, comparisons with simpler correction methods, and robustness. In response, we conducted the following additional experiments:

  1. Shift source quantification. Disentangled continued-training effects from cross-variable access effects. Access-related shifts account for the majority of representation shifts, and access-based corrections retain essentially all of SPAR's original gains (uXN9, QLug).
  2. Residual correction baselines. Added a parameter-matched Linear Residual Adapter and an output-difference gating baseline. SPAR outperforms the matched adapter in both MSE and MAE across all four backbones (uXN9, QLug, p5RU).
  3. Controlled exogenous-input evaluation. Using DoTime, tested useful, irrelevant, redundant, and noisy inputs. SPAR achieves 225% useful-gain retention and 94.12% to 100% harmful-input suppression, supporting its central motivation (uXN9).
  4. DUET rollback analysis. Reported three-seed variability, clarified eligible-channel counts, and showed that increasing the feature budget improves DUET rollback reversal beyond active-random selection (uXN9, QLug).
  5. Efficiency analysis. Profiled the complete training pipeline, quantifying approximately 40% additional offline training time alongside a 12.2% MSE reduction on iTransformer/Electricity (p5RU, QLug).
  6. Temporal robustness. Evaluated degraded exogenous-input reliability and controlled periodic-effect drift. Fixed SPAR consistently maintains predictive benefits under the tested perturbations (QLug).

Additional clarifications. We also provided clarifications addressing the remaining methodological and implementation concerns:

  1. Specified the actual phase periods and the construction of the mismatched-phase control (uXN9).
  2. Explained the different reference models used in Tables 1 and 2 (uXN9).
  3. Clarified that SPAR requires historical exogenous observations at inference and computes representation shifts from latent SAE activations (p5RU).
  4. Explained how forecast-supervised correction regulates selected representation shifts without imposing a complete rollback to O2O (p5RU).
  5. Clarified the conditional nature of the rollback statistic and the interpretation of eligible channels (uXN9, QLug).

Outcome & Score. The additional experiments and clarifications received positive feedback from Reviewer QLug:

  1. Nov 15: Reviewer QLug stated, "All of my concerns are addressed," and raised the rating from 3 to 4.
  2. As of Nov 19: Reviewers p5RU and uXN9 have not provided further responses following our rebuttals. Their original ratings were 3 and 2, respectively.

Overall, we believe the additional experiments and clarifications have substantially strengthened the empirical evidence and addressed the major concerns raised during the review process.

We sincerely thank the Program Chairs, Senior Area Chairs, and Area Chairs for their time and dedication to the research community.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.