- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SPAR, a post-hoc correction module for forecasting with other channels' histories as exogenous inputs. It trains an endogenous-only forecaster (O2O), adapts it to a multivariate forecaster (M2O), and encodes both pre-head states with a shared TopK sparse autoencoder. High-shift dictionary features are selected per target channel. Phase-conditioned signed gains modify these shifts along decoder directions, and the frozen M2O prediction head maps the edits to forecast increments. An output calibrator is fitted before the sparse correction, whose coefficients are estimated with ridge regression. Evaluation on four backbones, seven datasets, and four forecast horizons reports average relative MSE reductions of 5.74% to 8.02% over Adapted M2O.
Critical strengths
-
Broad evidence of forecasting improvements. The evaluation covers four backbones, seven datasets, and four horizons. SPAR improves horizon-averaged MSE in 27 of 28 backbone-dataset combinations in Table 1, with average relative reductions of 5.74% to 8.02%. This provides useful evidence that the correction is effective across the tested architectures rather than being specific to one forecaster.
-
Ablations support a contribution beyond output calibration. Table 3 separates sparse correction from calibration. On TimeXer across four datasets, sparse correction alone reduces MSE by 7.70% on average, whereas calibration alone yields 0.59%. Together with the feature-selection, coefficient, and Raw/PCA comparisons, these results show that the representation correction contributes materially to the reported gains.
Critical weaknesses
- The framing is stronger than the demonstrated mechanism and methodological advance. I find the prominence of phase-aware regulation in the title and narrative disproportionate to the demonstrated methodological advance. The mechanism conditions signed correction gains on sinusoidal features of predefined periodic positions at the forecast origin. This is a legitimate form of phase conditioning, but the framing can suggest a more substantive treatment of phase structure in the observed signals than is implemented. The static and mismatched-phase ablations support the utility of conditioning in the tested TimeXer setting, yet do not directly connect the learned gains to phase-dependent beneficial or harmful exogenous effects. Sparsity is also genuine, arising from an existing TopK SAE and a fixed budget of selected dictionary directions. With an affine prediction head, the correction is a structured ridge regression on products of activation differences and phase features (Eqs. 4 and 8; Appendix A.4). The paper should make the incremental methodological contribution clearer and bring the emphasis on phase awareness into proportion with the demonstrated mechanism and evidence.
More importantly, the interpretation as selective regulation of exogenous effects is not sufficiently established. The O2O-to-M2O transition changes both access to other channels and model parameters; Appendix A.1 explicitly acknowledges the latter. Their representation difference therefore mixes input-access effects with adaptation-induced parameter and representation changes. A shared SAE does not separate these sources. Successful residual fitting and rollback show that the differences are predictive and editable, but do not establish that the edits specifically preserve useful exogenous contributions and suppress harmful ones. This gap affects the interpretation and significance of the central contribution.
- The empirical controls do not sufficiently distinguish SPAR from ordinary residual adaptation. The overall benchmark coverage is substantial, and the existing calibration/sparse-correction ablations, output-difference gate, and Raw/PCA comparisons are useful. However, they do not fully isolate the value of the proposed SAE intervention interface against a comparably expressive residual adapter with the same O2O/M2O information, phase features, calibration, and tuning budget. The output-difference gate uses a scalar gain per channel, whereas SPAR has separate feature-specific gains, leaving a capacity difference. The Raw/PCA budget study covers only iTransformer on two datasets, and the detailed mechanism ablations cover only TimeXer. These controls leave uncertainty about how much of the gain depends on the proposed sparse interface.
The motivating claim about beneficial and harmful external information also warrants direct controlled tests with irrelevant, redundant, or noisy exogenous channels, or changing cross-variable relationships. Standard-benchmark improvements alone do not establish that behavior. The rollback evidence has a relevant exception: in DUET on Electricity, top-shift rollback is 36.4%, slightly below active-random selection at 37.2% (Figure 6). This is a diagnostic limitation, not a claim that SPAR fails to improve DUET forecasts. Stronger controlled evidence and validation on such exceptions would materially affect my assessment.
Questions for authors
-
Critical mechanism clarification. Can the authors quantify how much of the selected O2O-M2O shift reflects exogenous access rather than continued parameter training, using an O2O continuation with a comparable training budget or an access ablation under the same frozen checkpoint? Would the feature rankings and correction gains remain similar?
-
Critical comparison. How does SPAR compare with a similarly expressive linear or low-rank residual adapter given the same O2O/M2O information, phase conditioning, calibration, and tuning budget? This comparison would help establish the incremental value of the SAE-based correction interface.
-
Critical validation of the motivating claim. Can controlled irrelevant, redundant, or noisy exogenous-input tests substantiate the claimed preservation and suppression of useful and harmful effects? Please also clarify whether Figure 1 uses the conditional decoded loss-gap statistic defined for Figures 5-7, report the eligible-channel counts, and explain the DUET rollback exception.
-
Experimental protocol clarification. Please specify the actual phase periods and the precise construction of the mismatched-phase control, and explain why the Base results in Table 2 differ from the Adapted M2O results in Table 1. These details affect the interpretation of the reported comparisons.
7 comments
Response to Reviewer uXN9
Q1. To address your concern in W1 that O2O–M2O shifts conflate cross-channel access effects with continued training, we conduct two controls: (i) matched-budget O2O continuation, retaining the O2O access mask with optimizer updates matched to the selected M2O checkpoint; and (ii) frozen-M2O access ablation, toggling cross-channel access at the same frozen M2O checkpoint.
Using the same frozen O2O-trained SAE, we decompose the original shift as:
Here, and denote activations from the original and continued O2O models; and are from the same frozen M2O checkpoint with cross-channel access disabled and enabled. The three terms correspond to the continued-training, training-access, and inference-access shifts, respectively. The latter two components constitute the access-related shift (Appendix E.1).
(1) Shift source quantification. Shapley-style allocation of squared shift energy shows that access-related components account for the majority of the original shift, both on the SPAR-selected Top-5% features and across the full SAE dictionary.
Table R1-1. Shift-energy contributions (%).
| Backbone | Continuation | Access-related |
|---|---|---|
| iTransformer | 34.23 | 65.77 |
| TimeXer | 24.72 | 75.28 |
| CrossLinear | 30.37 | 69.63 |
| DUET | 38.93 | 61.07 |
(2) Feature ranking stability. The original total-shift ranking closely matches the access-related ranking, but not the continuation ranking.
Table R1-2. Feature ranking agreement (Spearman correlation / Top-5% overlap).
| Backbone | Total vs. Access | Total vs. Continuation |
|---|---|---|
| iTransformer | 0.9919 / 83.39% | 0.3708 / 15.59% |
| TimeXer | 0.9973 / 91.24% | 0.2594 / 12.72% |
| CrossLinear | 0.9295 / 86.82% | 0.2457 / 17.34% |
| DUET | 0.9300 / 83.48% | 0.4102 / 23.56% |
(3) Correction gains. We replace the original shift with source-specific shifts while keeping all other SPAR components and fitting protocols unchanged. Fixed support uses the original Top-5% features; re-selected support independently selects Top-5% features for each shift source.
Table R1-3. Relative test MSE reductions over the corresponding C-only baseline (%).
| Backbone | Support | SPAR-total | SPAR-access | SPAR-cont |
|---|---|---|---|---|
| iTransformer | Fixed | 11.06 | 11.06 | 4.63 |
| Re-selected | 11.06 | 11.54 | 4.55 | |
| TimeXer | Fixed | 17.92 | 17.92 | 5.25 |
| Re-selected | 17.92 | 17.98 | 6.07 | |
| CrossLinear | Fixed | 14.60 | 14.98 | 5.09 |
| Re-selected | 14.60 | 15.00 | 5.57 | |
| DUET | Fixed | 9.46 | 9.71 | 2.19 |
| Re-selected | 9.46 | 10.42 | 2.93 |
Across all backbones, access-related shifts account for 61.07%–75.28% of shift energy, closely preserve the original feature rankings (Spearman: 0.9295–0.9973; Top-5% overlap: 83.39%–91.24%), and retain approximately 100%–110% of SPAR's original correction gains. In contrast, continuation-based correction retains only 23%–42%. These results show that SPAR's gains are strongly associated with cross-channel-access-related adaptation rather than generic continued training.
Response to Reviewer uXN9
Q2. This directly addresses your concern in W2 that the existing controls do not sufficiently distinguish SPAR's SAE-based intervention interface from ordinary residual adaptation.
To isolate the incremental value of the SAE-based correction interface, we implemented a phase-conditioned Linear Residual Adapter (LA) that performs corrections directly in the raw hidden representation space.
Specifically, LA selects the top-K raw hidden coordinates according to their O2O–M2O representation shifts and learns phase-conditioned signed correction coefficients along these coordinates. It uses the same O2O/M2O representations, phase features, auxiliary calibration, frozen M2O prediction head, and training/validation protocol as SPAR. Importantly, LA and SPAR have exactly the same number of trainable correction parameters (81,855 on Electricity).
As shown in the new comparison on Electricity, SPAR consistently outperforms LA across all four backbones in both MSE and MAE. The values denote relative MSE/MAE reductions for each backbone relative to its corresponding C-only baseline.
These results show that SPAR's improvements cannot be explained solely by additional correction parameters or generic linear residual fitting. More broadly, SPAR is not simply a linear corrector operating in a different feature basis. Its central design is a shift-guided sparse representation intervention mechanism: adaptation-induced shifts identify where to intervene, SAE decoder directions define structured intervention directions, and forecasting supervision learns phase-conditioned signed corrections through the frozen native prediction head. The matched comparison further demonstrates that this structured intervention provides benefits beyond directly correcting raw hidden coordinates. Together with our shift-concentration, predictive-association, selective-rollback, and phase-ablation results, these findings support the effectiveness of the proposed representation-level intervention framework.
| Backbone | Method | MSE Gain (%) ↑ | MAE Gain (%) ↑ |
|---|---|---|---|
| iTransformer | Linear Adapter | 5.70 | 2.97 |
| SPAR | 10.08 | 9.60 | |
| TimeXer | Linear Adapter | 11.44 | 9.47 |
| SPAR | 17.92 | 10.42 | |
| CrossLinear | Linear Adapter | 9.50 | 3.03 |
| SPAR | 11.92 | 12.85 | |
| DUET | Linear Adapter | 3.94 | 6.14 |
| SPAR | 7.42 | 9.63 |
Response to Reviewer uXN9
Thanks very much for your thoughtful and constructive comments. Before addressing the individual concerns, we would like to clarify the central contribution of SPAR and two points raised in W1.
Our central finding is that representation shifts induced by incorporating cross-variable information are highly concentrated in a small subset of SAE features, rather than being uniformly distributed across the representation space. These high-shift features provide an effective intervention interface, through which forecast-supervised corrections can selectively regulate cross-variable effects and improve forecasting performance.
We emphasize that the sparsity we identify is distinct from the inherent activation sparsity imposed by the TopK SAE. The former concerns the concentration of O2O–M2O adaptation shifts across dictionary features, not simply the number of active features in an individual representation or the fixed intervention budget.
Regarding phase awareness, our method explicitly conditions each feature-specific signed correction gain on sinusoidal encodings of the forecast origin's position within predefined calendar cycles (Sec. 4.2, Eq. 4). It does not claim to learn a more elaborate phase structure from observed signals. We apologize that our original presentation may have inadvertently caused some misunderstanding regarding the scope and implementation of phase awareness. To clarify this, the revised manuscript provides the actual phase periods used for each dataset in Appendix A.6 and a detailed description of the mismatched-phase control design in Appendix C.1. The revised manuscript also explicitly specifies the cycle template and dataset-specific periods.
These revisions clarify the scope of our sparsity claim and the implementation of phase conditioning. We now address the broader concerns raised in W1 and W2 through the additional analyses and comparisons below. Specifically, Q1 and Q3 provide further evidence regarding the mechanism and selective regulation of cross-variable effects; Q2 establishes the incremental value of the SAE-based intervention interface through a matched-budget residual adapter comparison; and Q4 clarifies the phase-related experimental protocols. For ease of cross-reference, we organize our detailed responses according to your four questions, which together address both weaknesses.