- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Adaptive Cross-Scale Normalisation (ACSN), a RevIN-style normalization with learnable centring coefficients constrained to (0, 1) through a sigmoid. Using a full factorial benchmark of 3200 runs (four backbones, five datasets, four horizons, four normalization modes, ten seeds), the authors examine whether this coefficient adapts at all, and whether adaptation improves forecasting. They report that the coefficient moves only slightly from its initialization but does so consistently, with movement structured by architecture and dataset. Standard full centering (fixed) outperforms the adaptive variant on all primary metrics when pooled. The authors attribute part of this gap to the sigmoid bound and propose the learned coefficient as a diagnostic of when adaptation occurs, rather than as a better normalization method.
Critical strengths
- The benchmark is complete and seed-matched across all factors, and the analysis uses paired comparisons, effect sizes, multiplicity correction, and heterogeneity estimates rather than single-run averages.
- The authors report a negative result candidly, stating that 'Fixed' remains the strongest option and that they cannot recommend ACSN as a drop-in replacement for accuracy.
- The limitations are disclosed openly. The authors acknowledge that the current evidence cannot distinguish a near-optimal initial coefficient from insufficient optimization.
Critical weaknesses
- The contribution is limited. Although the paper's intended contribution is diagnostic, the current experiments do not establish what coefficient movement reveals beyond what the direct comparison against 'Fixed' already shows.
- The key baseline is missing. Without a fixed coefficient at the initial value of 0.5, or a dense sweep of fixed values, the paper cannot show whether learning the coefficient changes forecasting accuracy at all.
- The paper supports dataset-associated coefficient movement under its training protocol, but it does not establish that the coefficients reflect dataset-specific preferred or converged centering strengths. The coefficient shares the backbone's small learning rate and a limited number of epochs, and the datasets with the fewest optimizer updates show the least movement (Table 6).
- Proposition 1 does not explain the observed gap. The learned coefficients stay between roughly 0.44 and 0.59, far from sigmoid saturation. The experiments therefore do not support attributing the gap to the excluded endpoint rather than to the coefficient remaining near its initial value.
Questions for authors
- How does Adaptive compare with a matched model whose coefficient is frozen at 0.5?
- Do coefficients move further with a larger coefficient learning rate or more updates, and do different initializations converge?
- How are PatchTST and iTransformer embedded in the multi-scale refinement framework, and how many scale-specific coefficients does each learn? Figure 7 presents scale profiles only for Autoformer-MS and Informer-MS. How is the reported coefficient aggregated across scales and checkpoints?
- What evidence attributes the observed gap to endpoint exclusion, given that Proposition 1 states the infimum equals the Fixed loss? Could a fixed-α sweep or an endpoint-capable parameterization distinguish this explanation from optimization effects?
Thank you for the detailed review. We ran the controls behind Q1 and Q2 and answer Q4 and Q6 with numbers; the dense sweep and alpha = 1 + a were not run. Results: Sec. 5 and the appendix.
MW1 / Q1 (learning rate of a; 10x / 100x; start at 2 or 4; runs moving < 0.001). The coefficient had no learning rate of its own: it sits in the backbone's AdamW group (lr 1e-4, halved each epoch). Study 2 gives its raw parameter 10x and 100x that rate, and starts it at raw about -2 and +2 (alpha = 0.12 and 0.88; raw 4 was not run). On ER the mean final alpha is 0.543 (x10) and 0.678 (x100); on IL it stays at 0.500 and 0.497. At the shared rate it ends within 0.0072 of its start in every run. Runs within 0.001 of 0.5 in Study 1: 192/800 (24.0%): Electricity 1.9%, Traffic 5.0%, Weather 5.6%, ER 26.3%, IL 81.3% (Table 2). More epochs would add little: with the rate halved each epoch, 20 epochs instead of 10 add 0.1% to the summed learning rate (Sec. 3).
Q2 (fixed-alpha sweep over 0.25, 0.5, 0.75, 0.9, 0.99, 1.0). Of these points we have 0.5 (frozen, Study 2) and 1.0 (the Fixed mode of Study 1); 0.25, 0.75, 0.9 and 0.99 were not run. The comparison your question turns on, fixed 0.5 against Adaptive, is now in the paper: pairing the 240 Study 1 Adaptive runs on ER and IL (three backbones) with the Study 2 frozen-0.5 runs, Adaptive's MSE advantage is +0.44% (90% CI -0.45 to +1.34), equivalent to zero within +-2% (TOST p = 0.0023; not within +-1%); for MAE within +-1%. So at the shared learning rate fixed 0.5 matches Adaptive to within 2% in MSE, your second branch, and the paper is organised around it (separate job arrays; Comment 1b). On ER the start matters: 0.12 gives MSE 61.9% higher than frozen 0.5 and 0.88 gives 46.2% lower. Your first branch appears in part with a larger coefficient learning rate: the coefficient moves to the better side and lowers ER MSE, relative to frozen 0.5, by 10.5% (x10) to 26.7% (x100), but only part of the better-start gain. We did not re-run Adaptive against Fixed with a boosted coefficient, so we make no claim that Adaptive matches or beats Fixed.
MW2 / Q3 (alpha = 1 + a). Not run (Limitation 3). The bound does not explain the gap in our data: alpha = 0.88 is representable and lowers ER MSE by 46.2% (40.8 to 51.1). Proposition 1 is now a remark (Sec. 3).
MW3 / Q4 (benchmark wording; absolute MSE and MAE; published numbers). The paper now says "internal ablations"; DAIN, Dish-TS and SAN are a stated limitation, not compared (Limitation 4). On 6 Nov we said we were preparing absolute Fixed-mode MSE and MAE for all cells beside published results; the paper has this only for Autoformer-MS. Table 4 (MSE) and Table 15 (MSE and MAE, appendix) compare its Fixed mode with the published Autoformer (input length 96) in 16 dataset-horizon cells: MSE is lower than published in 14 of 16 (all 12 Electricity, Traffic and Weather cells, by 3.6% to 31.1%) and MAE in 12 of 16; on ER it is lower at 96 and 720 but higher at 192 (+40.6%) and 336 (+19.6%). This is comparable, not exact: optimiser, schedule, batch size and repeats differ (Sec. 5). We did not compare PatchTST, iTransformer or Informer-MS with published numbers (published PatchTST uses look-backs of 336 or 512; ours is 96; Limitation 6). The Fixed-mode MSE and MAE of all four backbones (80 cells, ten seeds each) are in the anonymous repository (folder rebuttal_evidence/), not in the paper.
mW1 / Q5 (PatchTST and iTransformer). They were not made multi-scale: each has one scalar alpha applied once, so the cross-scale motivation does not apply to them (Table 1, Sec. 3). PatchTST applies it inside RevIN and iTransformer to the input window; both also divide by sigma, whereas Autoformer-MS and Informer-MS (one coefficient per scale: five, four on IL) only centre and restore (a correction to Eq. 1; Comment 1b).
mW2. The share within 0.001 is in the text and Table 2; the paper now says the very large d_z on IL reflects near-zero seed variance, not large adaptation.
mW3 / Q6 (correlation over the 80 cells, CI, with and without ER). Over the 80 Study 1 cells (Informer-MS included) the Pearson correlation between a cell's mean |alpha - 0.5| and its Adaptive-versus-Fixed MSE benefit is r = 0.33 (95% CI 0.15 to 0.51, cluster bootstrap over the 20 dataset-backbone clusters); Spearman 0.37. The benefit is positive when Adaptive has the lower MSE, so against the vertical axis of Figure 6 (positive favours Fixed) the sign reverses. Dropping ER strengthens it: r = 0.41 (cluster CI 0.17 to 0.63; Spearman 0.46); leaving out each dataset in turn gives r between 0.26 and 0.41. The association is moderate and observational; it does not show that movement causes a smaller gap. Without Informer-MS it was not computed (Limitation 8).