- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how spectral readout parameterization affects optimization in time-series forecasting. It introduces a foldable Quadrature Reparameterization and derives SpecScale-Exact, a single-kernel optimizer that reproduces the two-branch AdamW trajectory under whole-model closure. It then develops a practical Fixed-Bin update multiplier and evaluates the approach through forecasting benchmarks, trajectory checks, and interventions intended to identify the source of the gains.
Critical strengths
-
SpecScale-Exact is the strongest technical contribution. It extends function equivalence to whole-model training trajectory equivalence by accounting for Adam’s moment states, finite epsilon, matched decay, and virtual global clipping.
-
The experiments investigate the mechanism rather than relying only on benchmark accuracy. Frozen-feature controls, bidirectional rate interventions, and simple multiplier controls support the importance of readout update scale.
Critical weaknesses
-
The theoretical distinction from prior work needs sharper positioning. Section 2 characterizes RepOptimizers [1] as primarily addressing SGD, although its paper and official implementation already include AdamW, and the manuscript does not discuss the stateful optimizer trajectory equivalence in [2]. The contribution should be identified more precisely as moment-state compression for the particular coordinate structure, together with finite-epsilon and global-clipping treatment.
-
The overall $ O(1/n) $ attenuation claimed in Section 3.1 does not follow from the per-bin $ c_b/n $ factor alone, because prediction displacement also depends on spectral amplitudes, the number of contributing bins, and update directions. A fixed-amplitude cosine under an unnormalized FFT can have a spectral coefficient proportional to n, cancelling this factor.
[1] Xiaohan Ding, Honghao Chen, Xiangyu Zhang, Kaiqi Huang, Jungong Han, and Guiguang Ding. Re-parameterizing Your Optimizers rather than Architectures. ICLR, 2023. [2] Lucas Torroba-Hennigen, Hunter Lang, Han Guo, and Yoon Kim. On the Duality between Gradient Transformations and Adapters. ICML, 2025.
Questions for authors
- Relative to the AdamW implementation of [1] and the trajectory-equivalence theorem of [2], which parts of Proposition 1 are new? Please distinguish general linear reparameterization equivalence from compression to one moment pair under the specific structure used here.