- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SpecOSS, which replaces the random initialisation of the diagonal state matrix in LinOSS with fixed frequency grids built from orthogonal bases. It evaluates on five UEA classification tasks, PPG-DaLiA heart-rate regression, WTH forecasting, PTB-XL, and small-scale language modelling, and reports gains over LinOSS and first-order SSMs.
Critical strengths
1.The paper asks an interesting question, namely how to initialise the oscillator spectrum in second-order SSMs, and the proposed change adds no extra cost to the recurrence. 2.The theory is hard to follow, but the arguments seem sound and build on each other.
Critical weaknesses
- The standard long-range benchmark is only partly reported. The paper leaves out EigenWorms, the longest dataset in the six-dataset benchmark that LinOSS uses, with no reason give. Please report EigenWorms for SpecOSS and its other baseline performance, or explain why it is left out.
- The results do not clearly back the main claim. Several SpecOSS variants in Table 2 do worse than LinOSS or fall within its variance, especially on Motor and SCP1, so it is unclear whether a structured spectrum is reliably better than a random one. Could the authors say when a structured spectrum should beat random initialisation, and why it does not in these cases?
- Forecasting is tested on one setting only. The forecasting results use a single dataset at a single horizon, where the multi-basis variant does best, but the classification results show the best variant changes from dataset to dataset. With one setting, it is unclear whether multi-basis stays the best choice for forecasting or whether the ranking shifts as it does in classification. Could the authors add more forecasting datasets and horizons, compared against LinOSS, to show whether the multi-basis advantage holds?
Questions for authors
See weakness
2 comments
I thank the authors for the detailed clarification and the additional experiments. My concerns have been addressed, and I will raise my score accordingly.
Reply to reviewer 3f7k
Thanks for the time and energy you put to review our paper. We answer each question separately:
Q The standard long-range benchmark is only partly reported. The paper leaves out EigenWorms ...
A: To address this concern we compare SpecOSS Multi FA against LinOSS with the same architecture, training budget and data splits. EigenWorms has 5 classes and sequences of 17,984 steps. As in LinOSS, each seed splits the data at random into 70/15/15 % train/validation/test. LinOSS uses A at random, as in its original formulation. We report results on three seeds and the average:
In final version of paper, we provide all other variants of SpecOSS (e.g. multi-SL or single families as well).
Q SpecOSS variants in Table 2 fall within variance of LinOSS especially on Motor and SCP1...
A: The results in Table 2 mainly favor SpecOSS (ours) over LinOSS.
Regarding SCP1, all of our reported configurations outperform both LinOSS variants, except Hermite-FA being a notable exception. Although the reported variance is relatively high, please note that matched seeds show that SpecOSS outperforms in each individual seed, even when absolute performance decreases because of initialization as we inspect each individual seed (i.e. we consistently lead in each individual seed).
Regarding Motor: The advantage of structured spectra comes from their systematic frequency coverage,unlike random initialization, basis-derived spectra allocate oscillator modes according to a known frequency distribution, reducing the risk of redundant modes or poorly covered frequency regions. Motor exp (as the reviewer pointed) shows this distinction, while the multi-basis variants are weaker, the single Hermite-FA spectrum reaches our strongest result, outperforming both LinOSS variants. One explanation is that Hermite-FA already provides an appropriate distribution of temporal scales, whereas combining additional bases divides the available state capacity among potentially less relevant frequencies. Thus, our claim is not that every structured spectrum universally outperforms random initialization, but that suitable basis-derived spectra provide a more effective starting point, with the optimal choice depending on the task. In rebuttal, we also did a simple hypothesis test by increasing the number of modes m which improved spectral resolution and model capacity at additional computational cost, giving us approximately 8% higher accuracy for Multi-FA (comparable with current Hermit-Fa performance). But, the same increase in state size did not produce comparable improvements for LinOSS (around 0.6%), further suggesting that structured frequency allocation can make more effective use of additional modes.
Q Forecasting is tested on one setting only...
A: For the rebuttal we added a second standard forecasting benchmark, ETTh1 (hourly electricity transformer data, seven channels, the standard 12/4/4-month split), at four horizons of 24, 168, 336 and 720 steps. We also evaluated WTH at the short horizon of 24 steps, in addition to the 720-step setting in the original paper (please see subsection 5.3 in rebuttal pdf). LinOSS uses the same backbone (with HPs tunning for each dataset), params, training budget and data splits are comparable with SpecOSS throughout. The table reports test MSE / MAE on standardised data (lower is better).
SpecOSS outperforms LinOSS at every horizon on both datasets, so the advantage of a structured spectrum in forecasting is not tied to one dataset or one horizon. Among our variants, the multi-basis model has the lowest error at the short horizon on both datasets. Our reading is the same as for Table 2: the reliable gain comes from replacing the random spectrum with a structured one. The multi-basis union is a safe default, and its margin over the best single basis depends on the task. Or equivalently increase the size of m as detailed in our previous answer to your query to improve the spectral resolution is a safe option.