- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper builds on Deep Spectral Kernel Networks (DSKNs; Xue et al., 2019), which build nonstationary representations from paired-frequency feature maps. It makes three contributions:
Phase analysis. In the single-coordinate stochastic phase map of DSKN, the two phases of a frequency pair are sampled independently. The paper shows that this makes the cross-frequency terms vanish in expectation, so the expected kernel at initialization is stationary. Sharing one phase per frequency pair recovers the full harmonizable integrand. DOSKN. The paper extends DSKNs to vector-valued outputs. Each spectral component is paired with a positive semidefinite (PSD) output-coupling operator. Spectral components are grouped by frequency-norm quantiles, and each group shares an orthonormal r-dimensional output subspace. This places a block-rank constraint on the output weights. The construction also decouples the number of frequencies from the layer width. Regularizer. Optimizing the component amplitudes under a trace budget gives a squared group-norm regularizer Ω, for which the paper derives an exact O(M log M) proximal operator.
Critical strengths
The phase analysis is correct and clearly checked. The derivation that independent phases make the cross-frequency terms vanish in expectation, while a shared phase recovers them, is short and correct (App. A.1–A.2). Table 2 confirms it numerically: with a shared phase, the kernel error decreases as M grows, while with independent phases it stays at a constant level.
The work is reproducible. The code is provided in the supplementary material. The proximal operator is checked numerically against bisection and by random perturbation tests.
Critical weaknesses
- Limited novelty, and the comparisons leave out the relevant baselines
Each component builds closely on existing work, and what the authors add on top is small:
The phase result is a two-line trigonometric identity. The grouped low-rank output coupling is a form of the low-rank and coregionalization structure that is standard in multi-output kernel learning (e.g., the Linear Coregionalization Model, which the paper itself reviews in Sec. 2). The squared group-norm regularizer and its link to multiple kernel learning come from Micchelli & Pontil (2005) and Bach (2008), as the authors acknowledge (L283–285). Its proximal operator is computed with a standard sort-and-threshold procedure.
The baselines are an MLP, a stationary random Fourier features (RFF) network, and two variants of the 2019 DSKN. The comparison includes no multi-output or operator-valued kernel method, and no more recent nonstationary or deep kernel approach. This is so even though contribution (5) claims parameter savings "relative to … operator-valued baselines", and the paper discusses Brault et al. (2016), multi-output Gaussian processes (MOGPs), and deep kernel learning in its related work.
- Presentation and experimental scale No figures. The paper has no figures at all. The paired-frequency construction, the grouped subspace coupling and the layer structure are all explained only through equations and Table 1. One or two diagrams of the architecture and the coupling geometry would make the paper much easier to follow. Unresolved citation and typos. The citation at L51 is unresolved ("(?)"), "SARCOS" is misspelled as "SACROS" (L407), and App. B.4 contains errors such as "accross" and "while while" (L854–856). Small experiments. The real-data experiments use small datasets: two regression tasks, and three multi-label tasks with as few as 593 samples. The synthetic coupling experiment generates data from exactly the block-rank structure the model assumes (Q* = 4, r* = 3), which favours the proposed method.
Questions for authors
- The central correction brings no measurable benefit in the experiments
On SARCOS (Table 5), the original DSKN with independent phases (0.0912 ± 0.0014) is slightly better than the corrected DSKN (0.0916 ± 0.0011) and better than DOSKN (0.0923 ± 0.0002). On Parkinson's Telemonitoring, the corrected DSKN (1.2900) is worse than the original (1.2466).
- The parameter savings are much smaller than claimed
The headline savings ("cuts coupling parameters by up to 76%", Abstract; 52.5% on SARCOS, Table 5) count only the output-coupling parameters. Total model size is dominated by the frequency parameters, which number 2·M·D_{l−1} per layer (L248–252). For example, in Table 6 the two-layer model has 2,879 coupling parameters out of 48,831 in total, about 6%. Even a large cut in coupling parameters therefore changes the overall model size very little. The paper should report total parameter counts next to every claimed reduction.
To be honest, I believe the presentation of this work needs to be revised to make it easier for readers to understand. At the very least, I found it extremely difficult to follow. Of course, if I have misunderstood anything during the review process, I am open to your rebuttal and willing to adjust my score.