- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper compares one-step and optimised latent inference on the same fixed decoder. Linear theory and nonlinear interventions are used to explain the GON--autoencoder gap.
Critical strengths
-
The within-decoder comparison separates reconstruction capacity from inference quality. Using the same decoder architectures also makes the family comparison meaningful.
-
The linear analysis explains the role of decoder coordinates. The step-size, origin, frozen-code and decoder-transfer controls strengthen the empirical result.
Critical weaknesses
- The geometric statistics in Table 1 need clarification. The strong-regularisation row reports and at (p. 7). These values do not match Equation 14 and Appendix B's full-spectrum definitions: \qquad \kappa(M)=\frac{\sigma_{\max}^2}{\sigma_{\min}^2}.$$ For ten singular values, $$\mathrm{pr}\ge\frac{40\sqrt{\kappa(M)}}{(1+\sqrt{\kappa(M)})^2}.$$ A mean $\kappa$ rounded to $1.1$ over three seeds gives $\kappa\le3\times1.15-2=1.45$ for each seed, since every $\kappa\ge1$. Thus each seed must have $\mathrm{pr}\ge9.914$, whose mean cannot round to $9.7$. Appendix I also defines conditioning over active directions.
Questions for authors
-
Does Table 1 use full-spectrum or active-direction conditioning?
-
It is unclear whether the identified effect survives in the regimes where it would matter in practice as the main claims rest largely on empirical observations from a narrow setup. This makes it difficult to tell whether the observed behavior is a general principle for popular transformer- or diffusion-based architectures.
Table 1. You are right, thank you. The "pr" column reported the stable rank ∑σ2/σ12, not the participation ratio the paper defines; the figure of spectra used the right definition. Recomputed, every pr entry is high, 8.9-10.0 out of 10 (GON 9.5, AE 8.9; the freeze row 9.5 and the unfreeze row 9.1, previously 6.2 and 4.5). The spectral contrast in the table is therefore carried by κ(M), which the text introducing the table now makes clear. The align column also changed under the correct definition (AE 0.36, GON 0.33), and we removed the sentence saying the transferred geometry drifts toward the AE regime. All changes are listed in
CORRECTIONS_LOG.mdin the revised supplementary material.Q1, which κ? Every reported κ is full-spectrum. The "active latent directions" definition in the appendix was unused and is removed; at n_z=10 the GON's κ is below the AE's in all ten dataset-architecture cells at thresholds 10−3, 10−2 and 10−1 as well.
Q2, other architectures. Our focus is the compatibility between a decoder and its one-step origin-gradient inference rule, rather than a comparison across generative architectures. The capacity–inference decomposition applies to differentiable decoders generally, and the linear theory does not depend on a particular network architecture. For the nonlinear experiments, we use MLP and convolutional decoders to investigate the proposed mechanisms under controlled conditions. Extending the experiments to transformers or diffusion models would require specifying comparable latent inference procedures and introduce additional factors beyond those we seek to isolate. We do not claim that these results generalise unchanged to these architectures, and clarify this scope in the revised limitations.