- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a nonlinear output representation method adapting to inverse scattering problems. In the proposed method, a VAE is pre-trained where its encoder predicts a vector latent space from the 2-D spatial information. In neural network training, the decoder is adopted and its weights are frozen. Direct pixel, PCA, and the proposed method are compared, where the backbone architecture is unchanged, and the proposed method scored the best in prediction accuracy.
Critical strengths
a The empirical evidence that output representation matters in neural operators is strong, demonstrating the effectiveness of replacing PCA with nonlinear latent decoding. Many ablation studies are performed, e.g. head size, pretraining, decoder training (Table 3), and multi-seed experiments make the conclusions more rigorous.
b. The proposed head can be attached to any backbone, and this method raises attention for output representations independent of the neural network architecture. The authors’ central claim is thus made clear: the choice of output parameterization is an important design decision.
Critical weaknesses
a. The mechanism of the proposed gain over the spatial head is not very clearly demonstrated in theoretical level. Besides, the scratch head experiment (randomly initialized decoder) still beats the baseline, and the local feature map experiment (removing global vector) performs worse than the baseline. It suggests that global mixing and decoder structure might be the reason (the authors acknowledged in Section 6.2). Therefore, more explanations are required to attribute the results to the nonlinear output representation.
b. The pretrained VAE may bring out-of-distribution issues, and the paper did not evaluate generalization beyond the training target distribution. The proposed pretrained decoder encodes a prior on the target distribution directly in the output space, so targets that this decoder cannot represent cannot be reconstructed, while the spatial representation baseline does not have such issue.
Questions for authors
a. How do the authors distinguish the effect of the nonlinear output representation from global mixing? E.g. Can spatial head with global mixing close the performance gap? (weakness 1)
b. Please demonstrate out-of-distribution generalizability for the proposed method, e.g. training on Shapes and testing on shapes with holes, and report the proposed method comparing with spatial representation? (weakness 2)
c. Please provide analysis on how the VAE-based output representation helps in training, or explain the mechanism in more detail, e.g. which types of nonlinear representations work, and which do not? Detailed analysis of the explainability of the latent space would be preferred.
2 comments
I thank the authors for the extensive experimental results, which addressed my questions, e.g. OOD concern . However, I did not see in-depth discussion to distinguish the proposed method from established latent-decoding strategies. Therefore, I keep my score.
Response to Reviewer uCBn
Thank you for three precise and constructive questions. Each proposed a specific experiment, we ran all three, and they are now Appendix G.5 (global mixing) and Section 6.3 (shapes with holes).
(a) Global mixing. This was an excellent suggestion. We built spatial heads with global Fourier and attention mixing, matched within 0.5% in parameters (Table 31). Fourier mixing closes about half of the gap (41 to 53%), so global routing is indeed part of the mechanism, as you anticipated. The latent head still wins on 10/10 seeds, so routing does not explain the rest. Attention mixing does not help.
(b) Out-of-distribution targets. We found this to be a very strong suggestion, and we thank you for laying out your proposed experiment. We ran it: train on Shapes, test on shapes with holes. The latent head with a frozen pretrained decoder generalizes far better than the spatial head. Its error is 60 to 71% lower on annuli, holed shapes, nested shapes and polygons (10/10 seeds), and it is ahead on every unseen family for all five backbones at 12,800 pairs (Section 6.3).
You also raised the concern that "targets that this decoder cannot represent cannot be reconstructed". We measured this directly, and in this experiment it does not arise. The decoder, trained only on solid shapes, reproduces shapes with holes almost exactly (NMSE 0.014 to 0.059 when its code is fitted to the true target). The spatial head can represent any image, yet its error on these targets is 2.5 to 3.4 times the latent head's.
We then looked for a case where the concern does apply, and found one on measured data: a nested pair of cylinders, which a prior trained on separate cylinders cannot represent. Even there the latent head beats the spatial head (lower NMSE on 4 of 5 cells, higher SSIM on 14 of 15 pairs). A frozen linear PCA basis does better still, so we now state this as a limitation, together with a simple check that needs only target samples, and that Guo et al. (2022) also recommend: encode and decode them.
(c) Which nonlinear representations work, and which do not. Thank you for asking for this; the discussion now states the answer directly (Section 8).
Presentation. Thank you for the feedback on clarity. Figure 6 adds a diagram of the three heads with tensor shapes.