acceptodds
Under review as a conference paper at ICLR 2027

Beyond Spatial Prediction: Nonlinear Output Representations for Inverse Scattering

Abstract

Inverse scattering problems represent a severely ill-posed regime where the spatially varying contrast of a medium is recovered from measurements of the scattered wavefield. Typically, machine-learning approaches output reconstructions in a discretized spatial parametrization, where the neural networks predict the contrast at each grid point. In this paper, we study this output choice and build a modular head which reconstructs the contrast field through a learned, lower-dimensional nonlinear parametrization, replacing spatial parametrizations. The network therefore predicts latent coordinates, whose space carries our prior on the structure of our contrast fields. Across three target distributions, three noise levels, and multiple seeds, pretrained latent decoding reduces normalized mean squared error by an average of 34% relative to the spatial-output variants of five leading neural-operator architectures. The gain holds in fairly matched studies, including controls on parameter counts, MACs, and training recipes. In one setting, we observe that a spatial network with the trainable parameters needs a longer training schedule just to reach parity with a nonlinear latent head. We also study the impact of nonlinearity, finding the largest gain over linear latents for sharp contrast fields and less for smooth low-rank fields. Our results show that nonlinear latents serve as a strong representation space for learned inverse scattering methods.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
inverse scattering, inverse problems, neural operators, operator learning, nonlinear output representations, r epresentation learning, nonlinear approximation, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uCBnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a nonlinear output representation method adapting to inverse scattering problems. In the proposed method, a VAE is pre-trained where its encoder predicts a vector latent space from the 2-D spatial information. In neural network training, the decoder is adopted and its weights are frozen. Direct pixel, PCA, and the proposed method are compared, where the backbone architecture is unchanged, and the proposed method scored the best in prediction accuracy.

Critical strengths

a The empirical evidence that output representation matters in neural operators is strong, demonstrating the effectiveness of replacing PCA with nonlinear latent decoding. Many ablation studies are performed, e.g. head size, pretraining, decoder training (Table 3), and multi-seed experiments make the conclusions more rigorous.

b. The proposed head can be attached to any backbone, and this method raises attention for output representations independent of the neural network architecture. The authors’ central claim is thus made clear: the choice of output parameterization is an important design decision.

Critical weaknesses

a. The mechanism of the proposed gain over the spatial head is not very clearly demonstrated in theoretical level. Besides, the scratch head experiment (randomly initialized decoder) still beats the baseline, and the local feature map experiment (removing global vector) performs worse than the baseline. It suggests that global mixing and decoder structure might be the reason (the authors acknowledged in Section 6.2). Therefore, more explanations are required to attribute the results to the nonlinear output representation.

b. The pretrained VAE may bring out-of-distribution issues, and the paper did not evaluate generalization beyond the training target distribution. The proposed pretrained decoder encodes a prior on the target distribution directly in the output space, so targets that this decoder cannot represent cannot be reconstructed, while the spatial representation baseline does not have such issue.

Questions for authors

a. How do the authors distinguish the effect of the nonlinear output representation from global mixing? E.g. Can spatial head with global mixing close the performance gap? (weakness 1)

b. Please demonstrate out-of-distribution generalizability for the proposed method, e.g. training on Shapes and testing on shapes with holes, and report the proposed method comparing with spatial representation? (weakness 2)

c. Please provide analysis on how the VAE-based output representation helps in training, or explain the mechanism in more detail, e.g. which types of nonlinear representations work, and which do not? Detailed analysis of the explainability of the latent space would be preferred.

2 comments
AuthorsOpenReview ↗

Response to Reviewer uCBn

Thank you for three precise and constructive questions. Each proposed a specific experiment, we ran all three, and they are now Appendix G.5 (global mixing) and Section 6.3 (shapes with holes).

(a) Global mixing. This was an excellent suggestion. We built spatial heads with global Fourier and attention mixing, matched within 0.5% in parameters (Table 31). Fourier mixing closes about half of the gap (41 to 53%), so global routing is indeed part of the mechanism, as you anticipated. The latent head still wins on 10/10 seeds, so routing does not explain the rest. Attention mixing does not help.

(b) Out-of-distribution targets. We found this to be a very strong suggestion, and we thank you for laying out your proposed experiment. We ran it: train on Shapes, test on shapes with holes. The latent head with a frozen pretrained decoder generalizes far better than the spatial head. Its error is 60 to 71% lower on annuli, holed shapes, nested shapes and polygons (10/10 seeds), and it is ahead on every unseen family for all five backbones at 12,800 pairs (Section 6.3).

You also raised the concern that "targets that this decoder cannot represent cannot be reconstructed". We measured this directly, and in this experiment it does not arise. The decoder, trained only on solid shapes, reproduces shapes with holes almost exactly (NMSE 0.014 to 0.059 when its code is fitted to the true target). The spatial head can represent any image, yet its error on these targets is 2.5 to 3.4 times the latent head's.

We then looked for a case where the concern does apply, and found one on measured data: a nested pair of cylinders, which a prior trained on separate cylinders cannot represent. Even there the latent head beats the spatial head (lower NMSE on 4 of 5 cells, higher SSIM on 14 of 15 pairs). A frozen linear PCA basis does better still, so we now state this as a limitation, together with a simple check that needs only target samples, and that Guo et al. (2022) also recommend: encode and decode them.

(c) Which nonlinear representations work, and which do not. Thank you for asking for this; the discussion now states the answer directly (Section 8).

  • Work: a global code followed by nonlinear synthesis, pretrained or not. The frozen VAE, a randomly initialized decoder and NOMAD's decoder all beat the spatial head by 46 to 73%, and a plain autoencoder is within 5% of the VAE.
  • Do not: a decoder fed a local 8x8 map, which is worse than the spatial head, and a linear basis on sharp targets.
  • In training: the frozen VAE head passes the spatial head's final error after a median of 300 of the 3,000 updates (10/10 seeds).
  • Latent space: the distance of a predicted code from the training codes tracks per-object error (Spearman 0.83), which gives a warning without ground truth.

Presentation. Thank you for the feedback on clarity. Figure 6 adds a diagram of the three heads with tensor shapes.

I thank the authors for the extensive experimental results, which addressed my questions, e.g. OOD concern . However, I did not see in-depth discussion to distinguish the proposed method from established latent-decoding strategies. Therefore, I keep my score.

Reviewer_k2pROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the role of output representations in learned inverse scattering. Instead of directly predicting spatial contrast fields, the authors proposed a modular framework that predicts latent coordinates and reconstructs the target through a nonlinear decoder. The paper systematically compares spatial prediction, linear PCA-based representations, and nonlinear autoencoder-based representations across five neural operator backbones. Experimental results show that nonlinear latent prediction reduces reconstruction NMSE compared to direct spatial prediction. The authors further investigated the effects of pretraining, parameter count, latent dimensionality, and target structure. Their findings suggest that nonlinear output representations are particularly beneficial for reconstructing sharp and structurally complex targets.

Critical strengths

1. Extensive and well-controlled experimental analysis

The authors conducted systematic comparisons across multiple neural operator backbones, target distributions, and noise levels. The experiments include carefully designed controls for model capacity, computational cost, decoder pretraining, and latent dimensionality.

2. Consistent performance improvements and useful empirical insights

The proposed nonlinear output representation achieved substantial reconstruction improvements across the evaluated backbones. The observation that nonlinear representations provide greater benefits for sharp structures than for smooth low-rank fields is particularly valuable. This finding highlights the importance of output representation design in inverse problems and may help guide future architecture development.

Critical weaknesses

1. Limited methodological novelty and insufficient comparison with existing latent representation approaches

Predicting latent coordinates and reconstructing outputs through a pretrained or jointly trained nonlinear decoder is a well-established strategy in representation learning and inverse problems. Related approaches (including nonlinear manifold decoders and autoencoder-based inverse models) have already explored similar ideas. While the authors acknowledge these studies, the proposed framework introduces relatively limited architectural novelty. Moreover, the experiments primarily compare nonlinear decoding against direct spatial prediction and PCA, without sufficiently evaluating alternative established latent representation approaches. Consequently, it remains unclear whether the reported improvements reveal a broadly new methodological insight or mainly reflect the advantages of an already established representation-learning strategy.

2. Limited generalization beyond controlled synthetic inverse-scattering settings

All main experiments use relatively small (64×6464\times64) contrast fields, a fixed wavenumber, and a full-aperture acquisition configuration. Although the authors evaluated multiple target distributions and noise levels, the models were trained and tested within the same target distribution. No experiments demonstrate generalization to unseen target structures, different acquisition geometries, or substantially different physical configurations.

Questions for authors

1. Generalization to unseen target distributions and acquisition configurations

Could the authors evaluate the proposed approach under distribution shifts, such as training on one family of contrast fields and testing on structurally different targets? In addition, how does performance change under different wavenumbers or limited-aperture acquisition settings?

2. Novelty relative to existing latent-space inverse methods

Could the authors distinguish the proposed framework from existing nonlinear manifold decoders and autoencoder-based inverse models, such as NOMAD [1], InvLINT [2], and Auto-Linear [3]? In particular, could they provide additional comparisons with representative existing latent-space reconstruction approaches under matched experimental conditions?

Reference:

[1] Seidman, J., Kissas, G., Perdikaris, P., & Pappas, G. J. (2022). NOMAD: Nonlinear manifold decoders for operator learning. Advances in Neural Information Processing Systems, 35, 5601-5613.

[2] Feng, Y., Chen, Y., Feng, S., Jin, P., Liu, Z., & Lin, Y. (2022, June). An intriguing property of geophysics inversion. In International Conference on Machine Learning (pp. 6434-6446). PMLR.

[3] Feng, Y., Chen, Y., Jin, P.,, Feng, S., & Lin, Y. (2024). Auto-linear phenomenon in subsurface imaging. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), Vol. 235. JMLR.org, Article 528, 13153–13174.

2 comments
AuthorsOpenReview ↗

Response to Reviewer k2pR

Thank you for the encouraging assessment of the controls and for two concrete suggestions: a matched comparison with NOMAD, InvLINT and Auto-Linear, and tests beyond the controlled setting. Both are now in the paper, and both sharpened its message.

Comparison with existing latent approaches. Following your suggestion, we ran all three methods under our matched protocol (Shapes, ten paired seeds, NMSE; Appendix G.3).

head200 pairs3,200 pairsunseen annuli / holed / nested
spatial0.4420.0350.819 / 0.541 / 1.026
scratch decoder (paper, trained jointly)0.2370.0130.746 / 0.414 / 0.966
frozen VAE (paper)0.1690.0180.241 / 0.218 / 0.374
NOMAD, parameter-matched0.1180.0140.860 / 0.350 / 1.108
NOMAD, compute-matched0.1110.0170.836 / 0.356 / 1.069
InvLINT0.3440.0880.433 / 0.430 / 0.597
Auto-Linear0.2140.0600.471 / 0.320 / 0.696
  • NOMAD is a nonlinear decoder trained jointly with the network, like our scratch decoder, so its accuracy supports our finding that the gain needs no pretraining. A per-pixel decoder cannot match our head in parameters and in test-time compute at once, so we ran it both ways. Matched in parameters, it is level with our scratch decoder at 3,200 pairs (0.014 against 0.013) while using 3.4 times the test-time compute. Matched in test-time compute, our scratch decoder is 26% lower (0.013 against 0.017, 9 of 10 seeds). With 200 pairs NOMAD is ahead either way, and matched in parameters it is 22 to 30% ahead of our frozen head in distribution. On unseen targets our frozen head is 38 to 72% below NOMAD on every family (3,200 pairs).
  • InvLINT and Auto-Linear. Our frozen head beats both in distribution at both training sizes, and on annuli, holed and nested shapes with 3,200 pairs.
  • A new methodological insight. We thank you for your question on if the gains "reveal a broadly new methodological insight," and this comparison helps us come to an answer. Two things turned out to matter, and they are separate. First, a decoder that builds the image from one global code and is trained from scratch with the network is already accurate on targets like the training set: with 3,200 pairs our scratch decoder and NOMAD are both about 60% below the spatial head. Pretraining is not needed for that. Second, only decoders trained on clean target images alone and then frozen stay accurate on new kinds of targets; decoders trained on the measurement-target pairs do not. Auto-Linear let us test the second point on a method we did not design. Its decoder is frozen, so we predicted before scoring it that it would beat the spatial head, our scratch decoder and NOMAD on annuli, holed and nested shapes, although with 3,200 pairs it is less accurate than the spatial head on familiar targets. It did, in 9 of 9 comparisons.
  • To our knowledge, none of the three compares output maps on one inverse network at matched parameters and compute, or compares the same decoder frozen and jointly trained. InvLINT and Auto-Linear do compare with InversionNet, a different jointly trained network, without matching.

Generalization beyond the controlled setting. We ran each case you listed.

  • Unseen target structures. Trained on Shapes and tested on polygons, annuli, holed and nested shapes, the latent head has 60 to 71% lower error than the spatial head (10/10 seeds) and is ahead for all five backbones (Section 6.3).
  • Wavenumbers and apertures (rule registered: lower on at least 6 of 8 settings). It is lower on 8 of 8, by 37 to 59% across k = 3 to 48. The gain narrows as the aperture closes, to 3.5% with a quarter of the receivers (Table 30). No head works at another wavenumber without retraining.
  • Contrast. It is 19 to 55% lower at four contrast scales (3/3 seeds each).
  • Measured data and a larger grid (100x100): Section 5.4.

We again thank you for your thoughtful review. With the above mentions incorporated into the paper, we feel we now capture a better understanding of the strengths and limitations of nonlinear parameterizations, and better understand the strength in generalization.

I thank the authors for the additional experiments and clarification. However, I still think the novelty is limited, considering the existing methods. Therefore, I will keep my recommendation.

Reviewer_2JC8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the inverse scattering problem where the contrast field is reconstructed from non-linear measurements of the scattered fields. The authors propose replacing the common pixel-wise contrast reconstruction with a learned nonlinear latent representation. The experiments on simulated data show that this approach consistently improves reconstruction accuracy over spatial and PCA-based baselines. The study examines the source of these gains and their dependence on target distributions.

Critical strengths
  1. The paper is well-written and clearly presented; the main contributions are easy to follow and understand.
  2. Experiments on simulated data demonstrate the effectiveness of the proposed nonlinear output representation in improving reconstruction accuracy.
Critical weaknesses
  1. The experiments are exclusively on simulated data. The authors are recommended to evaluate the proposed method on real experimental data to demonstrate its practical applicability.

  2. Despite the promising empirical results, the novelty and machine learning contribution seem limited. Leveraging the latent space of a pretrained VAE has been explored in other domains, such as latent diffusion models. Consequently, the overall contribution may be insufficient for publication at ICLR.

Questions for authors
  1. How does the proposed nonlinear latent representation fundamentally differ from existing approaches that use pretrained generative priors to parameterize outputs? See the following papers.

  2. How well does the proposed method generalize to real experimental measurements and target distributions not seen during training?

[1] Guo, Rui, et al. "A nonlinear model compression scheme based on variational autoencoder for microwave data inversion." IEEE Transactions on Antennas and Propagation 70.11 (2022): 11059-11069.

[2] Khorashadizadeh, AmirEhsan, et al. "Deep injective prior for inverse scattering." IEEE Transactions on Antennas and Propagation 71.11 (2023): 8894-8906.

[3] Zhang, Borong, et al. "Back-projection diffusion: Solving the wideband inverse scattering problem with diffusion models." Computer Methods in Applied Mechanics and Engineering 443 (2025): 118036.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2JC8

Thank you for the careful review and for the references [1]-[3], which are now cited and discussed in Section 7. Your two requests, measured data and a sharper statement of the contribution, led to the two largest additions to the paper.

Real experimental data. We agree this was the most important gap, and we are glad to report that it is closed. The paper now includes the Institut Fresnel database, the long-standing measured benchmark in inverse scattering, and the University of Manitoba database, built for training learned methods on measurements. Decision rules were fixed before any fit, and the latent head wins on both (Section 5.4, Table 2).

  • Institut Fresnel, trained on simulation only. The latent head passes the preregistered rule on all five backbones. Its NMSE is 9 to 44% lower on four of them. OFormer's spatial head needs a longer schedule, so we added a control with three times the training budget for both heads; there the latent head's NMSE is 49% lower (7 of 8 cells). Its reconstructions are also structurally closer to the targets: SSIM is higher on every cell for every backbone (0.88 against 0.79 on Transolver), and support overlap is higher on 39 of 40 backbone-cell combinations. A check that needs no ground truth agrees: its reconstructions fit the measured fields better. Figure 18 shows the reconstructions. Appendix G.8 gives the rig calibration, the tallies for every metric (centroid and permittivity errors are mixed), and the sensitivity to the target positions (Tables 33 to 37).
  • University of Manitoba, trained and tested on measurements. NMSE is 47% lower at held-out positions on 5/5 seeds, with SSIM 0.94 against 0.84, and still 33% lower when every head trains three times longer. The latent head also locates and sizes the specimens better: centroid error 1.16 mm against 2.20 mm, diameter error 0.37 mm against 0.76 mm, and support overlap 0.906 against 0.806 (5/5 seeds). For rotations never measured in training the two heads are within 4% in NMSE, and the latent head still locates and sizes the specimens better (Appendix G.9).
  • A measured target outside the prior's family (nested cylinders). The latent head still beats the spatial head (lower NMSE on 4 of 5 cells, higher SSIM on 14 of 15 pairs). A frozen PCA basis beats both, which marks where a frozen prior stops: ours was trained on separate cylinders. Reference [1] reconstructs this same target with a VAE trained on overlapping cylinders, so the limit is one of prior coverage (see the reply to uCBn).

Novelty, and the difference from pretrained generative priors [1]-[3]. We appreciate this question, because answering it let us state the contribution more precisely. The head differs from generative priors in what the decoder does and in why it helps.

  • What it does. In [1] and [2] a pretrained generator constrains an optimization that is run anew for every object (in [1], Gauss-Newton iterations on the 36 latent variables of a VAE). In [3] a diffusion model generates the image in pixel space. Our decoder is the last layer of a network that reconstructs in one pass with no solver. Per-object optimization is more accurate where it converges, and the two are complementary: started from our head's code, per-object optimization through the same decoder converges on all 128 test objects, where a cold start stalls on the strongest scatterers (Appendix G.6).
  • Why it helps. Not because of pretraining. A randomly initialized decoder at matched parameters and compute beats the spatial head in 7/7 settings on 10/10 seeds, and is 41% lower at 12,800 pairs on the matched controls' test set (registered rule, 3/3 seeds, Appendix G.2).
  • What pretraining adds. Robustness to unseen targets. With the decoder's architecture fixed, the frozen pretrained decoder is 28 to 45% below a randomly initialized one on unseen families (10/10 seeds, Table 27).

The machine-learning contribution is to separate two effects that prior work leaves entangled. Building the image from one global code with a nonlinear decoder is what makes a network accurate on targets like its training set, with or without pretraining. Training that decoder on clean target images alone, and then freezing it, is what keeps the network accurate on new kinds of targets. Latent diffusion models pretrain an autoencoder to compress; our controls show that here neither compression nor pretraining produces the gain.

Generalization to target distributions not seen during training. Thank you for raising this; it is now Section 6.3. The latent head has 60 to 71% lower error than the spatial head on four unseen target families (10/10 seeds). Across unrelated classes, with 50 to 800 pairs of the new class it is below the fine-tuned spatial head in 9 of 9 class pairs (Appendix G.7).

First, I would like to thank the authors for their extensive efforts in running additional experiments in such a short period, particularly the evaluation on real experimental data and the comprehensive ablation studies. These additions have strengthened the empirical evaluation of the paper. However, I remain unconvinced by its methodological novelty. The central idea of learning a low-dimensional nonlinear representation of the output and training a network to predict its latent coordinates is well established in machine learning. Although the authors provide a useful application of this strategy to inverse scattering and demonstrate its effectiveness through extensive experiments, I do not see a sufficiently fundamental methodological advance or a broadly applicable machine-learning insight beyond this application. In my view, the contribution is primarily empirical and domain-specific, and may therefore be better suited to a specialized journal in inverse problems or computational electromagnetics than to a general machine-learning conference such as ICLR. Therefore, I keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

We thank all three reviewers for their careful and constructive reviews. We are grateful for the recognition that the paper is "well-written and clearly presented" (2JC8), that its experimental analysis is "extensive and well-controlled" (k2pR), and that the "multi-seed experiments make the conclusions more rigorous" (uCBn). The suggestions in the reviews led directly to five sets of new experiments, and the paper is much stronger for them. All five are now in the paper (new text in red). Below, "latent head" is the paper's nonlinear head.

The reviews suggestedWhat we foundWhere
Real measured data (2JC8)The latent head beats the matched spatial head on both measured datasets. Institut Fresnel, the long-standing measured benchmark in inverse scattering, trained on simulation only: the preregistered comparison passes on all five backbones, with higher SSIM on every cell. University of Manitoba, trained on measurements: 47% lower NMSE, with centroid and diameter errors about halvedSection 5.4, Table 2
Unseen target distributions (all three)60 to 71% lower error than the spatial head on four unseen target families (10/10 seeds), and ahead on every unseen family for all five backbonesSection 6.3
Existing latent methods under matched conditions (k2pR)With 3,200 pairs our frozen head beats NOMAD, InvLINT and Auto-Linear on annuli, holed and nested shapes. In distribution it beats InvLINT and Auto-Linear, and with 3,200 pairs our jointly trained head beats NOMAD by 26% at matched test-time computeAppendix G.3
Other wavenumbers and apertures (k2pR)Lower error in 8 of 8 settings, across a 16-fold range of wavenumberAppendix G.4
Global mixing as the explanation (uCBn)Global mixing closes about half of the gap; the latent head still wins on 10/10 seedsAppendix G.5

What is new. To our knowledge, this is the first study to:

  1. compare spatial and latent output heads on the same inverse network with parameters and compute matched, across five backbones. This shows that the gain belongs to the output head, not to model size or to one architecture.
  2. report test errors, including on unseen targets, for the same decoder pretrained and frozen, fine-tuned, and trained from scratch. This shows that the gain needs no pretraining, and that pretraining is what makes the head robust to unseen targets.
  3. run seed-paired comparisons of output heads on measured data. This shows that the gain is not an artifact of simulation.

These comparisons are worth documenting because the output head is usually chosen by convention, and they show that the choice changes error by a third on average and determines how a network behaves on targets it has not seen.

Limits are stated in the replies below and in Section 8.

One correction, which this revision brought to light. While extending the learning-rate grids for the new comparisons, we found that higher rates improve two spatial baselines in the 12,800-pair table. In the corrected Table 23 the latent head remains ahead on all five backbones: by 3% on Transolver (submitted: 9.4%), by 14% on FFNO (submitted: 30.4%), and about as submitted on the other three.

In closing. We thank the reviewers again; your comments strengthened the science of this paper. This work was born from a desire for a more rigorous understanding of output parametrizations. The performance of latent-predicting networks has been researched before. Our contribution is a modular latent head, and with it a fair, controlled comparison of spatial, linear, and nonlinear outputs across backbones, controlling for parameters and compute, pretraining, rank, and supervision, which to our knowledge is new. It shows when nonlinearity pays off over a linear basis, and that the gain does not require pretraining, more parameters, or an undercomplete representation. ICLR is a conference on learning representations, and our evidence suggests that the output representation deserves the same consideration and testing as network design in learned inverse scattering.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.