acceptodds
Under review as a conference paper at ICLR 2027

Fisher-Informed Neural Operators for PDE-Constrained Inversion

Abstract

Scientific inverse problems often require repeated evaluations of high-dimensional PDE forward models, motivating neural surrogates as efficient alternatives to accurate numerical solvers. However, even when a surrogate closely approximates a numerical solver in , using it as the forward model can yield highly inaccurate parameter estimates. Here, we study gradient-based minimization of observation error for parameter inversion and propose training neural surrogates using the leading eigenvectors of an amortized Fisher information matrix. The resulting Fisher-informed Neural Operator (FINO) is a derivative-informed neural operator that learns the forward model Jacobian only along directions relevant to maximum-likelihood optimization. For computational efficiency, FINO uses randomized SVD to obtain an amortized set of Fisher-information eigenvectors. Our theoretical analysis relates Fisher directions to maximum-likelihood estimate recovery and characterizes how the effective rank of the inverse-problem Jacobian enters through spectral truncation and conditioning. We evaluate FINO on three forward problems with widely varying sensitivity structure: Darcy flow, two-dimensional Navier–Stokes flow, and seismic imaging with the acoustic wave equation. On Navier–Stokes, FINO reduces mean parameter RMSE by % and % relative to FNO trained only with and DINO, respectively. Controlled spectral-decay experiments show how FINO’s benefit depends on the Fisher spectrum. Its amortizable offline cost also makes FINO practical for scientific inverse problems.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural operators, PDE-constrained inverse problems, Fisher information, Derivative-informed learning, Scientific machine learning, Maximum likelihood estimation, Surrogate-based optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dULvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new method that uses Fisher information derived regulation to help the training of neural operators in the settings of sparse observed parameter recovery. It uses Fisher information to select the directions in which observations affect the parameters most. Their estimation is based on parameter sampling and random vector probing. The theory shows that the first r Fisher eigenvectors span a space that minimizes the maximum directional variance in constrained inversion compared to other r-dimensional subspaces. They also provide some error bound. Performance-wise, it does not generally perform better.

Critical strengths
  • The novelty in using likelihood-parameter subsampled Jacobian information to help full-field reconstruction
  • Theoretical work on the correctness and effectiveness of FINO
  • Well-written and reporting negative results faithfully
Critical weaknesses
  • The lack of baselines. There are many previous methods in sparse parameter recovery beyond FNO and PINO, such as Neural Inverse Operator and posterior sampling methods (DiffusionPDE, FunDPS, DDIS, etc).
  • The weak performance. In Table 1, FINO outperforms (limited) baselines in just 1/4 settings. Considering the additional compute overhead (20 hours compared to PINO's 0.5 hours in NS equations), it's hard to justify this method.
  • The theory does not consider the error from estimating the Fisher subspace. It would be good to see how sampling budget and random approximation affect the accuracy and efficiency.
Questions for authors
  • Could the uncertainty due to limited observations be handled in this framework? It seems that FINO can produce only one solution with no uncertainty estimation.
  • How's Fisher's subspace compared to a random subspace? It would be interesting to see the performance difference.
  • Does the result converge as the M and l used for approximating the Fisher matrix become larger and larger? What's the relation of random budget vs accuracy?
3 comments

We want to thank the reviewer and appreciate your suggestion on adding more baselines, and recognizing our theoretical contribution. We have carefully addressed every comment and have uploaded a revised version of the manuscript. All newly added or modified text in the PDF is shown in blue.

On weakness 1 and question 1: Baselines, scope and uncertainty On top of our three baselines, we have added NIO, DDIS (with FNO or with FINO) evaluations (Table 1 and 12-16). Uncertainty estimation is not the main objective of our method. FINO aims to answer ``What is the role of a deterministic neural network's Jacobian in solving gradient-based inverse problems''. Thus, we do not claim that our method gives better uncertainty estimation, and comparing with probabilistic baselines is not an apples-to-apples comparison.

In DDIS (Lin et al., 2026), we replace FNO with FINO while keeping the diffusion prior and paired sampling protocol fixed. On one Navier--Stokes target with 256 noiseless observations and 16 samples, FINO improves ensemble-mean RMSE in 16/18 guidance-budget settings, SSIM in all 18, and simulator-evaluated residual in 17/18. At D=4D=4, FINO with 500 guidance updates nearly matches FNO with 1,000 updates in RMSE (0.034120.03412 versus 0.034050.03405). FINO also yields tighter ensembles in all 18 settings, demonstrating that its use is not restricted to a single point estimate. These preliminary results, obtained while diffusion-prior training is ongoing, support improved limited-budget sampling-based recovery.

Separately, we tested FINO's recovery under noise. On 100 Burgers test samples, FINO improves point-recovery RMSE over FNO at both 11% noise (0.18790.1879 versus 0.24540.2454) and 55% noise (0.16820.1682 versus 0.20600.2060) (Table 17).

On weakness 2: Performance depends on Fisher spectrum Our theoretical analysis (in Section 4.3, Figure 4 and 6) identifies when the Fisher-informed training is useful in surrogate-based inversion. The explanation follows as:

i) Experiments are designed to show that even if forward errors are similar, inversion performance is different. Eq. (7) weights residual-dependent derivative error by δr−2\delta_r^{-2} and forward error by δr−1\delta_r^{-1}. This explains why forward accuracy alone is insufficient. In NS, FNO and FINO have nearly equal forward validation errors (±0.2\pm 0.2%), yet FINO reduces reconstruction RMSE by 42.842.8% versus FNO and 12.612.6% versus PINO, despite PINO's lower forward error.

ii) New experimental results on a new PDE inversion As Burgers' inverse problem's spectrum decays slowly, FINO's usefulness is demonstrated not only in the inversion, but also forward prediction, as the forward and inverse relationship are maintained through slowly decaying Jacobian. Darcy has to remain as a negative result as its Fisher spectrum decays fastest, according to our theoretical analysis in Section 4.3 and Theorem 1. Thus, combined with our spectral diagnostics in Figure 6, the expected FINO's success is most pronounced in this order: Burgers, followed by Navier--Stokes, and Darcy.

On question 3: Cost and sampling budget. Our contribution is a matrix-free algorithm, which makes previously infeasible computation, feasible. Section 3.1 and Appendix B.5 specifies MlMl VJP directions for basis construction and NLmN_Lm Jvp directions for output labels. With dd input and oo output coordinates, directional labels require O(NLom)O(N_Lom) entries rather than O(NLod)O(N_Lod) for full Jacobian labels. The factor d/md/m is 10.24 for Navier-Stokes and 81.92 for Darcy. Repeated inversion can amortize the cost (Appendix C.5).

The added Appendix L derives E∥H^−Hˉ∥F2=A/M+B/(Mℓ),\mathbb E\|\widehat{\boldsymbol H}-\bar{\boldsymbol H}\|_F^2=A/M+B/(M\ell), for independent inputs and Gaussian probes with finite Fisher second moments; AA and BB quantify input and probe variability. With a positive boundary eigengap, matrix perturbation bounds transfer this estimation error to the leading-subspace error. Increasing ℓ\ell leaves the A/MA/M floor; increasing MM at fixed ℓ\ell gives mean-square consistency. For NS, increasing ℓ\ell from 4 to 256 raises capture from 63.263.2% to 97.997.9% of the optimal rank-64 empirical capture (Table 18).

On weakness 3: Fisher-subspace estimation error. Eq. 10 bounds transfer via supervised derivative error and subspace mismatch; Appendix K separates amortization, Monte Carlo, and numerical errors. New measurements on three NS find 99.399.3--99.499.4% local observed-sensitivity capture along FINO trajectories, versus approximately 9.89.8% for rank-matched random bases. At final states, mean relative gradient error is 0.3270.327 for FINO versus 1.2591.259 for FNO (Table 19).

On question 2. Fisher versus random directions. Table 12 reports lower mean reconstruction RMSE for FINO than rank-matched random direction supervision in all three PDE comparisons. This supports selecting derivative directions according to observation sensitivity to improve inverse recovery.

Thank you for the reply. It addressed some but not all of my concern. It mostly addressed weakness 1 and 3. On weakness 2, it's still hard to justify the 20-hour overhead with marginal accuracy improvement. What's more, on question 2, the random subspaces are almost equal to FINO on the Darcy and the Burgers equations, which limits the advantage of this method to just NS. I believe that NS forward/inverse problem is a competitive field, and the current benchmarks are not enough. Therefore, I remain my score.

Thank you very much for your engagement and response! We understand your remaining concerns and your comment motivated four additional benchmark settings and a more direct evaluation of Fisher versus random supervision.

1. The benefits now extend beyond the original NS (Re=100) case. We added 2D Navier–Stokes (NS) at higher Reynolds number, Re=500, 3D NS at Re=500 (24324^3), and Burgers on higher dimension, 64² and 128² grids (previously, it was 32²). Updated Table 1 compares 5 baselines and FINO over 100 inverse targets, and expands the earlier 10 target random subspace diagnostic.

Mean parameter RMSE:

New settingFNODINOPINONIORandFINO
NS 2D, Re=500, 64²0.15580.26820.05570.19420.23280.0262
NS 3D, Re=500, 24³0.19260.19210.19170.23430.19870.1829
Burgers, 64²0.22800.34240.22410.50030.20990.1868
Burgers, 128²0.22690.35470.27820.47200.21420.1872

The new settings use equal training epoch counts across methods within each setting. We observe:

  • For 2D NS at Re=500, Rand’s mean RMSE is inflated by two extreme reconstructions, substantially increasing the parameter RMSE in the 100-target average.
  • Also, FINO also achieves higher mean SSIM than Rand (0.9078 versus 0.8419).

FINO now achieves the lowest mean RMSE in five of the six in Table 1 settings. In particular, SSIM improved by 0.57 (FINO) versus 0.51 (random) on Burgers 64², and 0.48 versus 0.42 on 128² (FINO vs Rand).

2. Direction selection matters with limited data and high-dimensional parameters The new Burgers sweep (Figure 5) varies training-set size while keeping 8 supervised directions. FINO outperforms random directions (Rand) at all training set size on the 64² and 128² grids. On 64², FINO with 32 training inputs achieves around 0.193 RMSE, outperforming Rand with 64 inputs (0.210). This directly demonstrates a benefit from selecting informative directions when training examples are limited. That is, as the dimension of the inverse problem increased and as we are limited to a smaller number of training data points, we observed greater efficiency of FINO over a random-direction supervised model.

Also, the existing seismic study provides complementary evidence for sparse supervision with limited training data. It uses only 202 synthetic training models on 256×256 velocity and supervises 64 directions, approximately 0.1% of a velocity field dimension. Across 20 held-out targets, FINO increases SSIM from 0.39 to 0.43 compared to FNO (Appendix C.3, Table 7). This demonstrates the feasibility of sparse directional supervision for high-dimensional inversion.

3. Offline costs are incurred once and spread across repeated inversions. Figure 5 shows the tradeoff between training-set size, reconstruction accuracy, and offline cost. On the 64² Burgers grid, FINO trained on 32 inputs achieves lower RMSE than Rand trained on 64 inputs (0.193 versus 0.210), with reported wall-clock offline costs of 11.00 vs 13.51 minutes. This comparison demonstrates how selecting informative derivative directions can improve recovery with fewer training examples, which in turn gives a smaller offline budget. Once trained, the surrogate can be reused across inverse problems for the same forward and observation setting. This makes the investment particularly relevant when representative training data are limited, high-dimensional and many inversions are required.

Also, the required offline investment varies with the simulator, training configuration, and supervision budget. The eight direction Burgers sweeps report approximately 9–38 minutes of offline cost, while Table 10 describes an offline cost for NS with 400 training inputs and 400 supervised directions. For that setting, the recorded runtimes yield a break-even after 21 inversions relative to direct numerical inversion (Appendix C.5).

4. Darcy helps clarify when Fisher-informed supervision is useful. Theorem 1 relates parameter recovery to both forward and directional-derivative errors, with Fisher eigenvalues controlling how these errors are amplified. This makes the Fisher spectrum a useful diagnostic for interpreting the value of derivative supervision. Darcy’s rapidly decaying spectrum concentrates inverse sensitivity in relatively few directions. In this setting, FINO does not improve on FNO, and Fisher and random supervision achieve similar recovery. This contrasts with the broader sensitivity spectra and larger observed gains on NS and Burgers. Together with the controlled spectral-decay experiment, the Darcy result connects our theoretical analysis to the observed problem dependence: understanding when Fisher-informed derivative supervision adds value is itself an important part of our contribution.

We appreciate your time, continued engagement and thoughtful comments throughout the review process.

Reviewer_mecTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper "FISHER-INFORMED NEURAL OPERATORS FOR PDE-CONSTRAINED INVERSION" introduces FINO, a novel neural operator formulation, that is trained using the leading eigenvectors of a Fisher information matrix. While the method has a significant offline cost, it is than computationally cheap during computation of inverse solutions. The authors perform multiple experiments and compare their neural operator to FNOs, DINOs and PINO.

Critical strengths

The derivation of the new method is sound according to the reviewer and the general approach of focusing on relevant eigenvectors based on the Fisher information is a good idea. The Reviewer especially likes Figure 1 that illustrates that the gradients of the original simulator and FINO align. The authors also have shared their code.

Critical weaknesses
  • For the inverse problem, there seems to be no probabilistic approach considered. Especially when recovering under noise this is a major drawback of the paper. In the Appendix there is only a single experiment with only a limited amount of noise and the resulting uncertainty in the estimated parameters is never considered. What would be needed to add this ? Are the authors only considering a maximum-likelihood estimate ? Could it also be used in a Bayesian setting with a prior ?
  • The method does not show clear empirical benefits. While for the Navier-Stokes example the performance in the inverse setting is the best for the new approach, the method performs inferior to competing approaches in the forward pass. For instance a Pareto front using forward and inverse performance could be a better way of visualizing the comparisons. For the Darcy problem, the method is not able to outperform FNO despite a higher computational cost.
  • How have the hyperparameters involved been chosen ?
  • As also outlined by the authors the computational cost of the approach is significantly higher than FNO.
Questions for authors

See weaknesses

3 comments

We appreciate the reviewer for recognizing the Fisher-informed formulation, and for the suggestions on additional experiments. We have carefully addressed every comment and have uploaded a revised version of the manuscript. All newly added or modified text in the PDF is shown in blue.

On Weakness 1: Probabilistic formulation and uncertainty The main theory concerns Maximum Likelihood Estimates (MLEs). Although this is not the main goal of this paper, as the other reviewer suggested, FINO can also supply likelihood gradients when combined with an inference prior. We now demonstrate this in preliminary DDIS (Lin et al., 2026) experiments using a learned diffusion prior, whose training is ongoing.

Across 18 guidance-budget settings, FINO improves ensemble-mean RMSE in 16, SSIM in all 18, and simulator-evaluated observation residual in 17 (Tables 13--15). At D=4D=4, FINO with 500 guidance updates nearly matches FNO with 1,000 updates in RMSE (0.034120.03412 versus 0.034050.03405). The RMSE advantage narrows and sometimes reverses at larger budgets. FINO also produces tighter ensembles in all 18 settings (Table 16). These results demonstrate compatibility with diffusion-prior-based inference and improved reconstruction at limited guidance budgets.

Separately, we tested FINO's recovery under noise. on 100 Burgers test samples, FINO improves point-recovery RMSE over FNO at both 11% noise (0.18790.1879 versus 0.24540.2454) and 55% noise (0.16820.1682 versus 0.20600.2060) (Table 17). Together, these results demonstrate sampling compatibility and recovery under noise.

On Weakness 2: Forward-Inverse tradeoff and experiment result on new PDE. Thank you for the reviewer's suggestion; this experiment actually helped demonstrate that the best forward predictor need not be the best inverse solver. Figure 18 provides the requested Pareto comparison. On Navier--Stokes, FINO has 42.842.8% lower reconstruction RMSE than FNO, 12.612.6% than PINO. For the newly added experiment, Burgers, FINO improves forward test error by 43.643.6% and reconstruction RMSE by 24.024.0% relative to FNO (Table 1). In Darcy, FNO outperforms FINO in both metrics. This mechanism is explained in Section 4.3 and Figure 4. The Fisher-informed training is especially useful for inverse problems with high effective Fisher rank and slowly decaying spectra : that is, accurate inversion requires derivative accuracy across more directions. Thus, our spectral diagnostics in Figure 6 suggests that expected FINO's success is most pronounced in the following orders: Burgers, followed by Navier--Stokes, and Darcy.

On weakness 3: Hyperparameter Supervision ranks were chosen through qualitative inspection of the sensitivity spectra, giving m=400/200m=400/200 for Navier--Stokes/Darcy. The corresponding raw derivative coefficients were 10−4/10−310^{-4}/10^{-3}. Basis construction used M=200M=200 inputs and ℓ=405/205\ell=405/205 probes per input, constructing 400/200400/200 directions, respectively. Checkpoint selection used forward validation/test errors. That is, to empirically support Theorem 1, we intentionally chose FNO/FINO with almost the same test error (±0.2\pm 0.2%). Loss normalization and the additional settings are specified in Appendices B.4 and C.6.

On weakness 4: Computational Cost FINO's higher offline cost reflects its additional supervision of inversion-relevant derivatives. DINO and DIFNO motivate this investment: accurate L2L^2 forward predictions alone do not guarantee accurate derivatives or solutions obtained through gradient-based inversion (O'Leary-Roseberry et al., 2023; Yao et al., 2026). Our analysis identifies how forward and directional-derivative errors enter the recovery bound.

Our matrix-free construction makes directional supervision tractable: label storage scales as O(NLdoutm)O(N_Ld_{\rm out}m) rather than O(NLdoutdin)O(N_Ld_{\rm out}d_{\rm in}) for full Jacobians (Appendix B.5). This investment is particularly relevant when representative training examples are limited, as in real-life application, and our seismic study with 202 synthetic velocity models. In certain realistic settings, such as Full-Waveform inversion, high-quality prior samples (realistic subsurface geology) are very scarce. Nonetheless, a physically differentiable simulator like Devito (Louboutin et al., 2019; Luporini et al., 2020) provides scalable stencil computations, which allow us to compute derivative labels with the dataset we have.

Derivative labels supply local sensitivity information beyond forward values; together with the forward loss, they target the two surrogate-error contributions in Eq. (7), which, in an idealistic scenario, guarantees recovery of the Maximum Likelihood Estimate. This motivates investing additional computation in extracting more information from each available model. We plan to apply this matrix-free approach to 3D seismic problems.

We sincerely thank you for your feedback and would welcome any further comments or suggestions.

The Reviewer thanks the authors for their answer and the updates to the paper. While the Reviewer believes that these updates have strengthened the paper, unfortunately not all concerns all sufficiently addressed and thus the initial score is maintained. Especially given the additional computational cost of the method, the performance gains are quiet small and for instance in the the ablation of Fisher vs random subspace in Table 12 in the Appendix, the Fisher subspace is not better in all cases.To thus justify the method, the Reviewer would require an experiment on a more challenging PDE such as Navier-Stokes with higher Reynolds-number or in 3D.

Thank you very much for the concrete suggestion to evaluate higher Reynolds number and three-dimensional flow! We have now added both, together with Burgers experiments in higher dimension.

1. The benefits now extend beyond the original NS (Re=100). We added 2D Navier–Stokes at higher Reynolds number, Re=500, 3D NS at Re=500 (24324^3), and Burgers on higher dimension, 64² and 128²(previously, it was 32²). Updated Table 1 compares 5 baselines and FINO over 100 inverse targets, which also expands the earlier ten-target random-subspace diagnostic.

Mean parameter RMSE:

New settingFNODINOPINONIORandFINO
NS 2D, Re=500, 64²0.15580.26820.05570.19420.23280.0262
NS 3D, Re=500, 24³0.19260.19210.19170.23430.19870.1829
Burgers, 64²0.22800.34240.22410.50030.20990.1868
Burgers, 128²0.22690.35470.27820.47200.21420.1872

The new settings use equal training epoch counts across methods within each setting. We observe:

  • Interestingly, for 2D NS at Re=500, Rand’s mean RMSE is inflated by two extreme reconstructions, substantially increasing the parameter RMSE in the 100-target average.
  • Also, FINO also achieves higher mean SSIM than Rand (0.91 versus 0.84).

FINO now achieves the lowest mean RMSE in five of the six in Table 1 settings. In particular, SSIM improved by 0.57 (FINO) versus 0.51 (random) on Burgers 64², and 0.48 versus 0.42 on 128² (FINO versus Rand).

2. Direction selection matters with limited data and high-dimensional parameters The new Burgers sweep (Figure 5) varies training-set size while keeping 8 supervised directions. FINO outperforms Rand at all training set size on the 64² and 128² grids. On 64², FINO with 32 training inputs achieves around 0.193 RMSE, outperforming Rand with 64 inputs (0.210). This directly demonstrates a benefit from selecting informative directions when training examples are limited. That is, as the dimension of the inverse problem increased and as we are limited to a smaller number of training data points, we observed greater efficiency of FINO over a random-direction supervised model.

Also, the existing seismic study provides complementary evidence for sparse supervision with limited training data. It uses only 202 synthetic training models on 256×256 velocity grids and supervises 64 directions, approximately 0.1% of a velocity field dimension. Across 20 held-out targets, FINO increases SSIM from 0.39 to 0.43 compared to FNO (Appendix C.3, Table 7). This demonstrates the feasibility of sparse directional supervision for high-dimensional inversion.

3. Offline costs are incurred once and spread across repeated inversions. Figure 5 shows the tradeoff between training-set size, reconstruction accuracy, and offline cost. On the 64² Burgers grid, FINO trained on 32 inputs achieves lower RMSE than Rand trained on 64 inputs (0.193 versus 0.210), with reported wall-clock offline costs of 11 vs 13 minutes. This comparison demonstrates how selecting informative derivative directions can improve recovery with fewer training examples, which in turn gives a smaller offline budget. Once trained, the surrogate can be reused across inverse problems for the same forward and observation setting. This makes the investment particularly relevant when training data are limited, high-dimensional and many inversions are required.

Also, the required offline investment varies with the simulator, training configuration, and supervision budget. The eight-direction Burgers sweeps report approximately 9–38 minutes of allocated offline cost, while Table 10 describes a separate NS offline cost with 400 training inputs and 400 directions. For that setting, the recorded runtimes yield a break-even after 21 inversions relative to direct numerical inversion (Appendix C.5).

4. Darcy helps clarify when Fisher-informed supervision is useful. Theorem 1 relates parameter recovery to both forward and directional-derivative errors, with Fisher eigenvalues controlling how these errors are amplified. This makes the Fisher spectrum a useful diagnostic for interpreting the value of derivative supervision. Darcy’s rapidly decaying spectrum concentrates inverse sensitivity in relatively few directions. In this setting, FINO does not improve on FNO, and Fisher and random supervision achieve similar recovery. This contrasts with the broader sensitivity spectra and larger observed gains on NS and Burgers. Together with the controlled spectral-decay experiment, the Darcy result connects our theoretical analysis to the observed problem dependence: understanding when Fisher-informed derivative supervision adds value is itself an important part of our contribution.

Your suggestion helped us test the method beyond the original benchmarks. We really appreciate your thoughtful feedback and time throughout the entire review process.

Reviewer_VszLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Fisher-Informed Neural Operators (FINO) for PDE-constrained inverse problems. The key idea is to improve inverse reconstruction by supervising the neural surrogate's directional derivatives along leading eigenvectors of an amortized Fisher information matrix, rather than focusing exclusively on forward prediction accuracy. The Fisher subspace is estimated using randomized VJPs and SVD without explicitly forming the full Jacobian. The authors further establish theoretical bounds relating forward and derivative errors to maximum-likelihood parameter recovery under affine observation models. Experiments on Navier–Stokes, Darcy flow, and seismic image-domain inversion demonstrate the potential benefits of Fisher-informed supervision, particularly for inverse problems with distributed observation sensitivity.

Critical strengths
  1. The paper identifies an important limitation of neural PDE surrogates: accurate forward predictions do not necessarily guarantee accurate inversion gradients or parameter recovery. Using Fisher information to allocate directional derivative supervision is a principled and interesting approach.

  2. The theoretical analysis provides useful insights into how forward errors, derivative errors, Fisher eigenvalues, and inverse-problem conditioning affect parameter recovery.

  3. The experiments demonstrate that FINO can substantially improve parameter recovery even when forward prediction accuracy remains nearly unchanged. In particular, FINO reduces the Navier–Stokes reconstruction RMSE by 42.8% compared with FNO.

  4. The controlled spectral-decay experiments and comparisons between field-informed and Fisher-informed supervision further support the motivation. The seismic imaging experiment also demonstrates the feasibility of sparse directional supervision in a higher-dimensional wave-equation setting.

Critical weaknesses
  1. The main recovery bounds rely on affine assumptions, whereas the considered PDE inverse problems are generally nonlinear and potentially nonconvex. The extent to which these theoretical conclusions explain the observed numerical improvements remains unclear.

  2. The effectiveness of the amortized Fisher basis depends on its alignment with local sensitivity subspaces encountered during inversion. This assumption may become restrictive when the Jacobian varies significantly across parameter space.

  3. The seismic experiment considers RTM-based image-domain inversion rather than conventional data-domain full-waveform inversion (FWI). Therefore, its conclusions regarding general seismic inverse problems should be interpreted with caution.

Questions for authors
  1. How sensitive is FINO to differences between the amortized Fisher subspace and the local Fisher subspaces encountered along inversion trajectories? Could the authors provide additional quantitative measurements of this alignment?

  2. The seismic experiment learns a velocity-to-RTM-image operator rather than the velocity-to-seismic-data mapping typically used in FWI. How directly do the reported improvements transfer to conventional data-domain FWI, where the optimization objective and sensitivity structure differ substantially?

  3. Since the motivation of FINO is to improve inversion-relevant derivatives, can the authors provide a quantitative comparison between the velocity gradients obtained through the learned RTM surrogate and those computed using the original wave-equation-based imaging operator?

2 comments

We appreciate the reviewer for recognizing our theoretical contribution and requesting alignment and gradient diagnostics. We have carefully addressed every comment and have uploaded a revised version of the manuscript. All newly added or modified text in the PDF is shown in blue.

On weakness 1: Affine theory and nonlinear inversion The key mechanism of Theorem 1 can extend to nonlinear inversion: forward and derivative errors remain separate sources of inversion gradient error. We added Appendix M to show that, when the simulator loss is strongly convex in a neighborhood containing both estimates of the simulator and the surrogate, these errors and the remaining optimization error bound the difference between surrogate and simulator reconstructions in the retained Fisher coordinates. This explains why derivative accuracy matters for local nonlinear recovery, even when forward predictions are accurate. The numerical experiments on Navier--Stokes and Burgers test this gradient mechanism directly.

On Weakness 2 and Question 1: Amortized-to-local alignment Eq. (10) and Appendix K quantify how misalignment affects the derivative accuracy, and separate amortized-to-local, Monte Carlo, and numerical-approximation errors. We've run additional experiments on Navier--Stokes, evaluating this alignment in identical initial, intermediate, and final FINO trajectory states (Table 19, Figure 17). The Fisher basis captures 99.399.3--99.499.4% of local observed-Jacobian energy, versus approximately 9.89.8% for rank-matched random bases. Final mean relative gradient error falls from 1.2591.259 (FNO) to 0.3270.327 (FINO), while signed cosine increases from 0.5910.591 to 0.9450.945. The common-residual comparison Jθ⊤rJ_\theta^\top r versus J⊤rJ^\top r also improves, isolating Jacobian action.

On Weakness 3 and Question 2: Transfer to FWI. We thank the reviewer for highlighting this distinction. We agree that our seismic experiment considers RTM-based image-domain velocity inversion rather than data-domain FWI, and we do not claim direct validation on FWI.

Our choice was motivated by the goal of evaluating FINO on high-dimensional velocity models with complex geological structures. While commonly used benchmarks like OpenFWI (Deng et al., 2022) often use 70×7070\times70 grids and five shots, our experiment considers 256×256256\times256 velocity models (65,536 parameters) and 32 shots, using only 64 supervised derivative directions (≈0.1\approx 0.1 % of the dimension). At this scale, neural-operator-based FWI introduces substantial computational challenges in learning multi-shot time-dependent waveforms and their derivatives. Image-domain inversion provides a more tractable setting by aggregating multi-shot into RTM images. To our knowledge, image-domain least-squares migration is a well-established method in seismology and commonly used in industry (Fletcher et al., 2016). Also, the relationship between velocity models and their seismic images provides a physically meaningful and visually interpretable inverse problem.

Also we wanted to note that this choice preserves a close connection to FWI. Both formulations rely on the wave-equation modeling operator and its linearization, represented by the Born operator B(m)=DmFseis(m)B(m)=D_mF_{\mathrm{seis}}(m). In particular, the averaged operator Em[B(m)⊤B(m)]\mathbb{E}_m[B(m)^\top B(m)], used to select derivative-supervision directions in our seismic experiment (Appendix C.3), is proportional to the amortized data-domain FWI Fisher information under isotropic Gaussian noise. Thus, although the image-domain and data-domain objectives differ, FINO's derivative supervision is informed by the same wave-equation sensitivity structure underlying the Gauss--Newton approximation in FWI. We are also very excited that this connection motivates a new direction for us, which is the use of our derivative-informed image-domain neural operators as preconditioners within conventional data-domain FWI, as recently explored with a related Jacobian-informed neural operator by Park (2026).

On Question 3: RTM-surrogate velocity gradients. We compare FNO and FINO (r=64r=64) directional derivatives against the ground-truth gradient obtained through the numerical simulator. On a test sample, FINO (r=64r=64) reduces mean directional relative error by 37.037.0% and increases mean signed cosine from 0.0980.098 to 0.5830.583, improving both metrics in all three directions. This provides quantitative evidence of improved agreement with the ground-truth RTM derivatives. Visual and quantitative comparisons of gradients have been added to Figure 19, Table 20 in Appendix C.11.

We are also working toward extending our experiments to FWI and are making every effort to strengthen the paper. We sincerely thank you for your feedback and would welcome any further comments or suggestions.

Thank you again for your careful reading and for encouraging us to examine more on gradient alignment and possible extension to FWI. We wanted to share a brief update following our previous response.

  1. The revised Table 1 now includes Navier–Stokes at higher Reynolds number, Re=500 in both two and three dimensions, together with Burgers on 64×64 and 128×128 grids, each evaluated over 100 inverse targets. FINO achieves the lowest mean reconstruction RMSE among the six compared methods in these four added settings. For instance, on mean parameter RMSE,
New settingFNODINOPINONIORandFINO
NS 2D, Re=500, 64²0.15580.26820.05570.19420.23280.0262
NS 3D, Re=500, 24³0.19260.19210.19170.23430.19870.1829
Burgers, 64²0.22800.34240.22410.50030.20990.1868
Burgers, 128²0.22690.35470.27820.47200.21420.1872
  1. We also added a Burgers training-set-size sweep (Figure 5). The intuition is that Fisher directions focus derivative supervision on parameter changes that matter for the observations, helping the model extract more inversion-relevant information from each training example. This is particularly valuable when we are limited to a small amount of training dataset. The sweep illustrates this benefit: on the 64² grid, FINO trained on only 32 inputs achieves lower RMSE than Rand trained on 64 inputs (approximately 0.193 versus 0.210), with both methods supervising eight directions. The corresponding allocated wall-clock offline costs are 11 versus 13 minutes. Thus, informative direction selection can reduce the amount of training data needed for accurate recovery and, in this comparison, achieve better recovery with a smaller offline budget.

  2. Following the latest manuscript revision, we would like to clarify the updated references for the diagnostics discussed in our earlier response. The alignment and gradient diagnostics under “Weakness 2 and Question 1: Amortized-to-local alignment” appear in Appendix C.9 (Figure 18 and Table 19). The RTM directional-derivative comparison under “Question 3: RTM-surrogate velocity gradients” appears in Appendix C.11 (Table 20 and Figure 20).

Your questions helped us clarify both the evidence supporting FINO and the scope of our conclusions. Thank you for the time you have spent evaluating the revision.

Comments on the paper

1 comment

We thank all reviewers for their careful reading and constructive feedback. Your suggestions helped us strengthen the experimental evaluation, clarify the connection between our theoretical analysis and practical inversion, and explore how our method can be used inside the probabilistic framework.

In the latest revision, we added higher Reynolds number Navier--Stokes benchmark and three-dimensional Navier–Stokes experiments, together with Burgers experiments on finer grids (Table 1). We also added a training set size sweep (Figure 5), showing how informative derivative directions can extract more useful inversion information from fewer training data samples, and in the reported comparison, achieve better recovery with a smaller offline budget. Together with the contrasting Darcy results and our theoretical analysis, these experiments help clarify when Fisher-informed supervision is most useful.

We have updated the manuscript references and provided reviewer-specific responses explaining the changes. We really appreciate all reviewers for the time and thought they have invested in this discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.