- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a new method that uses Fisher information derived regulation to help the training of neural operators in the settings of sparse observed parameter recovery. It uses Fisher information to select the directions in which observations affect the parameters most. Their estimation is based on parameter sampling and random vector probing. The theory shows that the first r Fisher eigenvectors span a space that minimizes the maximum directional variance in constrained inversion compared to other r-dimensional subspaces. They also provide some error bound. Performance-wise, it does not generally perform better.
Critical strengths
- The novelty in using likelihood-parameter subsampled Jacobian information to help full-field reconstruction
- Theoretical work on the correctness and effectiveness of FINO
- Well-written and reporting negative results faithfully
Critical weaknesses
- The lack of baselines. There are many previous methods in sparse parameter recovery beyond FNO and PINO, such as Neural Inverse Operator and posterior sampling methods (DiffusionPDE, FunDPS, DDIS, etc).
- The weak performance. In Table 1, FINO outperforms (limited) baselines in just 1/4 settings. Considering the additional compute overhead (20 hours compared to PINO's 0.5 hours in NS equations), it's hard to justify this method.
- The theory does not consider the error from estimating the Fisher subspace. It would be good to see how sampling budget and random approximation affect the accuracy and efficiency.
Questions for authors
- Could the uncertainty due to limited observations be handled in this framework? It seems that FINO can produce only one solution with no uncertainty estimation.
- How's Fisher's subspace compared to a random subspace? It would be interesting to see the performance difference.
- Does the result converge as the M and l used for approximating the Fisher matrix become larger and larger? What's the relation of random budget vs accuracy?
3 comments
Thank you for the reply. It addressed some but not all of my concern. It mostly addressed weakness 1 and 3. On weakness 2, it's still hard to justify the 20-hour overhead with marginal accuracy improvement. What's more, on question 2, the random subspaces are almost equal to FINO on the Darcy and the Burgers equations, which limits the advantage of this method to just NS. I believe that NS forward/inverse problem is a competitive field, and the current benchmarks are not enough. Therefore, I remain my score.
Thank you very much for your engagement and response! We understand your remaining concerns and your comment motivated four additional benchmark settings and a more direct evaluation of Fisher versus random supervision.
1. The benefits now extend beyond the original NS (Re=100) case. We added 2D Navier–Stokes (NS) at higher Reynolds number, Re=500, 3D NS at Re=500 (), and Burgers on higher dimension, 64² and 128² grids (previously, it was 32²). Updated Table 1 compares 5 baselines and FINO over 100 inverse targets, and expands the earlier 10 target random subspace diagnostic.
Mean parameter RMSE:
| New setting | FNO | DINO | PINO | NIO | Rand | FINO |
|---|---|---|---|---|---|---|
| NS 2D, Re=500, 64² | 0.1558 | 0.2682 | 0.0557 | 0.1942 | 0.2328 | 0.0262 |
| NS 3D, Re=500, 24³ | 0.1926 | 0.1921 | 0.1917 | 0.2343 | 0.1987 | 0.1829 |
| Burgers, 64² | 0.2280 | 0.3424 | 0.2241 | 0.5003 | 0.2099 | 0.1868 |
| Burgers, 128² | 0.2269 | 0.3547 | 0.2782 | 0.4720 | 0.2142 | 0.1872 |
The new settings use equal training epoch counts across methods within each setting. We observe:
- For 2D NS at Re=500, Rand’s mean RMSE is inflated by two extreme reconstructions, substantially increasing the parameter RMSE in the 100-target average.
- Also, FINO also achieves higher mean SSIM than Rand (0.9078 versus 0.8419).
FINO now achieves the lowest mean RMSE in five of the six in Table 1 settings. In particular, SSIM improved by 0.57 (FINO) versus 0.51 (random) on Burgers 64², and 0.48 versus 0.42 on 128² (FINO vs Rand).
2. Direction selection matters with limited data and high-dimensional parameters The new Burgers sweep (Figure 5) varies training-set size while keeping 8 supervised directions. FINO outperforms random directions (Rand) at all training set size on the 64² and 128² grids. On 64², FINO with 32 training inputs achieves around 0.193 RMSE, outperforming Rand with 64 inputs (0.210). This directly demonstrates a benefit from selecting informative directions when training examples are limited. That is, as the dimension of the inverse problem increased and as we are limited to a smaller number of training data points, we observed greater efficiency of FINO over a random-direction supervised model.
Also, the existing seismic study provides complementary evidence for sparse supervision with limited training data. It uses only 202 synthetic training models on 256×256 velocity and supervises 64 directions, approximately 0.1% of a velocity field dimension. Across 20 held-out targets, FINO increases SSIM from 0.39 to 0.43 compared to FNO (Appendix C.3, Table 7). This demonstrates the feasibility of sparse directional supervision for high-dimensional inversion.
3. Offline costs are incurred once and spread across repeated inversions. Figure 5 shows the tradeoff between training-set size, reconstruction accuracy, and offline cost. On the 64² Burgers grid, FINO trained on 32 inputs achieves lower RMSE than Rand trained on 64 inputs (0.193 versus 0.210), with reported wall-clock offline costs of 11.00 vs 13.51 minutes. This comparison demonstrates how selecting informative derivative directions can improve recovery with fewer training examples, which in turn gives a smaller offline budget. Once trained, the surrogate can be reused across inverse problems for the same forward and observation setting. This makes the investment particularly relevant when representative training data are limited, high-dimensional and many inversions are required.
Also, the required offline investment varies with the simulator, training configuration, and supervision budget. The eight direction Burgers sweeps report approximately 9–38 minutes of offline cost, while Table 10 describes an offline cost for NS with 400 training inputs and 400 supervised directions. For that setting, the recorded runtimes yield a break-even after 21 inversions relative to direct numerical inversion (Appendix C.5).
4. Darcy helps clarify when Fisher-informed supervision is useful. Theorem 1 relates parameter recovery to both forward and directional-derivative errors, with Fisher eigenvalues controlling how these errors are amplified. This makes the Fisher spectrum a useful diagnostic for interpreting the value of derivative supervision. Darcy’s rapidly decaying spectrum concentrates inverse sensitivity in relatively few directions. In this setting, FINO does not improve on FNO, and Fisher and random supervision achieve similar recovery. This contrasts with the broader sensitivity spectra and larger observed gains on NS and Burgers. Together with the controlled spectral-decay experiment, the Darcy result connects our theoretical analysis to the observed problem dependence: understanding when Fisher-informed derivative supervision adds value is itself an important part of our contribution.
We appreciate your time, continued engagement and thoughtful comments throughout the review process.
We want to thank the reviewer and appreciate your suggestion on adding more baselines, and recognizing our theoretical contribution. We have carefully addressed every comment and have uploaded a revised version of the manuscript. All newly added or modified text in the PDF is shown in blue.
On weakness 1 and question 1: Baselines, scope and uncertainty On top of our three baselines, we have added NIO, DDIS (with FNO or with FINO) evaluations (Table 1 and 12-16). Uncertainty estimation is not the main objective of our method. FINO aims to answer ``What is the role of a deterministic neural network's Jacobian in solving gradient-based inverse problems''. Thus, we do not claim that our method gives better uncertainty estimation, and comparing with probabilistic baselines is not an apples-to-apples comparison.
In DDIS (Lin et al., 2026), we replace FNO with FINO while keeping the diffusion prior and paired sampling protocol fixed. On one Navier--Stokes target with 256 noiseless observations and 16 samples, FINO improves ensemble-mean RMSE in 16/18 guidance-budget settings, SSIM in all 18, and simulator-evaluated residual in 17/18. At D=4, FINO with 500 guidance updates nearly matches FNO with 1,000 updates in RMSE (0.03412 versus 0.03405). FINO also yields tighter ensembles in all 18 settings, demonstrating that its use is not restricted to a single point estimate. These preliminary results, obtained while diffusion-prior training is ongoing, support improved limited-budget sampling-based recovery.
Separately, we tested FINO's recovery under noise. On 100 Burgers test samples, FINO improves point-recovery RMSE over FNO at both 1% noise (0.1879 versus 0.2454) and 5% noise (0.1682 versus 0.2060) (Table 17).
On weakness 2: Performance depends on Fisher spectrum Our theoretical analysis (in Section 4.3, Figure 4 and 6) identifies when the Fisher-informed training is useful in surrogate-based inversion. The explanation follows as:
i) Experiments are designed to show that even if forward errors are similar, inversion performance is different. Eq. (7) weights residual-dependent derivative error by δr−2 and forward error by δr−1. This explains why forward accuracy alone is insufficient. In NS, FNO and FINO have nearly equal forward validation errors (±0.2%), yet FINO reduces reconstruction RMSE by 42.8% versus FNO and 12.6% versus PINO, despite PINO's lower forward error.
ii) New experimental results on a new PDE inversion As Burgers' inverse problem's spectrum decays slowly, FINO's usefulness is demonstrated not only in the inversion, but also forward prediction, as the forward and inverse relationship are maintained through slowly decaying Jacobian. Darcy has to remain as a negative result as its Fisher spectrum decays fastest, according to our theoretical analysis in Section 4.3 and Theorem 1. Thus, combined with our spectral diagnostics in Figure 6, the expected FINO's success is most pronounced in this order: Burgers, followed by Navier--Stokes, and Darcy.
On question 3: Cost and sampling budget. Our contribution is a matrix-free algorithm, which makes previously infeasible computation, feasible. Section 3.1 and Appendix B.5 specifies Ml VJP directions for basis construction and NLm Jvp directions for output labels. With d input and o output coordinates, directional labels require O(NLom) entries rather than O(NLod) for full Jacobian labels. The factor d/m is 10.24 for Navier-Stokes and 81.92 for Darcy. Repeated inversion can amortize the cost (Appendix C.5).
The added Appendix L derives E∥H−Hˉ∥F2=A/M+B/(Mℓ), for independent inputs and Gaussian probes with finite Fisher second moments; A and B quantify input and probe variability. With a positive boundary eigengap, matrix perturbation bounds transfer this estimation error to the leading-subspace error. Increasing ℓ leaves the A/M floor; increasing M at fixed ℓ gives mean-square consistency. For NS, increasing ℓ from 4 to 256 raises capture from 63.2% to 97.9% of the optimal rank-64 empirical capture (Table 18).
On weakness 3: Fisher-subspace estimation error. Eq. 10 bounds transfer via supervised derivative error and subspace mismatch; Appendix K separates amortization, Monte Carlo, and numerical errors. New measurements on three NS find 99.3--99.4% local observed-sensitivity capture along FINO trajectories, versus approximately 9.8% for rank-matched random bases. At final states, mean relative gradient error is 0.327 for FINO versus 1.259 for FNO (Table 19).
On question 2. Fisher versus random directions. Table 12 reports lower mean reconstruction RMSE for FINO than rank-matched random direction supervision in all three PDE comparisons. This supports selecting derivative directions according to observation sensitivity to improve inverse recovery.