acceptodds
Under review as a conference paper at ICLR 2027

PIVOT: Physics-Informed Virtual Probing for Output-Space Tuning of Neural Operators

18 views

Abstract

Partial differential equations underpin complex physical modeling, but costly numerical solvers limit their use in applications requiring real-time computation. Neural operators offer efficient surrogates, yet distribution shifts can compromise physical consistency and accumulate errors over long rollouts. We find that frozen operators can retain reliable local physical knowledge despite inaccurate predictions: output changes under small, physically admissible input perturbations can still closely reflect the true system’s local responses. Existing adaptation methods typically update parameters or learn auxiliary corrections, underusing this retained physical information. We introduce PIVOT, a general adaptation framework that perturbs frozen operators’ inputs to extract local physical responses for prediction correction. For each target sample, PIVOT learns physically admissible directions and queries the operator under small, opposite perturbations. Comparing paired predictions extracts response directions and magnitudes as candidates for correction. Physical consistency checks assess their reliability, while a lightweight adapter learns weights at each spatial location to combine responses into corrections. PIVOT bounds correction magnitude and progressively reduces updates that increase physical residuals, keeping the pretrained operator fully frozen. Experiments on APEBench and RealPDEBench show improved physical consistency, predictive accuracy, and long-horizon stability across diverse physical systems and distribution shifts. The complete model implementation is provided in the supplementary material.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Physics-Informed Adaptation, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_cy7HOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes PIVOT a physics guided neural surrogate adaptation method that learns a probe controller and a local spatial response model to correct the outputs of the surrogate to have lower error and lower PDE residuals. By incorporating physics information, PIVOT outperforms the baseline adaptation methods when doing extrapolation on a variety of benchmarks.

Critical strengths
  • The PIVOT experimental results indicate that is it robust to cross-domain extrapolation and the ablation shows that each component of the method leads to real improvement in the final performance.
Critical weaknesses
  • PIVOT is compared to baselines that mostly do not have access to physics information. Besides POTT and F-Adapter, the other baselines are generic methods that don't receive the physics residual information as input. I think a comparison with a method that directly does physics informed output correction would highlight the advantages of probing based correction vs physics informed output correction.

  • The method is difficult to follow from the manuscript text and took a significant amount of effort to understand. There are many components to the correction process that are explained at a high level, but the details are missing or unclear. For example, the training and inference process differ but it's hard to parse which components are part of training and inference. A potential way to make this clearer would be to add a training algorithm and an inference algorithm to the appendix detailing how each component works and fits together.

Questions for authors

Does virtual probing outperform alternative ways of exploiting the same target physics, given the same frozen operator and target observations? Comparing PIVOT to a physics-informed LoRA or another physics-guided adapter that uses the same frozen surrogate will answer this question. For example the paper A Benchmark and Physics-Informed Optimization of LoRA for SAR Target Recognition proposes a physics informed adaptation method called Pi-ConvDoRA.

Reviewer_obMhOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses prediction degradation and physical inconsistency in pretrained neural operators under distribution shift. It proposes PIVOT, which uses responses to physically admissible input perturbations to correct predictions while keeping the backbone frozen. The method learns spatially varying response combinations and attenuates or rejects corrections that worsen monitored physical residuals. Experiments cover six datasets from APEBench and RealPDEBench using FNO, CNO, and Transolver, with comparisons against adaptation methods and other neural operators. Additional experiments examine diffusion shifts and simulation-to-experiment transfer with Reynolds-number extrapolation. Validation compares local responses against numerical solvers and evaluates prediction errors, spatial derivatives, PDE residuals, balance violations, frequency, energy, and distributional discrepancies. Ablations support the importance of physical acceptance, showing that similar prediction accuracy can coexist with substantially different physical consistency.

Critical strengths
  1. The evaluation distinguishes predictive accuracy from physical consistency by measuring PDE residuals, balance violations, frequency and energy errors, and distributional discrepancies alongside prediction errors. The physical acceptance ablation is particularly informative, showing that similar predictive accuracy can coexist with substantially different physical fidelity.

  2. The experiments demonstrate applicability across six datasets and three backbone architectures, with comparisons against both general-purpose adapters and physics-oriented adaptation methods. Evaluations under diffusion-coefficient shifts and simulation-to-experiment transfer further broaden the evidence for adaptation across different physical conditions.

  3. The method includes an explicit safeguard that reduces correction magnitudes or returns the original prediction when a proposed update fails the monitored physical-residual criteria. The theoretical analysis clarifies the conditions under which this safeguard bounds residual deterioration, making the scope of its guarantee explicit.

Critical weaknesses
  1. PIVOT introduces substantial computational overhead, requiring five backbone evaluations per prediction step. Although it reduces peak memory usage relative to DoRA in the Cylinder/CNO experiment, training time increases from 9.0 to 36.3 hours, highlighting the distinction between freezing backbone parameters and achieving computational efficiency.

  2. Direct validation of the proposed mechanism is limited to a subset of the datasets, since simulator responses to perturbed inputs are unavailable for Cylinder Flow and FSI. Performance improvements on these datasets therefore provide indirect evidence of the mechanism’s utility, but do not establish the accuracy of the extracted local responses.

  3. The method requires paired target-domain observations for training and access to physical constraints during inference. Its applicability when governing laws or necessary physical parameters are only partially known remains to be established.

  4. PIVOT’s performance advantage depends on the evaluation metric and comparator: on cross-domain Burgers, it outperforms the evaluated FNO-based adapters, while MemNO achieves lower prediction error. Its strengths are therefore better characterized as improved adaptation of existing backbones and stronger performance on selected physical diagnostics than as uniformly superior predictive accuracy.

Questions for authors
  1. Under what conditions do the frozen operator’s local responses become unreliable? How do response accuracy and PIVOT’s gains change as distribution shifts become larger or source-model quality decreases, and how strongly does response error correlate with downstream prediction improvement?

  2. Does PIVOT retain its advantages when training time or inference cost is matched across methods? How does the number of probes affect prediction accuracy, physical consistency, and inference latency?

  3. How robust is PIVOT to noisy observations, misspecified physical constraints, or inaccurate physical parameters? Could these uncertainties cause the physical acceptance check to reject useful corrections or accept harmful ones?

  4. Does PIVOT outperform a direct output-correction network trained with the same target data, a comparable trainable parameter budget, and the same physical acceptance check? Could this comparison isolate the contribution of the frozen operator’s local responses beyond learned output correction and physical acceptance alone?

Reviewer_Aye9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose PIVOT, a general framework for output-space adaptation of neural PDE operators that directly incorporates constraints from the target physics into the adaptation process. PIVOT builds on the observation that frozen neural operators may retain reliable local responses even when their absolute predictions are inaccurate. The method learns physically admissible perturbation directions and probes the inputs using small, opposite perturbations. Experiments on APEBench and RealPDEBench, covering diverse physical systems and distribution shifts, demonstrate improvements in predictive accuracy, physical consistency, and long-horizon rollout stability.

Critical strengths
  1. The observation that learned neural networks may retain useful local continuity or robustness properties, even when their predictions are inaccurate, is interesting and provides a promising motivation for the proposed approach.

  2. The paper is well structured, and the proposed methods are generally sound and technically valid.

Critical weaknesses
  1. Too many heuristic designs in the proposed method and hard to follow what design is important. The presentation of the core method is mixed with numerous implementation details, making it difficult to identify the central technical contribution and distinguish it from auxiliary design choices.

  2. Mismatch between the motivating hypothesis and the implemented mechanism. The paper is motivated by the observation that frozen neural operators may retain reliable local responses even when their absolute predictions are inaccurate. However, the proposed method subsequently applies extensive feasibility constraints, physics-based reliability checks, response attenuation, orthogonalization, and learned spatial reweighting, including the possibility of reversing response directions. This raises the question of whether PIVOT genuinely exploits the fidelity of the pretrained operator's local sensitivities or instead uses its perturbation responses as a flexible set of candidate correction fields.

  3. Comparisons with simpler adaptation strategies are needed. Although the proposed method outperforms the reported baselines, given its methodological complexity, I would expect comparisons with the following three straightforward alternatives:

    • Frozen backbone: How well does the pretrained operator perform without any adaptation?
    • Full fine-tuning: How much improvement can be achieved when all model parameters are trainable?
    • Simple output correction: What happens if the operator is frozen and a small residual CNN is trained directly on its predictions?

    These comparisons would help clarify whether the additional complexity of PIVOT is justified.

Questions for authors
  1. Assumptions on the pretrained model: Are there any explicit assumptions regarding the pretrained operator, particularly concerning the accuracy or reliability of its local first-order derivatives?

  2. Definition of perturbation lengths: It would be helpful to provide an explicit mathematical definition of hk±h_k^\pm in Eq. (2), particularly how the signed projections are computed under RMS-normalized perturbation directions.

Reviewer_sGf4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PIVOT, a method for adapting a frozen pretrained neural operator by adjusting its outputs rather than retraining the entire model. It probes the operator with small paired input perturbations and differences the outputs to obtain local response fields. A small CNN then combines these responses with spatially varying weights to correct the baseline prediction. At inference, the correction is scaled down or rejected if it increases a physical residual.

Critical strengths

The paper is well written and the empirical analysis is well designed. To be specific, evaluation separates accuracy from physical consistency, showing that the adapters with near-identical field error can differ substantially in physical behavior. Moreover, the core observation where frozen operators retain accurate local responses under shift despite degraded predictions, is verified directly against a numerical solver rather than inferred from downstream gains.

Critical weaknesses
  1. Physical consistency gains are not isolated to the main contribution. The ablation shows most of the Balance Error gap comes from the physical acceptance step, which is a generic post-hoc check that could be applied to any adapter. Since no baseline receives this step, the comparison cannot show what probing itself contributes to physical consistency.

  2. On KS, every PEFT adapter is at or worse than the frozen backbone for both CNO and Transolver (cf. Table 1, 10, 11), which needs explanation. The main text shows only favorable cases. Table 2 reports KS where PIVOT wins, while on Burgers MemNO is 19 times better (cf. Table 7).

Questions for authors
  1. What are the Balance Error and PDE Defect of the baseline adapters when the same physical acceptance step is applied to their outputs?
  2. Why do PEFT adapters perform worse than the frozen backbone on KS?
  3. What explains the large difference in gains across backbones on KS (CNO 5.5x vs. FNO 2%)? Under what conditions does PIVOT not help?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.