- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper shows adaptation capacity from parameter accessibility. It proposes a short unrestricted warm-up, followed by groupwise functional and loss-based scoring, to select where a parameter-efficient adapter should be placed. Experiments examine adaptation under physical shifts in two models.
Critical strengths
-
The proposed interventions are useful. They show that increasing rank within an unsuitable parameter group can saturate, while a smaller adapter placed in a useful group can perform better.
-
The paper includes informative boundaries and controls. These include reversals at very small budgets, diffuse DeepONet regimes, and failures of a single-gradient placement diagnostic.
Critical weaknesses
-
The incremental value of target-aware selection is not sufficiently established on FNO. The probe selects the spectral group for all six shifts. These experiments strongly support adapting spectral parameters, but provide weaker evidence that placement must be selected for each target.
-
The DeepONet results do not resolve this concern. Branch placement outperforms trunk placement does not establish that the proposed selection strategy improves over a reasonable broad-placement baseline.
Questions for authors
- After accounting for probe overhead, how much improvement does target-aware selection provide over fixed spectral placement?
- Is there a setting within the same architecture where the optimal parameter group changes across target shifts and the probe reliably captures that change?
4 comments
Follow-up to Reviewer 6B5b
We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the need for target-aware selection on FNO. In a new target-functional test, the same FNO backbone is retrained for . The probe then selects the projection group in 3/3 seeds, and projection is the best of four placements at 25% of the target data (0.0732 against 0.1041 for fixed spectral placement; Section 6.3, Table 6). The placement decision follows the target and the amount of target data. The rebuttal also covers the comparison with a broad placement, which is now reported at matched budget in both directions. On FNO the broad arm is 8.4% and 13.7% worse than the concentrated spectral arm (Table 7), and on the diffuse DeepONet shift the probe spreads the budget (Section 6.4, Table 4). We hope these additions give a fuller picture of the work.
We would also like to highlight the main result of the paper. Parameter-efficient fine-tuning with LoRA-type adapters is already the standard way to adapt large pretrained models, and it is now applied to neural operators with the placement conventions of language models. The paper shows that two constraints govern PEFT for neural operators, accessibility and capacity. When adaptation is restricted away from the useful parameter group, an error floor remains that additional rank cannot remove. On FNO, a standard adapter placed where the probe points reaches 1.05-1.39x the full fine-tuning error at 2.98% of the trainable parameters, and it beats standard LoRA, DoRA and AdaLoRA at matched budget on all six shifts. These results are, to the best of our knowledge, new in the literature on fine-tuning neural operators. Your review notes that increasing rank within an unsuitable parameter group can saturate, while a smaller adapter placed in a useful group can perform better, and we hope the new controls complete this picture.
We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.
Thank you for your time and consideration. We look forward to your reply.
Thank you for the clarification and additional experiments, which address my concerns. I have therefore raised my score. However, I do not have sufficient expertise to assess this submission, so I ask the AC to consider my assessment alongside those of reviewers with greater expertise.
Thank you for carefully reading our rebuttal and updating your score. Your review was among the most insightful and constructive we received, and your questions directly motivated new experiments that strengthened our paper. We particularly appreciate your willingness to engage with the new evidence and reconsider your assessment. We believe this reflects genuine expertise and scientific rigor, and we are grateful for your thoughtful approach to peer review.
Response to Reviewer 6B5b
We thank the reviewer for their comments.
W1: target-aware selection on FNO
On FNO the probe's contribution is the decision, not a higher accuracy, and the paper states this in the abstract and the conclusion. Against the standard fixed placements at matched budget - LoRA, DoRA and AdaLoRA - probe-guided placement is better on all six shifts, and (IA)3 is beaten at its native smaller budget (Section 6.2, Table 2). Against a DoRA adapter fixed on the spectral group with a 25% smaller budget, the fixed placement is better on five of the six shifts (0.01868 against 0.01990; viscosity, 0.0057 against 0.0068), and we report it as the stronger within-subspace adapter. What the probe adds is that it reaches the same group from target data alone, with no per-architecture prior, and the same rule carries to DeepONet and to a changed target functional. Section 6.2 keeps those two comparisons separate, and the abstract and Section 7 carry the concession.
You grant that the experiments strongly support adapting spectral parameters; the paper now states that explicitly, so the claim is the decision rather than an accuracy gain.
W2: the DeepONet results do not settle the comparison
The broad arm is now measured in both directions. On FNO, where the spectral group dominates, a matched-budget broad arm is 8.4% and 13.7% worse than the concentrated spectral arm on the viscosity and initial-condition shifts (Table 7). On DeepONet the ordering reverses: broad/branch is 0.4106/0.4101 (+0.1%), 0.2096/0.2159 (-2.9%) and 0.0953/0.1036 (-8.0%) at matched budget (Section 6.4, Table 4). The branch is where the full-FT displacement localizes on the two localized shifts, but the trunk's fixed Fourier basis needs little task-specific adaptation, so spreading is at least as effective there. A fixed broad strategy would lose the FNO shifts and a fixed concentrated strategy loses the diffuse DeepONet shift, which is the call the probe makes from target data. Section 6.4 and App. K state this.
Q1: gain over fixed spectral placement after probe overhead
Net of overhead the FNO comparison does not change: the probe costs about 0.3 s, 2-3% of a 10-16 s fine-tuning run (App. A.2), while DoRA on the pointwise group, the fixed-placement baseline it is measured against, is 1.7-2.3x worse on the two original shifts (Section 6.2, Table 2), and that cost is charged in the accounting of App. B.3. The probe uses only the target training split and a validation probe split, so no test labels and no full-FT solution enter the placement decision. Read against fixed spectral placement, the accuracy gain is absent: the probe selects the group a fixed rule would. Read against the standard placements at matched budget, the probe-guided arm wins on every shift (Section 6.2, Table 2).
Q2: reliability of the captured change
The target-functional test is that setting, and it is a new run matrix rather than a reanalysis. We retrain the same FNO backbone for y=u(x,T)2 with the source inputs and splits, so only the target functional changes, and the probe selects the projection group in 3/3 seeds (C=0.277). At 25% of target data projection is the best of four placements (0.0732 against 0.1041 for fixed spectral). At 100% the fixed spectral arm takes the lead (0.0351 against 0.0442), so the choice follows the target and the amount of target data rather than the architecture. Section 6.3 reports the test under "Target-functional probe test", Table 6 (App. L) gives the sweep, and contribution 4 in Section 1 states the result.
On reliability, which is the part your question adds: the projection choice is the same in every seed (C=0.277±0.001), and that cell sits far above the placement threshold, so the call is not a margin artifact.