acceptodds
Under review as a conference paper at ICLR 2027

Learning Where to Adapt: Target-Aware Parameter-Efficient Fine-Tuning for Neural Operators

Abstract

Parameter-efficient fine-tuning (PEFT) adapts pretrained neural operators to new physical regimes with a small trainable budget, but performance depends on where that budget is placed. We show that neural-operator PEFT is governed by two distinct constraints: accessibility, whether the trainable parameterization can reach useful target updates, and capacity, how much of those updates it can represent. Across controlled physical shifts, useful updates concentrate in different architectural parameter groups—the spectral kernels of a Fourier Neural Operator and the branch network of a DeepONet. Once capacity is adequate, restricting adaptation away from these updates imposes an error floor that additional rank cannot remove. We turn this observation into a cheap target-aware placement rule: a short multi-batch warm-up on target training data, scored by groupwise functional and task-loss improvement, identifies useful parameter groups and detects diffuse regimes where concentrated placement is unwarranted. On FNO, placing a standard adapter where the probe points reaches – the full-fine-tuning error using 3–12% of the trainable degrees of freedom and outperforms the tested fixed-placement and adaptive-rank PEFT baselines at matched budget on all six shifts. On DeepONet, the probe identifies branch-localized adaptation while assigning low concentration to diffuse regimes. Increasing PEFT capacity cannot compensate for useful target updates that its trainable parameterization cannot access.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, parameter-efficient fine-tuning, scientific machine learning, physical distribution shift, model adaptation, physics-informed machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6B5bOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows adaptation capacity from parameter accessibility. It proposes a short unrestricted warm-up, followed by groupwise functional and loss-based scoring, to select where a parameter-efficient adapter should be placed. Experiments examine adaptation under physical shifts in two models.

Critical strengths
  • The proposed interventions are useful. They show that increasing rank within an unsuitable parameter group can saturate, while a smaller adapter placed in a useful group can perform better.

  • The paper includes informative boundaries and controls. These include reversals at very small budgets, diffuse DeepONet regimes, and failures of a single-gradient placement diagnostic.

Critical weaknesses
  • The incremental value of target-aware selection is not sufficiently established on FNO. The probe selects the spectral group for all six shifts. These experiments strongly support adapting spectral parameters, but provide weaker evidence that placement must be selected for each target.

  • The DeepONet results do not resolve this concern. Branch placement outperforms trunk placement does not establish that the proposed selection strategy improves over a reasonable broad-placement baseline.

Questions for authors
  • After accounting for probe overhead, how much improvement does target-aware selection provide over fixed spectral placement?
  • Is there a setting within the same architecture where the optimal parameter group changes across target shifts and the probe reliably captures that change?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 6B5b

We thank the reviewer for their comments.

W1: target-aware selection on FNO

"The incremental value of target-aware selection is not sufficiently established on FNO. The probe selects the spectral group for all six shifts."

On FNO the probe's contribution is the decision, not a higher accuracy, and the paper states this in the abstract and the conclusion. Against the standard fixed placements at matched budget - LoRA, DoRA and AdaLoRA - probe-guided placement is better on all six shifts, and (IA)3^3 is beaten at its native smaller budget (Section 6.2, Table 2). Against a DoRA adapter fixed on the spectral group with a 25% smaller budget, the fixed placement is better on five of the six shifts (0.01868 against 0.01990; viscosity, 0.0057 against 0.0068), and we report it as the stronger within-subspace adapter. What the probe adds is that it reaches the same group from target data alone, with no per-architecture prior, and the same rule carries to DeepONet and to a changed target functional. Section 6.2 keeps those two comparisons separate, and the abstract and Section 7 carry the concession.

You grant that the experiments strongly support adapting spectral parameters; the paper now states that explicitly, so the claim is the decision rather than an accuracy gain.

W2: the DeepONet results do not settle the comparison

"The DeepONet results do not resolve this concern. Branch placement outperforms trunk placement does not establish that the proposed selection strategy improves over a reasonable broad-placement baseline."

The broad arm is now measured in both directions. On FNO, where the spectral group dominates, a matched-budget broad arm is 8.4% and 13.7% worse than the concentrated spectral arm on the viscosity and initial-condition shifts (Table 7). On DeepONet the ordering reverses: broad/branch is 0.4106/0.4101 (+0.1%), 0.2096/0.2159 (-2.9%) and 0.0953/0.1036 (-8.0%) at matched budget (Section 6.4, Table 4). The branch is where the full-FT displacement localizes on the two localized shifts, but the trunk's fixed Fourier basis needs little task-specific adaptation, so spreading is at least as effective there. A fixed broad strategy would lose the FNO shifts and a fixed concentrated strategy loses the diffuse DeepONet shift, which is the call the probe makes from target data. Section 6.4 and App. K state this.

Q1: gain over fixed spectral placement after probe overhead

"After accounting for probe overhead, how much improvement does target-aware selection provide over fixed spectral placement?"

Net of overhead the FNO comparison does not change: the probe costs about 0.3 s, 2-3% of a 10-16 s fine-tuning run (App. A.2), while DoRA on the pointwise group, the fixed-placement baseline it is measured against, is 1.7-2.3x worse on the two original shifts (Section 6.2, Table 2), and that cost is charged in the accounting of App. B.3. The probe uses only the target training split and a validation probe split, so no test labels and no full-FT solution enter the placement decision. Read against fixed spectral placement, the accuracy gain is absent: the probe selects the group a fixed rule would. Read against the standard placements at matched budget, the probe-guided arm wins on every shift (Section 6.2, Table 2).

Q2: reliability of the captured change

"Is there a setting within the same architecture where the optimal parameter group changes across target shifts and the probe reliably captures that change?"

The target-functional test is that setting, and it is a new run matrix rather than a reanalysis. We retrain the same FNO backbone for y=u(x,T)2y=u(x,T)^2 with the source inputs and splits, so only the target functional changes, and the probe selects the projection group in 3/3 seeds (C=0.277C=0.277). At 25% of target data projection is the best of four placements (0.0732 against 0.1041 for fixed spectral). At 100% the fixed spectral arm takes the lead (0.0351 against 0.0442), so the choice follows the target and the amount of target data rather than the architecture. Section 6.3 reports the test under "Target-functional probe test", Table 6 (App. L) gives the sweep, and contribution 4 in Section 1 states the result.

On reliability, which is the part your question adds: the projection choice is the same in every seed (C=0.277±0.001C = 0.277 \pm 0.001), and that cell sits far above the placement threshold, so the call is not a margin artifact.

AuthorsOpenReview ↗

Follow-up to Reviewer 6B5b

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the need for target-aware selection on FNO. In a new target-functional test, the same FNO backbone is retrained for y=u(x,T)2y = u(x,T)^2. The probe then selects the projection group in 3/3 seeds, and projection is the best of four placements at 25% of the target data (0.0732 against 0.1041 for fixed spectral placement; Section 6.3, Table 6). The placement decision follows the target and the amount of target data. The rebuttal also covers the comparison with a broad placement, which is now reported at matched budget in both directions. On FNO the broad arm is 8.4% and 13.7% worse than the concentrated spectral arm (Table 7), and on the diffuse DeepONet shift the probe spreads the budget (Section 6.4, Table 4). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. Parameter-efficient fine-tuning with LoRA-type adapters is already the standard way to adapt large pretrained models, and it is now applied to neural operators with the placement conventions of language models. The paper shows that two constraints govern PEFT for neural operators, accessibility and capacity. When adaptation is restricted away from the useful parameter group, an error floor remains that additional rank cannot remove. On FNO, a standard adapter placed where the probe points reaches 1.05-1.39x the full fine-tuning error at 2.98% of the trainable parameters, and it beats standard LoRA, DoRA and AdaLoRA at matched budget on all six shifts. These results are, to the best of our knowledge, new in the literature on fine-tuning neural operators. Your review notes that increasing rank within an unsuitable parameter group can saturate, while a smaller adapter placed in a useful group can perform better, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Thank you for the clarification and additional experiments, which address my concerns. I have therefore raised my score. However, I do not have sufficient expertise to assess this submission, so I ask the AC to consider my assessment alongside those of reviewers with greater expertise.

Thank you for carefully reading our rebuttal and updating your score. Your review was among the most insightful and constructive we received, and your questions directly motivated new experiments that strengthened our paper. We particularly appreciate your willingness to engage with the new evidence and reconsider your assessment. We believe this reflects genuine expertise and scientific rigor, and we are grateful for your thoughtful approach to peer review.

Reviewer_csFuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the limitations of fixed adapter placement in parameter-efficient fine-tuning (PEFT) for neural operators. The authors distinguish two key constraints: parameter accessibility and adaptation capacity, arguing that increasing LoRA rank cannot compensate for adapting the wrong parameter groups. To address this issue, they propose a target-aware placement method that uses a short warm-up probe and groupwise functional and loss-based scores to identify suitable adaptation locations. The method also determines whether adaptation should be concentrated or distributed across modules. Experiments on Fourier Neural Operators (FNO) and DeepONet, covering multiple physical distribution shifts, demonstrate improvements over standard LoRA, DoRA, AdaLoRA, and other baselines, with performance approaching full fine-tuning in several settings.

Critical strengths

1.The paper distinguishes parameter accessibility from adaptation capacity, providing a useful explanation for why increasing LoRA rank cannot compensate for inappropriate adapter placement. This insight is supported by controlled, matched-budget experiments.

2.The proposed warm-up probe provides a low-cost approach to identifying important adaptation modules. Experiments across six FNO shifts demonstrate improvements over several PEFT baselines, supporting the practical value of appropriate adapter placement.

Critical weaknesses
  1. Insufficient Evidence for the Necessity of Target-Aware Selection: The proposed probe selects the spectral group across all six FNO shifts, leaving its practical advantage over a simple architecture-specific fixed placement unclear. Although the spectral adaptation itself is effective, the experiments do not sufficiently demonstrate that target-dependent probing is necessary, particularly in settings where the preferred parameter group remains unchanged across target shifts.

  2. Limited Cross-Architecture Benefits and Generalization: On DeepONet, broad adaptation matches or outperforms concentrated placement across the three tested shifts, raising questions about the incremental benefit of the adaptive selection policy. Furthermore, the evaluation is limited to relatively small neural operators and controlled one-dimensional PDEs, leaving its scalability and applicability to more complex physical systems insufficiently validated.

Questions for authors
  1. Necessity of the Proposed Probe: Since the probe selects the spectral group on all six FNO shifts, could the authors demonstrate scenarios within the same architecture where different target shifts require different parameter groups, thereby establishing the necessity of target-aware selection over fixed placement?

  2. Cross-Architecture Effectiveness: Given that broad adaptation matches or outperforms concentrated placement across the DeepONet shifts, what additional performance benefit does the complete probe-guided policy provide over a simple fixed broad-placement strategy at matched budgets?

  3. Robustness: While Appendix F examines the number of warm-up steps, could the authors further evaluate sensitivity to the concentration threshold and the amount of target training data, particularly regarding the stability of group selection and final performance?

  4. Generalization: Could the authors provide additional evidence or analysis regarding the applicability of the proposed method to larger neural operators and more realistic two-dimensional or three-dimensional physical systems?

2 comments
AuthorsOpenReview ↗

Response to Reviewer csFu

We thank the reviewer for their comments.

W1: necessity of target-aware selection

"The proposed probe selects the spectral group across all six FNO shifts, leaving its practical advantage over a simple architecture-specific fixed placement unclear. Although the spectral adaptation itself is effective, the experiments do not sufficiently demonstrate that target-dependent probing is necessary, particularly in settings where the preferred parameter group remains unchanged across target shifts."

The six Burgers shifts are exactly the case where the group does not change, so the probe's contribution there is the decision, not an accuracy gain. Revision. the abstract and Section 7.

W2: cross-architecture benefit and scope

"On DeepONet, broad adaptation matches or outperforms concentrated placement across the three tested shifts, raising questions about the incremental benefit of the adaptive selection policy. Furthermore, the evaluation is limited to relatively small neural operators and controlled one-dimensional PDEs, leaving its scalability and applicability to more complex physical systems insufficiently validated."

This splits in two: the DeepONet half is answered in our response to Reviewer 6B5b (W2), and the scale half under Q4. For the DeepONet half specifically: the policy does not claim a win over broad there, and Section 6.4 now says what it does claim, which is the spread call on the diffuse shift. Revision. Section 6.4 and App. K.

Q1: a within-architecture case where the group changes

"Since the probe selects the spectral group on all six FNO shifts, could the authors demonstrate scenarios within the same architecture where different target shifts require different parameter groups, thereby establishing the necessity of target-aware selection over fixed placement?"

The target-functional test is such a scenario (see our response to Reviewer 6B5b, Q2). Its low-data cell is where the probe's group and the fixed spectral group separate and the probe's choice is the better one. Revision. Section 6.3 and Table 6.

Q2: benefit over a fixed broad placement

"Given that broad adaptation matches or outperforms concentrated placement across the DeepONet shifts, what additional performance benefit does the complete probe-guided policy provide over a simple fixed broad-placement strategy at matched budgets?"

On DeepONet the policy has no accuracy win over a fixed broad placement; what it adds is the choice, which the fixed broad strategy makes unconditionally (Section 6.4, Table 4, Table 7).

Q3: sensitivity to the concentration threshold and the amount of target data

"While Appendix F examines the number of warm-up steps, could the authors further evaluate sensitivity to the concentration threshold and the amount of target training data, particularly regarding the stability of group selection and final performance?"

The threshold is not tuned per setting: it is fixed at 0.05 before any test evaluation (Section 4, Step 3). The printed concentrations fall in two clusters, 0.013 on the diffuse DeepONet shift against 0.277-0.632 on the six FNO shifts and the u2u^2 target. Any threshold between the clusters returns the same decision on every printed cell, and the low cluster is also the refusal case: there the policy spreads the budget, which beats concentrating on the probe's own argmax (Table 4). Table 8 (App. M) gives C per shift over three seeds: 0.309-0.621 under the functional score with a seed maximum of 0.632, and 0.077-0.138 under the parameter-norm ablation. For the amount of target data, the sweep in our response to Reviewer 6B5b (Q2) is the sensitivity test: the best group changes with the data fraction, and at the full-data end the probe's choice stops being the better one, as Section 6.3 states. Group selection itself is stable: at K = 15 warm-up steps the probe picks the correct group on six of six FNO shifts, and K = 4 on five of six (App. F), which is why we run K = 30.

Q4: larger operators and 2D or 3D systems

"Could the authors provide additional evidence or analysis regarding the applicability of the proposed method to larger neural operators and more realistic two-dimensional or three-dimensional physical systems?"

We did not run two- or three-dimensional systems, and the paper's claims are bounded to the setting it measures. App. K states the design scope: two architectures (FNO width 64, DeepONet width 256) and two PDE families at 1D resolution, 512 samples per regime and four budgets. The mechanism the paper argues for is measured at those sizes: a budget placed outside the group holding the useful update saturates well above full fine-tuning. The transfer we show is across PDE (Burgers to Kuramoto-Sivashinsky) and architecture, not across resolution or model size. App. K also states that the F-Adapter comparison is method-level on the 1D testbed.

AuthorsOpenReview ↗

Follow-up to Reviewer csFu

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the within-architecture case that you asked for, in which the selected group changes. In a new target-functional test, the same FNO backbone is retrained for y=u(x,T)2y = u(x,T)^2. The probe then selects the projection group in 3/3 seeds, and projection is the best of four placements at 25% of the target data (0.0732 against 0.1041 for fixed spectral placement; Section 6.3, Table 6). The rebuttal also covers the concentration threshold, which is fixed at 0.05 before any test evaluation and separates two clusters of concentrations (0.013 on the diffuse DeepONet shift against 0.277-0.632 on the FNO cells; App. M, Table 8), and the matched-budget broad arm (Table 7). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. Parameter-efficient fine-tuning with LoRA-type adapters is already the standard way to adapt large pretrained models, and it is now applied to neural operators with the placement conventions of language models. The paper shows that two constraints govern PEFT for neural operators, accessibility and capacity. When adaptation is restricted away from the useful parameter group, an error floor remains that additional rank cannot remove. On FNO, a standard adapter placed where the probe points reaches 1.05-1.39x the full fine-tuning error at 2.98% of the trainable parameters, and it beats standard LoRA, DoRA and AdaLoRA at matched budget on all six shifts. These results are, to the best of our knowledge, new in the literature on fine-tuning neural operators. Your review notes that the paper distinguishes parameter accessibility from adaptation capacity and that this insight is supported by controlled, matched-budget experiments, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Reviewer_aSMsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that PEFT depends on both update capacity and access to the appropriate parameter groups. A short target-data warm-up identifies groups whose updates change the function or reduce target loss. The method then concentrates or distributes the adaptation budget, with experiments on FNO and DeepONet under controlled physical shifts.

Critical strengths
  • The deployment problem is real. A pretrained physical surrogate may need adaptation when viscosity, forcing, or input conditions change. Choosing an effective small update can reduce the cost of maintaining such models.
  • The operator-specific diagnosis is useful. The experiments show that adapting only non-spectral modules can be limiting even when their adapter rank increases. This is a concrete warning for applying language-model PEFT conventions to neural operators.
Critical weaknesses
  • Choosing where to update is not new. Surgical Fine-Tuning [1], Layer Card [2], and PAGE/DomLoRA [3] study this question. F-Adapter also studies efficient updates for scientific models in the Fourier domain [4]. The paper needs to show what its new selection rule adds.
  • The need for the selection step is unclear. Every FNO test selects the spectral parameters. A fixed rule would make the same choice. Updating several groups also works well on DeepONet.
  • Most tests use small, one-dimensional systems. The paper does not yet show clear benefits for large models used in practice. It also needs a direct comparison with the full F-Adapter method.

[1] Lee et al. Surgical Fine-Tuning Improves Adaptation to Distribution Shifts. ICLR 2023.

[2] Xu et al. Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models. arXiv preprint, 2026.

[3] Zhang et al. Rethinking Adapter Placement: A Dominant Adaptation Module Perspective. arXiv preprint, 2026.

[4] Zhang et al. F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning. **NeurIPS 2025.

Questions for authors

Refer to weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer aSMs

We thank the reviewer for their comments.

W1: choosing where to update is not new

"Choosing where to update is not new. Surgical Fine-Tuning [1], Layer Card [2], and PAGE/DomLoRA [3] study this question."

We agree, and Section 2 already opens its placement paragraph with the same concession: the choice of which modules to touch is fixed in advance and inherited across tasks, and our work targets that choice. The rule differs on three axes. The object is the architectural parameter group of an operator, and the signal is early functional and task-loss improvement from a short warm-up rather than a gradient, variance or Fisher proxy. The third is target dependence: the placement follows the shift, where PAGE reports task-stable module choice in language models. Revision. Section 2 states the three axes and Section 6.5 compares against F-Adapter's within-subspace rule.

W2: the need for the selection step

"Every FNO test selects the spectral parameters. A fixed rule would make the same choice."

For the FNO half, the probe and the fixed rule agree, and Table 3's second column is now labelled "dominant group", with "-" for the diffuse DeepONet shift where no group dominates. Revision. Table 3 and Section 6.3.

W3: scale, and the direct F-Adapter comparison

"The paper does not yet show clear benefits for large models used in practice. It also needs a direct comparison with the full F-Adapter method."

The scale half is answered in our response to Reviewer csFu (Q4). On F-Adapter, we compare a band re-implementation of its module at 15,464 real DOF, which is 1.44x (viscosity) and 1.73x (initial condition) worse than the concentrated spectral arm (Section 6.2, Table 7), and we did not run its official 3D Navier-Stokes benchmark. Within the group its fixed low-frequency prior is the best of the six allocation policies we tested, at 0.0172 mean test error over the six FNO shifts against 0.0237 for displacement-guided allocation (Table 5). Section 6.5 reports that as a separate, complementary problem. App. H states that the PDEBench 3D-NS data was not obtainable in our environment, so the comparison is method-level on the FNO testbed.

Q1: questions referred to the weaknesses

"Refer to weakness"

Your question is answered by the three weaknesses above: W1 on novelty, W2 on the selection step, and W3 on scale and F-Adapter.

AuthorsOpenReview ↗

Follow-up to Reviewer aSMs

We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the need for the selection step. In a new target-functional test, the same FNO backbone is retrained for y=u(x,T)2y = u(x,T)^2. The probe then selects the projection group in 3/3 seeds, and projection is the best of four placements at 25% of the target data (0.0732 against 0.1041 for fixed spectral placement; Section 6.3, Table 6). The rebuttal also covers the relation to earlier work on choosing where to update, which Section 2 now states on three axes, and the comparison with F-Adapter (Section 6.5, Table 5 and Table 7). We hope these additions give a fuller picture of the work.

We would also like to highlight the main result of the paper. Parameter-efficient fine-tuning with LoRA-type adapters is already the standard way to adapt large pretrained models, and it is now applied to neural operators with the placement conventions of language models. The paper shows that two constraints govern PEFT for neural operators, accessibility and capacity. When adaptation is restricted away from the useful parameter group, an error floor remains that additional rank cannot remove. On FNO, a standard adapter placed where the probe points reaches 1.05-1.39x the full fine-tuning error at 2.98% of the trainable parameters, and it beats standard LoRA, DoRA and AdaLoRA at matched budget on all six shifts. These results are, to the best of our knowledge, new in the literature on fine-tuning neural operators. Your review notes that the operator-specific diagnosis is useful and gives a concrete warning for applying language-model PEFT conventions to neural operators, and we hope the new controls complete this picture.

We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.

Thank you for your time and consideration. We look forward to your reply.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their constructive feedback and valuable insights. The reviews converge on one question about FNO, where the probe selects the spectral group on all six shifts and a fixed spectral rule would make the same choice, on one doubt about DeepONet, where broad placement matches or beats the branch and the policy has no measured win over a fixed broad baseline, and on the 1D scale of the evaluation, which two reviews question.

The strongest new evidence is a target-functional test: we retrain the same FNO backbone for y=u(x,T)2y=u(x,T)^2 with the source inputs and splits, the probe selects the projection group in 3/3 seeds (C=0.277C=0.277), projection is the best of four placements at 25% of target data (0.0732 against 0.1041 for fixed spectral), and fixed spectral takes the lead at 100% (0.0351 against 0.0442), so the placement decision follows the target and the amount of target data rather than the architecture; these are new runs. The abstract, the conclusion, Section 6.2, Section 6.4, Section 6.5 and App. K carry the matching statements, including the FNO concession that a fixed spectral DoRA at a 25% smaller budget is stronger on five of six shifts; Section 6.3 reports the test under "Target-functional probe test", Table 6 (App. L) gives the sweep, and Table 7 (App. L) the matched-budget broad arm. Revisions are marked in blue in the revised PDF.

AuthorsOpenReview ↗

Summary of the Rebuttal for the Area Chair and Reviewers

We thank the Area Chair and the reviewers for their time and valuable feedback. As the discussion period is ending, we thought it could be useful to summarize the current state of the rebuttal.

We would first like to highlight the main result of the paper. Parameter-efficient fine-tuning with LoRA-type adapters is already the standard way to adapt large pretrained models, and it is now applied to neural operators with the placement conventions of language models. Before this work there was, to the best of our knowledge, no controlled study of where the trainable budget must be placed in a neural operator. Our paper provides this study. It shows that two constraints govern PEFT for neural operators, accessibility and capacity. When adaptation is restricted away from the useful parameter group, an error floor remains that additional rank cannot remove. On FNO, a standard adapter placed where the probe points reaches 1.05-1.39x the full fine-tuning error at 2.98% of the trainable parameters, and it beats standard LoRA, DoRA and AdaLoRA at matched budget on all six shifts. All three reviews list this placement finding as a strength of the paper.

We initially received scores 2, 2, 2. Based on reviewer feedback, we added the following to our manuscript:

  • A new target-functional test. The same FNO backbone is retrained for y=u(x,T)2y = u(x,T)^2. The probe selects the projection group in 3/3 seeds, and projection is the best of four placements at 25% of the target data (0.0732 against 0.1041 for fixed spectral placement). The placement decision follows the target and the amount of target data (Section 6.3, App. L, Table 6).
  • A new matched-budget broad arm. On FNO it is 8.4% and 13.7% worse than the concentrated spectral arm, and on the diffuse DeepONet shift the probe spreads the budget (Section 6.4, Tables 4 and 7).
  • A new table of the concentration score per shift over three seeds, which shows two separated clusters around the fixed threshold of 0.05 (App. M, Table 8).
  • A discussion of the relation to earlier work on choosing where to update, stated on three axes (Section 2), and a comparison with F-Adapter (Section 6.5, Table 5).
  • A statement of what the probe contributes on FNO, namely the placement decision from target data with no per-architecture prior (Abstract, Section 6.2).

During the discussion phase, we provided detailed responses to the concerns raised by all three reviewers and revised the paper accordingly. The rating changes and current status of the three reviewers are as follows:

ReviewerOriginal RatingAfter Discussion
6B5b22 (No Reply)
csFu22 (No Reply)
aSMs22 (No Reply)

Although the reviewers have not replied yet, our responses have addressed all of their comments. Given this, we hope the Area Chair can take our experiments and clarifications into account. We also kindly ask the reviewers to read our responses and to update their scores if their concerns are resolved.

We are extremely grateful for the feedback given by the reviewers, and sincerely believe their recommendations improved the paper. We are happy to answer any question about the paper or the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.