acceptodds
Withdrawn from ICLR 2027

Functional Tangent Geometry Selects Adapter Placement in Equivariant Interatomic Potentials

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The best adapter is determined more by placement than parameter count because different locations expose different energy-force correction spaces. We formalize these spaces with functional tangent geometry, using a Jacobian to map bounded adapter coefficients to reachable output corrections. A held-out transfer score then tests whether coefficients fitted on one configuration split represent the same correction on another. The score is invariant to adapter reparameterization when the coefficient metric transforms with it. Controlled experiments recover all four generating placements, and local molecular experiments show that tangent rankings predict constrained nonlinear adaptation. Across six rMD17 molecules, native-coordinate geometry selects the readout adapter. It remains best on untouched 1,000-frame holdouts, reducing normalized residual by 20.71% relative to the first equivariant product block. A local Ti-Al task instead selects that product block, whereas column normalization reverses the molecular ranking. Functional tangent geometry chooses adapter locations from the corrections they can express, avoiding a nonlinear sweep over placements.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
parameter-efficient fine-tuning, equivariant neural networks, interatomic potentials, adapter placement, functional tangent geometry, transfer learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qrkMOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes choosing where to insert a fixed-basis adapter in an equivariant MLIP (MACE-OFF23/MACE-MP-0) by a held-out linear reachability score. For each candidate placement, a d=32d=32 orthonormal slice of the first-order-active adapter factor is taken. The energy-force Jacobian on a fit split is used to solve a ridge-regulararized, trust-region-constrained least squares for the required correction. The same coefficients are applied through the validation-split Jacobian and the normalized residual τ\tau is the score. Prop.1 specifically notes this score is invariant under an invertible coordinate change if the coefficient metric is co-transformed.

Critical strengths

1.Framing placement as a problem about which output-space corrections a module can express, with forces as mixed coordinate-parameter derivatives, is a reasonable and clearly stated viewpoint for MLIPs.

  1. The experimental bookkeeping is unusually careful: prospective vs. post-hoc analyses are labelled, untouched test frames are kept untouched, Jacobians are verified by finite differences checks, and the locality remainder ϵ\epsilon is reported separately from rank agreement.

  2. This paper is honest about limitations, even though slice-coverage imbalance, non-local regime, and the LoRA reversal would undercut the main claim.

Critical weaknesses
  1. The proposed selector does not outperforms trivial alternatives. On the six-molecule cohort the paper reports that fit-only residual, gradient norm, and Gauss-Newton trace also select the readout in 18/18 cases (Sec 4.4) and Table 16 showcases fit-only ties the held-out score on every other task. However, an "always choose readout" rule reproduces every molecular selection. Therefore, the specific contribution that the authors champion, the held-out transfer construction, is therefore never shown to add anything over a fit-only residual or a gradient norm.

  2. The readout preference might be a slice-coverage evidence. The 32-dimensional slice covers 50% of the readout's 64 coordinates but 12.5%/1.6%/0.6%/3%/6% of the other five spaces (Table 2). A random half of a small space naturally reaches more of that space's corrections than a random 0.6% of a large one. This paper though acknowledges this shortly ("unequal fractional coverage may contribute") but never runs the obvious control: full native spaces or matched fractional coverage.

  3. Prop. 1 is a change of variables (a=Tb,G′=TTGTa = Tb, G'=T^{T}GT) and does not warrant the framing as a "reparameterization principle". The question, for instance, which metric to use, is left to the user ("the native metric is meaningful when it matches the deployed parameterization"), and the paper shows that the answer would change the conclusion.

Questions for authors

See above weaknesses.

Reviewer_9fVeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks where to place adapters when adapting a pretrained interatomic potential to a small downstream dataset. For each candidate placement, it linearizes the energy and force predictions, fits adapter coefficients on part of the data, and scores the transferred correction on held-out configurations. This selects a placement without running nonlinear adaptation for every candidate. The experiments compare the predicted ranking with actual adaptation under matched constraints and examine failures when the adapter basis changes.

Critical strengths
  1. The placement question is useful and complements adapter designs such as ELoRA. Comparing the energy and force corrections available at different placements gives a physical interpretation of what each adapter can change. Proposition 1 also clarifies why reparameterization preserves the score only when the coefficient metric is transformed consistently.
  2. The paper distinguishes local fidelity from rank transfer: an inaccurate linear approximation can still predict the ordering of placements. Reporting these separately, together with the failure under unrestricted LoRA, helps explain what the results establish.

The work builds on linearized-network adaptation and earlier placement methods, including PLoP. Its contribution is the energy–force selection criterion and the study of when its predictions hold.

Critical weaknesses
  1. The main experiments do not show a benefit over simpler selectors. Section 4.4 reports that always choosing the readout reproduces every molecular selection, as do fit-only, gradient-norm, and Gauss–Newton criteria. The expanded Ti–Al comparison shows that a different task can favor another placement, but it is post hoc and does not establish that the held-out score makes a better choice than the simpler criteria. The placement question is interesting; the need for this particular selector remains unproven.
  2. The successful ranking results concern a restricted adaptation procedure. The nonlinear tests use the same fixed basis, coefficient metric, and norm constraint as the selector. This is a valid test of the stated prediction, but leaves its usefulness for ordinary adapter training unclear. The unrestricted LoRA result shows that the initial tangent ranking can fail when both factors change. Moreover, Section 4.4 shows that column normalization reverses the molecular choices. Equal-dimensional slices also cover different fractions of the native spaces, which the authors acknowledge may contribute to the readout preference. Equal trainable dimension is a reasonable comparison, but these results make the choice of metric and candidate space central to the practical conclusion.

The presentation needs more explanation before the formal definitions in Section 3. A simple example showing how a task correction is fitted and then scored on held-out configurations would make the method easier to follow.

I lean toward rejection. The analysis is useful, but the main results do not establish an advantage over simpler selectors or show when the constrained procedure is preferable in practice.

Questions for authors
  1. In which main-paper setting does the held-out criterion select a better placement than the simpler criteria? In particular, do those criteria also select the product block on Ti–Al?
  2. When should a user adopt the fixed-basis, norm-constrained adaptation procedure studied here? How can they choose the coefficient metric and radius without running a full adaptation sweep, and what evidence supports those choices?
Reviewer_uvnqOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a held-out functional tangent criterion for selecting adapter placements in equivariant interatomic potentials. It evaluates whether bounded updates can reproduce task-specific energy–force corrections and transfer across configurations, then examines whether the resulting rankings predict constrained nonlinear adaptation.

Critical strengths

Clear motivation. Comparing reachable output corrections provides a meaningful perspective beyond adapter parameter counts.

Careful analysis. The paper distinguishes ranking accuracy from local approximation fidelity and explicitly discusses dependence on the coefficient metric and trust radius.

Informative controlled experiments. Synthetic interventions recover generating placements, while strict-local experiments support agreement between tangent predictions and nonlinear adaptation.

Critical weaknesses

Limited evidence of added value over simpler selectors. Across the six-molecule cohort, always choosing the readout, fit-only residuals, gradient norms, and Gauss–Newton criteria recover the same winner. These results do not establish the benefit of the proposed held-out criterion.

Limited task-dependent validation. The Ti–Al experiment provides a useful contrasting case, but broader evidence is needed to demonstrate reliable selection when optimal placements vary across tasks. The six-placement Ti–Al comparison is also post hoc.

Restricted applicability. The main validation concerns fixed-basis updates within bounded coefficient regions. Unrestricted two-factor LoRA exhibits an ordering reversal, limiting applicability to more flexible adaptation settings.

The headline improvement does not isolate selector quality. The reported 20.71% reduction is relative to fixed product-0 adaptation, rather than the strongest simple placement rule. It therefore demonstrates a placement advantage without establishing an advantage of the proposed selection procedure.

Questions for authors
  1. Which existing results demonstrate an advantage of held-out tangent selection over fit-only residuals, gradient norms, or Gauss–Newton criteria?

  2. Beyond the post-hoc Ti–Al comparison, is there evidence that the criterion prospectively identifies task-dependent changes in the optimal placement?

  3. What is the total selection cost, including Jacobian construction and fitting, relative to simpler selectors and a limited nonlinear placement sweep under comparable hardware conditions?

  4. Please clarify the intended practical scope of the method given the fixed-basis restriction and the ordering reversal observed with unrestricted LoRA.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.