acceptodds
Under review as a conference paper at ICLR 2027

Parameter-Efficient Is Not Function-Stable: Function Trust for Model Adaptation

Abstract

Parameter efficiency constrains where a model changes, but not how far its predictive function moves. We introduce \trust, a parameterization-agnostic framework that adapts a model subject to a user-specified budget on the forward KL divergence from a frozen source predictor over cached source inputs. A primal–dual Powell–Hestenes–Rockafellar augmented Lagrangian approximately solves the constrained problem, while the budget formulation remains independent of the solver. Conditional on achieving the replay constraint within tolerance , we bound population functional drift by and source-risk change by , where captures replay-to-population generalization; a margin corollary controls deterministic prediction changes. Across TinyBERT and Qwen3-0.6B, \trust consistently reduces source forgetting and held-out KL for LoRA, full fine-tuning, and our structured adapter, with small or parameterization-dependent changes in target quality. Yet equal replay budgets need not imply equal held-out behavior: increasing TinyBERT replay from to examples reduces validation KL from to , while low-margin replay at fixed reduces it from to . Fixed-penalty and augmented-Lagrangian frontiers overlap; our contribution is therefore not solver dominance. The evidence instead shows that functional stability is not a by-product of parameter efficiency: it must be explicitly budgeted, measured, and audited.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Model Adaptation, Small Language Models, Machine Learning, Function Trust

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dd2EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the preservation of predictive behavior during single-step task adaptation, emphasizing that parameter efficiency alone does not ensure functional stability. It introduces Function Trust, a parameterization-agnostic framework that constrains the average forward KL divergence from a frozen source predictor to the adapted model on cached source inputs, using a primal–dual augmented Lagrangian solver. It also proposes BridgeOFT, a structured adapter combining Gaussian optimal transport based on activation second moments with trainable blockwise orthogonal transformations.

The theoretical analysis bounds source-population functional drift and bounded-loss stochastic risk changes, conditional on empirical feasibility and a uniform generalization bound, and provides a margin-dependent bound on deterministic prediction changes. Experiments primarily evaluate TinyBERT and Qwen3-0.6B on SST-2→MRPC adaptation, with additional analyses of replay size and selection, constraint feasibility, and fixed-KL-penalty baselines.

Across the tested matched comparisons, Function Trust reduces source forgetting and held-out KL, with model- and parameterization-dependent effects on target performance. The results also demonstrate that retention depends on replay coverage and composition, and that empirical feasibility does not guarantee held-out stability. The paper does not establish solver superiority over tuned fixed penalties or treat BridgeOFT’s geometry as a retention guarantee; its central emphasis is on explicitly budgeting and auditing functional drift.

Critical strengths
  • S1: Consistent retention improvements across adaptation methods support the central premise. Matched comparisons on TinyBERT and Qwen3-0.6B show that adding Function Trust reduces source-task forgetting and held-out source KL across the tested configurations. These comparisons help separate the effect of the functional constraint from the choice of adapter. This strengthens the empirical case for explicitly controlling predictive drift and demonstrates that the framework’s benefits are not confined to BridgeOFT.

  • S2: The diagnostic analysis provides useful evidence about the limits of replay-based retention. The paper examines replay size and composition, constraint violations, and discrepancies between replay and held-out KL, alongside fixed-penalty controls. These analyses show why satisfying an empirical budget need not ensure held-out stability and distinguish the value of the budget formulation from solver superiority. This contributes practical insight beyond headline performance gains and makes the paper’s conclusions more credible and informative.

Critical weaknesses
  • W1: The independent value of BridgeOFT remains insufficiently established. The experiments do not consistently demonstrate the benefit of its Gaussian OT construction over simpler alternatives. In the TinyBERT geometry ablation, different bases achieve similar forgetting levels under Function Trust, although the covariance-based variant yields somewhat higher target F1 (Section 5.2 and Table 4). On Qwen, BridgeOFT trades lower target F1 for better source retention relative to LoRA, without establishing an advantage at matched retention levels (Table 1). In generative transfer, hard BridgeOFT achieves lower mean translation chrF and GSM8K exact-match accuracy than both Euclidean OFT and metric OFT (Table 6). Because BridgeOFT is presented as a separate methodological contribution, the absence of clear conditions under which its additional structure provides reproducible benefits weakens the paper’s methodological significance.

  • W2: Validation of Function Trust is concentrated in a narrow task regime. The primary evidence comes from SST-2→MRPC adaptation, supplemented by SST-2→RTE and a reverse-direction pilot. These experiments evaluate preservation of task-specific classification behavior, including when using Qwen3-0.6B. The supplementary GSM8K→translation study does not directly compare Function Trust against a no-Trust counterpart and lacks a matched LoRA baseline, leaving its effectiveness for generative capability preservation untested in that protocol. Although the authors appropriately acknowledge this scope, the limited task diversity reduces confidence that the demonstrated benefits extend beyond the classification settings studied, constraining the contribution’s broader empirical significance.

Questions for authors
  • Q1: The replay findings have limited empirical and theoretical coverage. The strongest controlled evidence for replay size and composition comes from TinyBERT on SST-2→MRPC. Moreover, the low-margin selection strategy falls outside the theorem’s assumption of i.i.d. sampling from the source input distribution, so the stated population guarantee does not cover this intervention. This does not invalidate its empirical benefits, but leaves their reliability beyond the tested setting unresolved. Because replay composition is a central empirical contribution, limited cross-task validation and the gap between the selection strategy and the theoretical guarantee weaken the evidence for a broadly applicable retention approach.

  • Q2: Formatting. The paper’s font and layout differ from those commonly seen in conference submissions, which affects its visual presentation. However, whether these choices violate the formatting requirements would need to be checked against the official template.

Reviewer_NvCgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates the problem of forgetting during fine-tuning. Given a source task which the model was trained on and a target task which the model needs to be fine-tuned on, it considers multiple fine-tuning variants, such as retraining the entire backbone, just the head or using parameter-efficient fine-tuning techniques (LoRA or BridgeOFT, which is their own contribution). The authors investigate the extent to which each technique leads to forgetting by caching a sample of outputs for the source task and measuring how much the fine-tuning leads leads the model's performance to drift away from the cached samples. They then propose two complimentary contributions:

  • Function Trust is an optimisation technique that constraints fine-tuning by imposing a limit on the drift of the KL divergence can between the original model's predictions and fine-tuned model's predictions on the cached inputs.
  • BridgeOFT is itself an adapter, an alternative to LoRA. It works by considering the difference in activation covariance between the source and target tasks and computes a fixed transformation matrix to align them. Additionally, learnable rotations give the adapter flexibility to fit the target. To be parameter-efficient only the rotation and output head are trained.

The results show that Function Trust reduces forgetting across all fine-tuning settings (full-tuning, LoRA, BridgeOFT), with small F1 losses in most settings compared to the non-Function Trust version for both TinyBERT and Qwen3-0.6B. BridgeOFT's results are less strong or consistent. On TinyBERT it improves F1 but increases forgetting, on Qwen3 it reduces forgetting but at the cost of F1.

Critical strengths

The paper's key strength is the quality of the empirical analysis of section 5 and the insights they provide. The authors carry out extensive analysis to show the mechanics of forgetting/retention. I found two findings to be of particular interest:

  • That forgetting concentrates on low-confidence source predictions and is heavy tailed. This shows that simply considering the average drift as a metric for forgetting actually hides the few badly-damaged examples that are particularly important in diagnosing it
  • What drives retention is constraining the model's outputs during fine-tuning more than the specific geometry of the PEFT adapter
Critical weaknesses

The paper's main weakness is that, whilst the qualitative analysis of the mechanisms behind forgetting/retention is informative, it has not translated in any practical advantage over established techniques such as a fixed-weight KL penalty to the loss. The paper's augmented Lagrangian construction is shown in Tables 3 and 5 to not provide any significant advantage over fixed KL with regards to both F1 and forgetting. The authors acknowledge this (i.e. in section 5.5) and argue that the main advantage of their method is that it offers greater interpretability, in the sense that a user can specify a drift budget directly. However they also demonstrate that the cached set of source task examples is not representative of the full held-out set, which undermines the explainability claim. This is because even when the user-specified drift target is met during fine-tuning, KL drift on the full held-out set is up to 21 times larger.

Questions for authors
  1. The authors describe a contrast between function-space and parameter-space constraining to mitigate forgetting and argue that restricting the parameter space does not control functional drift. However they do not provide any comparison of their model against parameter-space techniques such as EWC. Could the authors provide such comparison?

  2. The authors recommend to sweep ρ on held-out source data to recalibrate their Function Trust construction. This is also the standard practice for finding an appropriate λ in fixed-penalty methods. What, then, is the practical benefit for a user for sweeping for ρ rather than λ ? Are there any settings in which ρ transfers but λ doesn't ?

Reviewer_RhcyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper fine-tunes a model on a new task while penalizing the KL divergence between its outputs and the frozen original model’s outputs on a cache of old-task inputs. As in output distillation / Learning without Forgetting, with the change that the user states a KL budget and an augmented-Lagrangian solver chooses the penalty weight. A second, independent proposal is an adapter that combines an optimal-transport map between source and target activation covariances with a trainable orthogonal rotation. On TinyBERT and Qwen3-0.6B, the KL penalty reduces forgetting for LoRA, full fine-tuning and the new adapter with little change to target F1. Larger replay caches and low-margin replay examples give better held-out retention.

Critical strengths

The experiment reporting is transparent, including of negative results.

Critical weaknesses

The main weakness is that the budgeted solver does not outperform a fixed penalty. In the authors’ sweep (Fig 5) and three-seed control (Table 5), a fixed weight of λ=1 reaches the same forgetting as the default budget with higher target F1. The paper therefore rests on the argument that a KL budget is easier to interpret than a penalty weight. That argument is weakened by the paper’s own numbers, as the requested budget is often not achieved and the replay-cache KL understates the full-validation KL, so the requested value does not describe held-out behavior. The theorem is Pinsker’s inequality combined with an assumed uniform-convergence bound with an uncontrolled term left. The adapter is also not supported, as Table 4 shows that the covariance geometry does not affect retention, the generative experiment shows that orthogonal fine-tuning without the transport map does better on both target and source and Qwen3-0.6B BridgeOFT target F1 is close to the head-only baseline. The observation that parameter-efficient methods still forget is already in the literature the paper cites. The experimental scope is a 4-layer BERT and a 0.6B model on one task pair, with several single-seed cells. The finding that more replay data gives better retention is expected. The writing hedges every result until it is hard to say what is being claimed.

Questions for authors
  • Given that a fixed λ matches or beats the budgeted solver, what is the practical case for Function Trust over Learning without Forgetting with a tuned λ?
  • Your factorial shows the covariance bridge does not help retention and your generative experiment shows it hurts target and source relative to plain orthogonal fine-tuning. What does BridgeOFT add given these results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.