acceptodds
Under review as a conference paper at ICLR 2027

CenWrite: Support-Centered LoRA with Boundary Updates for Cross-Domain Few-Shot Learning

Abstract

Cross-domain few-shot learning adapts a pretrained vision model to a new domain with scarce labels. Such adaptation must use the same small support set to accommodate domain differences and distinguish target classes. It must also produce reliable predictive probabilities despite having only very limited labeled evidence about the target domain. Low-rank adaptation (LoRA) offers a parameter-efficient approach, but uncentered support activations make the support-set mean response and sample-specific deviations jointly shape its low-rank factors. Direct centering changes this learning path but also removes the shared response from the forward pass. We propose CenWrite to center factor learning while preserving the shared response and enabling independent boundary adjustment. CenWrite learns low-rank factors from centered support activations, restores the shared response through stop-gradient compensation, and learns a separate boundary residual. Experiments across multiple domains demonstrate that CenWrite achieves higher average classification performance and lower probability errors than the compared parameter-efficient baselines. Ablation studies further corroborate that centering and forward compensation improve average probability quality, while the boundary residual improves average classification performance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
cross-domain few-shot learning, parameter-efficient fine-tuning, low-rank adaptation, support-set centering, predictive reliability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QNgkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CenWrite, a method specifically designed for cross-domain  few-shot learning. This method centralizes the support set and uses the centralized activation values to learn LoRA. Furthermore, trainable residual adjustment is introduced to modify the classification boundaries. The experiment covered multiple datasets, including EuroSAT, PlantVillage, ISIC, NIH ChestX, as well as UHCS and NFFA, and provided a comparison with various adapter methods.

Critical strengths

1.The method is simple and easy to implement. 2.The experimental metrics are comprehensive, reporting Accuracy, Macro-F1, NLL, Brier, and ECE.

Critical weaknesses

1.In this paper, the optimization process is adjusted by removing the mean component of the support set features from the LoRA gradient. Although the authors provided derivations and ablation experiments, they failed to adequately explain why removing this component can improve the quality of predicted probabilities and how it specifically affects the optimization process. 2.CenWrite constructs feature centers by averaging all the supported set images and tokens. However, the rationale for this design is still unclear. This global averaging might mix the category-specific semantics with non-discriminative features. Since the feature centers directly guide the LoRA gradient modification,the authors should justify using a global mean over alternatives like class-wise means.

Questions for authors

1.Why does removing the gradient components related to the mean of the support set improve the quality of prediction probabilities? Please further analyze the underlying mechanism.

2.Why is the center of the support set calculated by averaging all the support set images and token positions? Have alternative strategies such as class centers been considered?

3.Why did CenWrite improve the probability metrics on DINOv3 but slightly reduce the classification accuracy and Macro-F1? Please further explain this trade-off and the reason for the inconsistent improvement between DINOv3 and DINOv2.

4.The paper primarily highlights CenWrite's improvements in probabilistic metrics such as NLL, Brier Score, and ECE. However, the experiments mainly compare against different parameter-efficient fine-tuning methods, lacking comparisons with simple probability calibration techniques.

5.The paper primarily adopts 5-way 1-shot and 5-way 5-shot settings, lacking a systematic evaluation across a wider range of class counts and support set sizes.

Reviewer_AWZROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that ordinary LoRA uncentered support activations make the support-set mean response and sample-specific deviations jointly shape its low-rank factors, and proposes CenWrite to center factor learning while preserving the shared response and enabling boundary adjustment.

Critical strengths

Eq. 4 and Eq. 5 make the contribution unambiguous: the forward value is preserved; only the local factor gradient changes. This is unusually clean for an adapter paper.

Critical weaknesses
  1. The "shared shift" vs. "relative geometry" construct is not identified. μ is fixed while later layers adapt, so h − μ at step 50 is not the same decomposition as at initialization. Table 61 already shows ||h50 − μ|| and ||h50 − h0|| differ between methods. The decoupling is a property of the gradient formula, not of the adapted representation.

  2. Absolute performance is low, and accuracy claims are fragile. The method proposed in the paper fails to outperform the original LORA in terms of performance across multiple tables and multiple datasets.

  3. Natural baselines are missing. All comparators change the update representation or optimizer. Full fine-tuning with weight decay and an explicit two-branch shared-bias/sample-factor adapter are absent, so it is unclear whether the gain comes from the specific construction or simply from having a separate shared pathway.

Questions for authors
  1. The obvious alternative, compensating with a non-detached mean response, is not tested.
Reviewer_5T6BOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CenWrite, a LoRA variant for cross-domain few-shot learning that alters only the factor-gradient path while leaving the forward pass unchanged. The adapter's input is centered by a fixed support-set mean μ, the resulting shared mean response is restored via stop-gradient compensation rather than removed, and a per-layer residual provides an independent boundary adjustment; the trained model folds back into ordinary LoRA plus a bias at inference. Evaluated on four image domains and two materials datasets with DINOv3/DINOv2, 200 matched episodes per setting, and five metrics, CenWrite achieves the lowest mean NLL, Brier, and ECE on DINOv3 while matching LoRA's accuracy, and reports higher mean accuracy on DINOv2. A six-variant ablation attributes the probability gains to centering plus compensation rather than to added capacity.

Critical strengths
  1. The questions raised by the author are accurate. Through the author's arrangement of the indicators, I was indeed able to understand the author's concerns. 2.The author has very honestly presented the comparison results between the method and similar ones, and the indicators are quite detailed.
Critical weaknesses
  1. The author should re-examine the innovation points of the paper. Essentially, the modification involves "subtracting the batch mean of the input + detaching it". From my perspective, this innovation point is not particularly novel.
  2. I am not clear on what the author’s contribution is. The author repeatedly emphasizes the limitations of their own method while overlooking its actual contributions. Furthermore, when compared to contemporary approaches—such as MPS-Tuning (Chen et al., ICLR 2026)—the author’s method fails to demonstrate a decisive advantage. I consider this a significant issue: although the author has conducted a comprehensive set of experiments, they have not succeeded in proving the superiority of their proposed method.
  3. In the author's table of indicators, I am unable to determine whether the gains were brought about by centralization or by any non-trivial alterations made to the factor gradient path. The author needs to conduct additional experiments to clarify this point.
Questions for authors

Q1.Could you provide the results for CenWrite + LoRA-Pro and CenWrite + RITE under the same 50-step, 200-episode protocol as the main text? Q2.Why is mps-tuning considered the most relevant but no relevant experiments have been conducted for comparison? Could the benefits of cenwrite be made more explicit? And could a small-scale control experiment using mps-tuning be carried out? Q3. Why is the residual presented as a co-equal component when dropping it (NB 1.201/20.44/0.537) beats the full method on NLL/ECE/Brier (F 1.203/20.59/0.538, intervals excluding zero) and only buys back part of the macro-F1 loss the other two components cause?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.