- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the semantic interface drift in target-adapted VLMs. By defining semantic interface drift as the changes in direct prompt compatibility, prototype relations, local neighborhoods, and held-out zero-shot behavior, the paper shows that fine-tuning a VLM can improve target-task accuracy while degrading zero-shot generalization. The proposed same-image anchoring can address the semantic interface drift while preserving the adaptation accuracy.
Critical strengths
1.The paper studies and proposes semantic interface drift, which is useful and worthing researching.
2.The proposed same-image anchoring is simple yet effectively reduces semantic interface drift.
Critical weaknesses
1.The novelty of same-image anchoring is somewhat limited. As stated in the related works, many previous methods had utilized similar strategies to avoid drifting from the original feature, which weakens the contribution of the proposed method. L_anchor is very similar to the fine-tuning loss in LDIFS.
2.The definition of semantic interface drift is not enough comprehensive. The definition of semantic drift, including changes in prompt compatibility, prototype relations, local neighborhoods, are mostly geometric-wise, not essentially describing semantical capabilities.
Questions for authors
Could you restate the main contribution of the paper and explain each evaluation and how does it support your claims?
Thank you for the discussion. We have completed an additional three-seed comparison under a common visual LoRA protocol, covering task-only adaptation, three reference assignments, and the LDIFS and Lipsum-FT objectives.
The CIFAR-100 validation accuracies are: Method Validation accuracy (%) Task-only 83.96 ± 0.10 Same-image L2 84.53 ± 0.02 Global-shuffle L2 84.22 ± 0.03 Within-class L2 84.18 ± 0.04 LDIFS objective 84.17 ± 0.02 Lipsum-FT objective 83.57 ± 0.12
Values are means and sample standard deviations across three seeds, using validation-selected checkpoints. These additional results show that same-image anchoring maintains strong target-task utility within the shared adaptation setting. They complement the previously reported held-out comparisons, which examine retained geometry, pretrained prediction agreement, and transfer.
Together, the experiments address two separate questions: whether preservation remains compatible with target adaptation, and what different reference relationships preserve. The latter remains our principal contribution.