- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets the improvement of generality in multimodal knowledge editing. It groups equivalent vision–language inputs into a knowledge unit and defines generality as consistent predictions within that unit. The proposed Adversarial Subspace AlignMent (ASAM), adds two plug-in pieces to existing editors: Latent Adversarial Robustification, which perturbs joint embeddings by one projected gradient step, and Rank-Constrained Subspace Learning, which aligns the resulting edit-layer states with a singular-value loss while detaching the original state. Experiments on MMEdit and MMKE-Bench cover several multimodal large language models and editing methods. The results show improved generalization while largely preserving editing success on the original input and performance on unrelated knowledge.
Critical strengths
-
The main idea is a genuine shift in how an edit is supervised. Instead of anchoring the update to one multimodal sample, ASAM builds a latent adversarial neighborhood and collapses it onto a shared low-rank direction at the edit layer, while detaching the original state so reliability and locality are not rewritten.
-
The same alignment term transfers across WISE, MEND, T-Patcher, and UniKE, and across encoder–decoder and recent decoder-centric MLLMs. Generality rises without replacing the host editor, for example WISE on BLIP2-OPT E-VQA (84.80 to 93.97) and T-Patcher on Qwen2-VL E-VQA (56.03 to 83.17).
-
The technical combination is novel rather than a direct reuse of existing editing losses. It is distinct from paraphrase augmentation, which rewrites surface text, and from pairwise feature matching, which never requires a rank-1 subspace at the edit layer.
-
The paper addresses a problem that matters for multimodal editing in practice. A reliable edit is of limited use if it breaks under a rephrased question or a shifted image, and closing that generality gap is what makes an edit usable beyond the original query.
Critical weaknesses
-
The connection between the theoretical condition and the evaluation metric needs a clearer explanation. Proposition 1 concerns whether normalized edit-layer representations are identical, whereas the experiments measure whether generated answers exactly match the reference answers. The paper should briefly explain when identical representations lead to identical answers, even if the argument assumes a fixed decoder.
-
The definition of locality in Eq. (1) is unclear in two places. First, it does not explicitly identify ( ) as the knowledge unit being edited. Second, the equation compares full conditional probability distributions, while the preceding text defines discrete predictions. Clarifying the edited unit and making the notation consistent with the evaluation would improve this definition.
Questions for authors
-
The original representation does not receive gradients from the alignment loss, but it still participates in the singular-value calculation. Is it therefore a fixed reference, with the alignment loss moving only the perturbed representations toward it? How does this design relate to the different degrees of editing scope discussed in CH3?
-
The text introduces both () and (). Which objective is actually optimized during training? Are the reliability loss () and locality loss () taken directly from the underlying editor without modification?
-
For memory-based editors such as WISE, is ASAM used only when learning the edit, or does it also change retrieval at test time? A brief explanation of how ASAM connects to the underlying editor would clarify which stages it affects.
3 comments
Thank you for the clarifications.
The response clarifies the consistency between representation-level alignment and answer-level evaluation. The additional ablation study also provides empirical support for the necessity of stopping gradient propagation through the original representation.
Overall, the rebuttal partially addresses the concerns.
Thanks for recognizing our work and your valuable suggestions
Thank you very much for your recognition and suggestions. We really appreciate your time and effort in reviewing this paper! We will incorporate these additional experimental results and corresponding analyses introduced in the rebuttal into the revised version. We sincerely thank you again for your valuable comments.
Dear Reviewer,
Thank you for the positive assessment and for these questions. We make each point precise below, with the corresponding revisions.
W1. Representation-level alignment versus answer-level generality
Proposition 1 concerns edit-layer representation geometry, while answer-level generality is evaluated through predictions. We now explain how the two are connected.
Identical complete conditioning states with deterministic decoding yield identical predictions. Proposition 1 concerns normalized representations at the edit layers, which are identified as the locus of knowledge. Identical normalized representations there correspond to the same knowledge, and, as ROME argues, the answer is determined at these layers, so answer consistency follows. We therefore treat Lalign as a representation-level surrogate for consistency within a knowledge unit.
W2. Locality definition and the edited knowledge unit
The revision identifies the edited knowledge unit in Eq. (1) and matches the discrete prediction metric.
For an edit on knowledge unit Ki, locality is evaluated on input pairs outside that unit, and Dout,i contains no q=(xv,xt) from Ki.
The metric measures preservation of discrete predictions. With reference answer y∗=(y1∗,…,yTq∗) and predicted token y^θ,t(q,y∗_<t)=argmaxv∈Vpθ(v∣q,y∗_<t),
Loci=Eq∼Dout,i[Tq1∑t=1TqI(y^θ′,t(q,y∗<t)=y^θ,t(q,y∗_<t))],
i.e., whether the edited model preserves the original predictions on out-of-scope inputs.
Q1. Detached reference and its relation to editing scope
Yes. The original representation is a per-step detached reference, Hs=[sg(h^0);h^1;…;h^n], with sg stop-gradient. No gradient flows through its branch, so the alignment gradient acts through the perturbed representations, while the reliability and locality terms keep the original path.
On CH3, we move beyond single-sample supervision. LAR builds a bounded latent neighborhood around the input, and RCSL regularizes the edit-layer representations of these variants. The effect is to broaden local training coverage, and no continuous editing-scope boundary is learned. ASAM guarantees neither continuous nor monotonic growth of influence. The symmetric-versus-asymmetric ablation isolates this routing choice.
Q2. Optimized objective and host-editor losses
ASAM adds only one objective. For host editor E,
LASAM(E)=Lhost(E)+βLalign,
where Lhost(E) is E's original objective and Lalign the singular-value regularizer on the input and its LAR variants, weighted by β.
Editors with explicit reliability and locality objectives have Lhost(E)=Lrel+Lloc with original coefficients, so ASAM only adds βLalign. Editors that preserve implicitly through memory or retrieval may lack Lloc. There the host objective stays whole and only the regularizer is added.
We will use Lalign consistently, revise Lrobust so it no longer suggests a competing objective, and specify where the term enters each editor, including MEND.
Q3. Memory-based editors and test-time retrieval
ASAM is an editing-time regularizer with no new inference-time retrieval. LAR generates variants during edit construction, and the alignment loss is evaluated on their edit-layer representations, shaping the edit that WISE writes to memory.
The architecture, retrieval keys, scoring, and routing rule remain WISE's. Memory contents and representations may differ because the edit is optimized under the additional constraint, so retrieval outcomes can change indirectly. Edit-induced changes in activation statistics can affect the gate's decisions. This is an effect of the edited model itself, since no retrieval stage is added.
At inference, no variants are generated and neither loss nor singular-value computation is performed. The model uses the host's original forward and retrieval procedures. MEND, T-Patcher, and UniKE likewise add no test-time module or extra computation.
Thank you again for the constructive feedback. Every comment above has been addressed in the revised manuscript, with all changes highlighted in blue and cross-referenced to this response to confirm that each revision is traceable. We would be glad to clarify any remaining point.