acceptodds
Under review as a conference paper at ICLR 2027

Beyond Binary Edits: Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment

Abstract

Multimodal large language models (MLLMs) need efficient mechanisms to update knowledge without degrading existing capabilities. While intrinsic multimodal knowledge editing achieves strong reliability and locality, it often exhibits limited generality, failing to propagate edits across semantically equivalent visual and linguistic variations. This issue arises from the lack of explicit semantic supervision, rigid editing scopes, and biased anchoring to individual samples in high-dimensional multimodal spaces. We address robust intrinsic multimodal knowledge editing by explicitly targeting generalization. We formalize robustness through knowledge units that group semantically equivalent multimodal inputs and define generality as consistent predictions within each unit. To expose fragile semantic regions, we introduce Latent Adversarial Robustification (LAR), which generates adversarial yet semantically coherent variants in the joint latent space. We further propose Rank-Constrained Subspace Learning (RCSL), enforcing low-rank alignment of adversarial representations at the edit layer via a singular value–based objective. Extensive analysis demonstrates the effectiveness of ASAM empirically.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
knowledge editing, multimodal learning, multimodal large language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GS2WOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets the improvement of generality in multimodal knowledge editing. It groups equivalent vision–language inputs into a knowledge unit and defines generality as consistent predictions within that unit. The proposed Adversarial Subspace AlignMent (ASAM), adds two plug-in pieces to existing editors: Latent Adversarial Robustification, which perturbs joint embeddings by one projected gradient step, and Rank-Constrained Subspace Learning, which aligns the resulting edit-layer states with a singular-value loss while detaching the original state. Experiments on MMEdit and MMKE-Bench cover several multimodal large language models and editing methods. The results show improved generalization while largely preserving editing success on the original input and performance on unrelated knowledge.

Critical strengths
  1. The main idea is a genuine shift in how an edit is supervised. Instead of anchoring the update to one multimodal sample, ASAM builds a latent adversarial neighborhood and collapses it onto a shared low-rank direction at the edit layer, while detaching the original state so reliability and locality are not rewritten.

  2. The same alignment term transfers across WISE, MEND, T-Patcher, and UniKE, and across encoder–decoder and recent decoder-centric MLLMs. Generality rises without replacing the host editor, for example WISE on BLIP2-OPT E-VQA (84.80 to 93.97) and T-Patcher on Qwen2-VL E-VQA (56.03 to 83.17).

  3. The technical combination is novel rather than a direct reuse of existing editing losses. It is distinct from paraphrase augmentation, which rewrites surface text, and from pairwise feature matching, which never requires a rank-1 subspace at the edit layer.

  4. The paper addresses a problem that matters for multimodal editing in practice. A reliable edit is of limited use if it breaks under a rephrased question or a shifted image, and closing that generality gap is what makes an edit usable beyond the original query.

Critical weaknesses
  1. The connection between the theoretical condition and the evaluation metric needs a clearer explanation. Proposition 1 concerns whether normalized edit-layer representations are identical, whereas the experiments measure whether generated answers exactly match the reference answers. The paper should briefly explain when identical representations lead to identical answers, even if the argument assumes a fixed decoder.

  2. The definition of locality in Eq. (1) is unclear in two places. First, it does not explicitly identify ( Ki\mathcal{K}_i) as the knowledge unit being edited. Second, the equation compares full conditional probability distributions, while the preceding text defines discrete predictions. Clarifying the edited unit and making the notation consistent with the evaluation would improve this definition.

Questions for authors
  1. The original representation does not receive gradients from the alignment loss, but it still participates in the singular-value calculation. Is it therefore a fixed reference, with the alignment loss moving only the perturbed representations toward it? How does this design relate to the different degrees of editing scope discussed in CH3?

  2. The text introduces both (Lrobust\mathcal{L}{\text{robust}}) and (Ltotal=Lrel+Lloc+βLalign\mathcal{L}{\text{total}}=\mathcal{L}{\text{rel}}+\mathcal{L}{\text{loc}}+\beta\mathcal{L}{\text{align}}). Which objective is actually optimized during training? Are the reliability loss (Lrel\mathcal{L}{\text{rel}}) and locality loss (Lloc\mathcal{L}_{\text{loc}}) taken directly from the underlying editor without modification?

  3. For memory-based editors such as WISE, is ASAM used only when learning the edit, or does it also change retrieval at test time? A brief explanation of how ASAM connects to the underlying editor would clarify which stages it affects.

3 comments

Dear Reviewer,

Thank you for the positive assessment and for these questions. We make each point precise below, with the corresponding revisions.

W1. Representation-level alignment versus answer-level generality

Proposition 1 concerns edit-layer representation geometry, while answer-level generality is evaluated through predictions. We now explain how the two are connected.

Identical complete conditioning states with deterministic decoding yield identical predictions. Proposition 1 concerns normalized representations at the edit layers, which are identified as the locus of knowledge. Identical normalized representations there correspond to the same knowledge, and, as ROME argues, the answer is determined at these layers, so answer consistency follows. We therefore treat Lalign\mathcal{L}_{\mathrm{align}} as a representation-level surrogate for consistency within a knowledge unit.

W2. Locality definition and the edited knowledge unit

The revision identifies the edited knowledge unit in Eq. (1) and matches the discrete prediction metric.

For an edit on knowledge unit Ki\mathcal{K}_i, locality is evaluated on input pairs outside that unit, and Dout,i\mathcal{D}_{\mathrm{out},i} contains no q=(xv,xt)q=(x_v,x_t) from Ki\mathcal{K}_i.

The metric measures preservation of discrete predictions. With reference answer y∗=(y1∗,…,yTq∗)y^*=(y_1^*,\ldots,y_{T_q}^*) and predicted token y^θ,t(q,y∗_<t)=arg⁡max⁡v∈Vpθ(v∣q,y∗_<t)\hat y_{\theta,t}(q,y^*\_{<t})=\arg\max_{v\in\mathcal{V}}p_{\theta}(v\mid q,y^*\_{<t}),

Loc⁡i=Eq∼Dout,i[1Tq∑t=1TqI(y^θ′,t(q,y∗<t)=y^θ,t(q,y∗_<t))],\operatorname{Loc}_i=\mathbb{E}_{q\sim\mathcal{D}_{\mathrm{out},i}}\left[\frac{1}{T_q}\sum_{t=1}^{T_q}\mathbb{I}\left(\hat y_{\theta',t}(q,y^*{<t})=\hat y_{\theta,t}(q,y^*\_{<t})\right)\right],

i.e., whether the edited model preserves the original predictions on out-of-scope inputs.

Q1. Detached reference and its relation to editing scope

Yes. The original representation is a per-step detached reference, Hs=[sg⁡(h^0);h^1;…;h^n]H_s=[\operatorname{sg}(\hat h_0);\hat h_1;\ldots;\hat h_n], with sg⁡\operatorname{sg} stop-gradient. No gradient flows through its branch, so the alignment gradient acts through the perturbed representations, while the reliability and locality terms keep the original path.

On CH3, we move beyond single-sample supervision. LAR builds a bounded latent neighborhood around the input, and RCSL regularizes the edit-layer representations of these variants. The effect is to broaden local training coverage, and no continuous editing-scope boundary is learned. ASAM guarantees neither continuous nor monotonic growth of influence. The symmetric-versus-asymmetric ablation isolates this routing choice.

Q2. Optimized objective and host-editor losses

ASAM adds only one objective. For host editor EE,

LASAM(E)=Lhost(E)+βLalign,\mathcal{L}_{\mathrm{ASAM}}^{(E)}=\mathcal{L}_{\mathrm{host}}^{(E)}+\beta\mathcal{L}_{\mathrm{align}},

where Lhost(E)\mathcal{L}_{\mathrm{host}}^{(E)} is EE's original objective and Lalign\mathcal{L}_{\mathrm{align}} the singular-value regularizer on the input and its LAR variants, weighted by β\beta.

Editors with explicit reliability and locality objectives have Lhost(E)=Lrel+Lloc\mathcal{L}_{\mathrm{host}}^{(E)}=\mathcal{L}_{\mathrm{rel}}+\mathcal{L}_{\mathrm{loc}} with original coefficients, so ASAM only adds βLalign\beta\mathcal{L}_{\mathrm{align}}. Editors that preserve implicitly through memory or retrieval may lack Lloc\mathcal{L}_{\mathrm{loc}}. There the host objective stays whole and only the regularizer is added.

We will use Lalign\mathcal{L}_{\mathrm{align}} consistently, revise Lrobust\mathcal{L}_{\mathrm{robust}} so it no longer suggests a competing objective, and specify where the term enters each editor, including MEND.

Q3. Memory-based editors and test-time retrieval

ASAM is an editing-time regularizer with no new inference-time retrieval. LAR generates variants during edit construction, and the alignment loss is evaluated on their edit-layer representations, shaping the edit that WISE writes to memory.

The architecture, retrieval keys, scoring, and routing rule remain WISE's. Memory contents and representations may differ because the edit is optimized under the additional constraint, so retrieval outcomes can change indirectly. Edit-induced changes in activation statistics can affect the gate's decisions. This is an effect of the edited model itself, since no retrieval stage is added.

At inference, no variants are generated and neither loss nor singular-value computation is performed. The model uses the host's original forward and retrieval procedures. MEND, T-Patcher, and UniKE likewise add no test-time module or extra computation.

Thank you again for the constructive feedback. Every comment above has been addressed in the revised manuscript, with all changes highlighted in blue and cross-referenced to this response to confirm that each revision is traceable. We would be glad to clarify any remaining point.

Thank you for the clarifications.

The response clarifies the consistency between representation-level alignment and answer-level evaluation. The additional ablation study also provides empirical support for the necessity of stopping gradient propagation through the original representation.

Overall, the rebuttal partially addresses the concerns.

AuthorsOpenReview ↗

Thanks for recognizing our work and your valuable suggestions

Thank you very much for your recognition and suggestions. We really appreciate your time and effort in reviewing this paper! We will incorporate these additional experimental results and corresponding analyses introduced in the rebuttal into the revised version. We sincerely thank you again for your valuable comments.

Reviewer_K682OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ASAM is a plug-and-play method for improving generality in multimodal knowledge editing. For each edit sample, LAR applies one step of gradient ascent to the concatenated image-text embedding and generates several perturbed versions that make it harder for the model to give the target answer. RCSL then normalizes the hidden states of the original input and these versions at the editing layer, and increases the proportion of the first singular value. With Asymmetric Gradient Flow, the original hidden state is detached in the alignment loss, so the gradient of this loss only passes through the perturbed versions. The goal is to make one edit apply to more equivalent inputs while preserving unrelated knowledge. ASAM is added as an extra loss term to WISE, MEND, T-Patcher, and UniKE, and is evaluated on E-VQA and E-IC from MMEdit, as well as on MMKE-Bench.

Critical strengths
  1. The paper brings established ideas from adversarial learning and robust optimization into knowledge editing, which is a worthwhile attempt. Since the perturbed versions are generated from a single edit sample, the method does not need extra equivalent image-text samples. It is also added as an extra loss term, so it can be used with several existing editors.

  2. Gen. improves in 28 of the 32 comparisons in Table 1, and some of the gains are large. On E-VQA, for example, Gen. increases from 84.80 to 93.97 for BLIP2-OPT + WISE, and from 56.03 to 83.17 for Qwen2-VL + T-Patcher. The gain also holds in sequential editing: with WISE, E-VQA Gen. after 100 edits increases from 34.90 to 50.90 (Table 2).

Critical weaknesses
  1. Some gains in Gen. come with a clear drop in M-Loc., and the paper should explain and discuss this trade-off in more detail. On E-VQA, Qwen2-VL + T-Patcher improves Gen. from 56.03 to 83.17, while M-Loc. drops from 68.62 to 46.86. In sequential editing with WISE, M-Loc. after 100 edits drops from 7.33 to 3.38 on E-VQA and from 9.91 to 5.72 on E-IC. These results do not fully support the claim that ASAM maintains locality.

  2. The paper does not directly compare ASAM with ODEdit, a closely related plug-and-play method. ODEdit also targets generalization in multimodal knowledge editing and, like ASAM, adds extra training objectives to WISE, MEND, T-Patcher, and UniKE. Although the paper cites ODEdit, it does not compare with it under the same setting. The two papers also report different results for the base editors, so their tables cannot be compared directly. As a result, the current experiments show that ASAM improves the base editors, but not whether it has an advantage over ODEdit.

Questions for authors
  1. Does ASAM still achieve higher Gen. at a similar M-Loc.? It would help to show how the two metrics change with different values of β. Could the authors also provide some heuristic analysis or practical guidance on when both metrics improve together and when a clear trade-off appears?

  2. Could the authors compare ASAM with ODEdit under the same models, data, and base editors, and report Rel., Gen., T-Loc., M-Loc., and computational cost? It would also help to discuss how the two methods differ and relate to each other, and whether combining them further improves the metrics.

2 comments

Dear Reviewer,

Thank you for the careful reading. We address both points with additional controlled experiments.

W1 / Q1. Generality-locality trade-off and β\beta

ASAM is additive: the host's native locality term retains its original form and weight, while ASAM adds only βLalign\beta\mathcal{L}_{\mathrm{align}}, discarding no preservation term. For Qwen2-VL + T-Patcher on E-VQA, Gen. improves from 56.03 to 83.17 (+27.14), while M-Loc. decreases from 68.62 to 46.86 (−21.76). For WISE + BLIP2-OPT over 100 sequential edits, Gen. improves from 34.90 to 50.90 (E-VQA) and 56.30 to 67.99 (E-IC), while M-Loc. decreases from 7.33 to 3.38 and 9.91 to 5.72, respectively. These results show substantial generality gains with relatively small absolute locality losses in the sequential setting.

The coefficient β\beta controls the contribution of alignment relative to the host objective. Larger values emphasize representation consistency, potentially improving generality but increasing interference with knowledge preservation. Thus, β\beta navigates rather than eliminates the generality-locality trade-off. Temperature τ\tau further controls the singular-value distribution and alignment strength. We recommend selecting β\beta using the validation Pareto frontier to balance generality gains and locality degradation.

W2 / Q2. Same-setting comparison with ODEdit

ODEdit is the closest plug-and-play comparator. However, results from the original papers are not directly comparable due to differences in base-editor performance and experimental settings. We therefore re-evaluate both methods using identical BLIP2-OPT and WISE backbones, edit requests, data splits, preprocessing, and evaluation protocols. Each method receives a matched validation grid and run budget.

DatasetMethodRel.Gen.T-Loc.M-Loc.
E-VQAWISE99.0084.80100.08.99
E-VQA+ ODEdit99.0088.5799.8811.11
E-VQA+ ASAM99.0093.97100.09.73
E-ICWISE100.082.5499.8410.28
E-IC+ ODEdit100.083.0797.0915.05
E-IC+ ASAM100.086.22100.010.74

Both methods improve over WISE, but ASAM achieves larger Gen. gains on both datasets (+9.17 vs. +3.77 on E-VQA; +3.68 vs. +0.53 on E-IC), obtaining the highest Gen. in both settings. ASAM also maintains base-level Rel. and T-Loc. while improving M-Loc. (+0.74 and +0.46). ODEdit achieves higher M-Loc. but reduces T-Loc. on E-IC (97.09 vs. 99.84).

Their computational overheads also differ. ODEdit introduces a trainable three-layer MLP and primal-dual optimization, whereas ASAM requires additional forward/backward computations for gradient-guided perturbation and representation alignment, without an auxiliary trainable network.

Methodologically, ODEdit models tripartite out-of-distribution risks using KL divergence for locality and multi-scale Gaussian-RBF MMD for generality, optimized through a primal-dual procedure. ASAM instead generates adversarial variants directly in the joint vision-language embedding space and applies temperature-scaled spectral alignment to edit-layer representations. Unlike ODEdit, which relies on semantic rephrasings, ASAM generates latent variants from original edit samples without additional equivalent image-text pairs. Thus, ODEdit constrains output and feature distributions, whereas ASAM explicitly encourages low-rank consistency in edit-layer representations. Their distinct constraint spaces suggest complementary approaches to multimodal knowledge editing.

Q3. Combining ODEdit and ASAM

We agree that combining the two methods is promising. Their different optimization mechanisms and constraint spaces may offer complementary benefits. However, overlapping objectives could also introduce conflicting gradients, affecting locality and computational efficiency. As this combination has not been experimentally evaluated, we leave its effectiveness open and identify it as future work.

Thank you again for these constructive questions. The additional experiments establish a fair comparison with ODEdit and further characterize ASAM's generality-locality trade-off. ASAM achieves larger generality gains on both datasets while maintaining base-level reliability and T-Loc., whereas ODEdit achieves higher M-Loc. These findings clarify the respective strengths and limitations of both methods. Every comment above has been addressed in the revised manuscript, with all changes highlighted in blue and cross-referenced to this response to confirm that each revision is traceable. We would be glad to clarify any remaining point.

Dear Reviewer K682:

Thank you very much for your appreciation of our paper and for your insightful suggestions, which have helped us further improve our work. We have tried our best to address all of your concerns in our responses. Please let us know if any questions remain, and we will be happy to provide further clarification.

Thanks again!

Authors

Reviewer_RH89OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ASAM, a plug-and-play framework to improve generality in multimodal knowledge editing while preserving reliability and locality. The method combines latent adversarial perturbations with rank-constrained representation alignment. Specifically, LAR generates challenging variants through gradient-guided perturbations in the joint vision-language embedding space, while RCSL uses an SVD-based objective to encourage consistent hidden representations across these variants. The authors evaluate ASAM with four existing knowledge editors across several multimodal models on MMEdit and MMKE-Bench. Experimental results show improvements in generality across most settings, with reliability and locality largely preserved, although some configurations exhibit performance trade-offs.

Critical strengths

(1) The paper is well organized and clearly written.

(2) The overall design is reasonable and well motivated. LAR uses gradient-guided perturbations to construct challenging variants, while RCSL encourages consistency across their hidden representations. Setting aside the semantic preservation assumption, I find this a coherent regularization approach to improving editing generality.

(3) The plug-and-play design is a useful aspect of the method. ASAM can be integrated into different knowledge editing frameworks, and the experiments cover multiple multimodal backbones and editing methods. The results show improvements in generality across a range of settings.

Critical weaknesses

(1) Small perturbations in the joint embedding space do not necessarily preserve task-relevant semantics, raising questions about whether the generated adversarial variants are semantically equivalent to the original inputs. Although Table 3 compares LAR with original samples and SimCSE-based paraphrases, these comparisons do not isolate the contribution of adversarial gradient guidance. A more direct comparison with norm-matched random latent perturbations, as explored in prior multimodal knowledge editing work, would help determine whether adversarial optimization offers benefits beyond random latent augmentation. An analysis of the prediction loss induced by these perturbations would also help characterize the generated variants.

(2) As discussed by [1], conventional knowledge editing metrics may not fully reflect performance under autoregressive generation. An additional LLM-as-a-Judge evaluation could help strengthen the empirical findings. In particular, it would be useful to examine whether the generality-locality trade-off observed in some settings persists under this evaluation.

(3) While ASAM improves generality in most settings, it does not consistently preserve locality, with some configurations showing noticeable degradation. Balancing knowledge updates and preservation has been extensively studied in knowledge editing, and this trade-off remains an important limitation of the proposed method.

[1] The Mirage of Model Editing: Revisiting Evaluation in the Wild, ACL 2025.

Questions for authors

(1) Could the authors provide a simple ablation comparing asymmetric gradient flow with symmetric gradient flow (i.e., without detaching the original representation) to examine its impact on generality and locality?

(2) LAR applies joint perturbations to both visual and text embeddings. To what extent does each modality contribute to the improvement in generality? Is a joint perturbation more effective than perturbing either modality individually? It is recommended to include comparative experiments for "visual-only," "text-only," and "joint" perturbations.

(3) The paper presents ASAM as a plug-and-play framework across different knowledge editors. However, these editors employ different parameter update mechanisms, and Appendix A.3 provides limited details on how the alignment loss is integrated into each method. Could the authors clarify which parameters receive the alignment gradients and how the original optimization procedures are adapted?

(4) Could ASAM be combined with existing knowledge-preservation techniques, such as null-space constraints, to mitigate the observed locality degradation while retaining its generality improvements?

4 comments

Dear Reviewer,

Thank you for the careful reading and for recognizing the motivation, organization, and transferability of ASAM. We answer each point with new controlled experiments.

W1. Random-perturbation control and the prediction loss

LAR builds local variants that act as a robustness proxy with t-SNE and similarity plots as qualitative evidence on geometry. The question is whether adversarial guidance beats random augmentation. We ran an additional experiment with random augmentation on WISE + BLIP-2 using the E-IC dataset.

VariantRel.Gen.T-Loc.M-Loc.
ASAM100.086.22100.010.74
Random100.085.39100.09.35

Adversarial guidance dominates. ASAM consistently matches or outperforms random augmentation across all four metrics, with clear improvements in Gen. (+0.83) and M-Loc. (+1.39), while maintaining identical Rel. and T-Loc., and the gain therefore demonstrates the performance of gradient guidance.

Regarding prediction loss, we agree that analyzing the induced prediction changes is important for characterizing the generated variants. Importantly, small perturbation norms alone do not guarantee semantic preservation. We therefore distinguish the local prediction sensitivity induced by adversarial perturbations from semantic equivalence of the corresponding inputs.

W2. LLM-as-a-Judge evaluation

We re-evaluated ASAM under WILD, the LLM-as-a-Judge protocol you point to, and will cite it in the camera-ready. The setup is WISE + BLIP-2 on E-IC and E-VQA, with one frozen judge for all rows.

DatasetMethodRel.Gen.T-Loc.M-Loc.
ICBaseline92.042.078.017.0
IC+ ASAM96.051.590.023.0
VQABaseline31.032.521.011.0
VQA+ ASAM34.037.038.016.0

The result is uniform. ASAM improves all four metrics on both datasets (+9.5 Gen. on IC, +17.0 T-Loc. on VQA), so the generality gains persist under evaluation in the wild and do not depend on exact match alone.

W3. The locality trade-off

The objective change is additive. The host's native locality term keeps its form and weight, ASAM adds only$\beta\mathcal{L}_{\mathrm{align}}$, and no preservation term is discarded. The generality gain does trade against locality. Gen. rises while M-Loc. falls in both the single-edit and sequential settings. However, in the sequential WISE runs the locality baselines are already low, so the absolute loss is small, and the revision reports relative deltas alongside absolute points.

Q1. Asymmetric versus symmetric gradient flow

The setting matches W1.

VariantRel.Gen.T-Loc.M-Loc.
ASAM (asym.)100.086.22100.010.74
Symmetric99.8585.3199.9510.04

Asymmetric routing is better on all four metrics (Gen. +0.91, M-Loc. +0.70), consistently supporting the design.

Q2. Modality ablation

The setting matches W1, and the reference row is joint.

VariantRel.Gen.T-Loc.M-Loc.
ASAM100.086.22100.010.74
Text-only99.9185.2099.2910.25
Image-only100.084.63100.09.40

Only the joint perturbation attains the best Gen. and M-Loc. simultaneously, while text-only and image-only trail it on Gen. by 1.02 and 1.59 points. This supports joint perturbation.

Q3. Integration with different editors

The objective is$\mathcal{L}_{\mathrm{ASAM}}^{(E)}=\mathcal{L}_{\mathrm{host}}^{(E)}+\beta\mathcal{L}_{\mathrm{align}},where, whereE$ is the host editor, whose native objective keeps its own terms and weights. The alignment gradient is taken over each editor's designated trainable tensors. For WISE these are the masked MLP weight and activation memory, for T-Patcher the patched MLP weights, for UniKE the adapter MLPs and edit weights, and for MEND the update-generating network during meta-training, with the edit-time archive from that run. The paths differ in whether the gradient is first-order in the edited state or passes through an update-generating network, and the revision states modules, feature location, and update path per editor.

Q4. Combining ASAM with preservation constraints

In principle, a null-space projection can constrain parameter updates to reduce interference with previously acquired knowledge, while ASAM provides additional supervision for cross-modal generality. Importantly, the projection should be applied to the effective parameter update so that the preservation constraint is respected by the combined update. However, the compatibility depends on the update mechanism of each editor, and a locally estimated null space does not guarantee global behavioral preservation. We will clarify this distinction and discuss the integration with methods such as AlphaEdit as a promising extension.

Thank you again for the constructive suggestions. Every comment above has been addressed in the revised manuscript, with all changes highlighted in blue and cross-referenced to this response to confirm that each revision is traceable. We would be glad to clarify any remaining point.

Dear Reviewer RH89:

Thank you very much for your appreciation of our paper and for your insightful suggestions, which have helped us further improve our work. We have tried our best to address all of your concerns in our responses. Please let us know if any questions remain, and we will be happy to provide further clarification.

Thanks again!

Authors

Thanks for the response. I appreciate the wild evaluation results. However, as perturbation-based approaches are not new to multimodal knowledge editing, I still have some reservations about the connection between embedding-space perturbations and generality, especially given the modest gains over random perturbations. Therefore, I maintain my original rating.

Dear Reviewer,

Thank you for the second look and for acknowledging the WILD results. We separate the two objections.

Part 1. What is new is not perturbing, but what the perturbation is used for

It is true that perturbation-based approaches have been used in multimodal knowledge editing before, but we do not claim perturbation itself. Our contribution is two-fold: the LAR design, i.e., how the probe is built and used, as a norm-bounded, gradient-guided variant that serves only for alignment and never as an extra supervision target; and more importantly, what the ASAM framework does with these probes, namely aligning them under a rank constraint at the edit layer and routing the alignment gradient one-sidedly. The three points below separate the probe from the constraint that gives it effect.

(i) The perturbation is used for probing instead of as a training sample. Perturbed inputs are never supervised against their own predictions; they only populate the state matrix Hs\mathbf{H}_s, from which one geometric constraint is derived, and are then discarded. The same pipeline with random directions of equal norm gives 85.39 Gen / 9.35 M-Loc. instead of 86.22 / 10.74, and perturbing only one modality gives 85.20 or 84.63. The quality and coverage of the probe are the variables.

(ii) The constraint acts on the singular spectrum instead of on samples. Replacing the rank constraint with cosine or ℓ2\ell_2 alignment on the same probes drops E-IC Gen to 83.05 and 83.00. Pairwise terms constrain each variant separately; the rank term constrains the span they occupy together.

(iii) The constraint is one-sided. With the anchor detached, the probes move toward it but not the reverse. Symmetric flow gives 85.31 / 10.04 against 86.22 / 10.74, which is why host losses and their weights stay untouched and no test-time module is added.

Part 2. Why an embedding perturbation reaches generality

Semantic equivalence does not imply representational proximity: inputs that should share an answer occupy a region, not a point, and their distance from the anchor predicts how far the answer drifts. Let h(x)h(x) be the edit-layer state, ff the frozen upper layers, and Uη={x:D(p(⋅∣x),p(⋅∣x0))≤η}U_\eta=\{x:D(p(\cdot|x),p(\cdot|x_0))\le\eta\}. Membership is a behavioural budget, not a norm ball: at equal norm some directions stay in UηU_\eta and others leave it. A conventional edit constrains h(x0)h(x_0) alone; the set BB on which ff returns the target is a neighbourhood whose boundary radius, not the number of paraphrases, bounds the covered part of UηU_\eta.

At equal radius the two probes differ. The answer depends on a low-dimensional sensitive subspace, so a random displacement puts only an O(k/d)O(\sqrt{k/d}) fraction of its length into it and with high probability lands in the interior of BB, where the alignment gradient carries no information. The gradient of the answer loss is to first order the projection onto that subspace, i.e. an estimate of the outward normal of ∂B\partial B. Supervising there constrains the boundary rather than the interior: the difference between moving BB and constraining it where it is already satisfied.

The same account predicts the sign of the locality effect. Directions inert for the edited unit are shared with out-of-scope inputs, so random probing perturbs unrelated predictions (M-Loc. 9.35, below the editor's 10.28) while the edit-specific probe does not (10.74). Locality is a signature of probing the right direction, not a trade we accept.

Supervising at hh rather than at the input intervenes at the mediator: no human-readable paraphrase is needed, and the constraint transfers to any editor writing at that layer. Proposition 1 is its rank form, since coincident normalized states imply the same next-token distribution under frozen upper layers, so the constraint reaches the probe set, not the anchor alone.

Two consequences are testable and hold. The gain survives at ϵ=10−5\epsilon=10^{-5} and Gen varies by less than 0.5 point over four orders of magnitude (85.76-86.23), as a first-order direction estimate predicts and augmentation does not; and Gen rises monotonically in β\beta (83.29 to 87.04), as a constraint-based account predicts.

On magnitude

E-IC single edit saturates Rel. and T-Loc. at 100.0, leaving only Gen and M-Loc. observable. There +0.83 Gen and +1.39 M-Loc. (+14.9% relative) come against a random arm below the editor on locality. This is our least informative setting: E-VQA Gen moves 92.00 to 93.97, sequential editing 34.90 to 50.90, and Gen improves in 28 of 32 comparisons. The effect is bounded by the unit's boundary layer, so it is smallest at ceiling.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear reviewers and area chair,

We thank you for your time in reviewing and discussing our submission. Below, we summarize the key outcomes of the review and discussion process.

1. Recognized Strengths across Reviewers

  • A genuine shift in how an edit is supervised, and a novel design rather than a reuse of existing editing losses (GS2W, K682).
  • A well-organized, clearly written, and easy-to-follow presentation, with a reasonable and well-motivated design (RH89, GS2W).
  • A plug-and-play framework that transfers across WISE, MEND, T-Patcher, and UniKE, and across encoder–decoder and decoder-centric MLLMs (RH89, K682, GS2W).
  • Substantial gains across the majority of metrics, with gains that persist in sequential editing (K682, GS2W).
  • A practical and important problem of multimodal knowledge editing (GS2W).

We are also grateful to GS2W, who confirmed that our response clarifies the representation-to-answer connection and that the new ablation supports the asymmetric gradient routing, and to RH89, who acknowledged the WILD evaluation results.

2. Comprehensive Responses Addressing All Reviewer Concerns

Reviewer GS2W confirmed that our response have solved his concerns and maintain the positive score (Clear Accept). Reviewer RH89 returned to the discussion, acknowledged the WILD results. Reviewer K682 posted no follow-up. We address the remaining reservation in the first row below.

ReviewerScoreConcerns & Our ResponsesRevisions in Paper
RH8921. Control against random latent perturbation. On E-IC at identical Rel. and T-Loc. (100.0 both), ASAM reaches 86.22 vs. 85.39 Gen. and 10.74 vs. 9.35 M-Loc. The headroom is therefore in Gen. and M-Loc. (+0.83, +1.39; +14.9% relative on M-Loc.), and the gain comes from rank-constrained alignment with asymmetric routing, not from perturbing latents: symmetric 85.31, text-only 85.20, image-only 84.63 vs. 86.22.App. B.1, B.3, B.4 (Tables 9, 11, 12)
2. LLM-as-a-Judge evaluation. Re-evaluated under WILD on E-IC and E-VQA, all four metrics improve (+9.5 Gen. on E-IC, +17.0 T-Loc. on E-VQA), so the gains are not an exact-match artifact.App. B.2 (Table 10)
3. Consistent locality preservation. The claim is now qualified: near-preserved Rel. and T-Loc. with editor-dependent M-Loc. trade-offs; the cited drops occur where base M-Loc. is already 7.33 and 9.91.Sec. 5.2; App. F
4. Asymmetric versus symmetric gradient flow. Asymmetric routing is better on all four metrics (+0.91 Gen., +0.70 M-Loc.) on E-IC, supporting the detached reference.Sec. 4.3; App. B.3 (Table 11)
5. Contribution of each modality. Text-only and image-only perturbations trail the joint one by 1.02 and 1.59 Gen.; only joint is best on Gen. and M-Loc. together.Sec. 4.1; App. B.4 (Table 12)
6. Per-editor gradient routing. We document which trainable tensors receive the alignment gradient in WISE, MEND, T-Patcher, and UniKE.App. A.3
7. Combination with null-space constraints. We discuss how a null-space projection composes with ASAM, keeping the host locality term unchanged.App. F
K6822 (No response)1. Trade-off between generality and locality. Stated in the main text and App. F; the cited cases are in Table 1. Gen. improves in 28 of 32 comparisons, and in 16 without any decrease in M-Loc.Sec. 5.2; App. F
2. Same-setting comparison with ODEdit. Both methods are re-run on a single edit with WISE + BLIP2-OPT, sharing checkpoint, requests, splits, preprocessing, and decoding: 93.97 vs. 88.57 Gen. on E-VQA and 86.22 vs. 83.07 on E-IC, while ODEdit has higher M-Loc. but lower T-Loc. on E-IC.App. B.5 (Table 13)
3. Behavior of the two metrics under β. β and τ are ablated with a heuristic reading, recommending the smallest β that suppresses the secondary singular values.App. A.4
GS2W4 (Clear acceptance)1. Alignment versus answer-level generality. Proposition 1 is restated in normalized form, and the answer-level implication is spelled out.Sec. 4.2 (Prop. 1); App. A.2
2. Definition of locality in Eq. (1). Locality is now evaluated on the out-of-scope set excluding the edited unit, with a token-level metric definition.Sec. 3; App. A.2
3. Gradient through the original representation. The original state is a detached anchor in the alignment loss; the symmetric-versus-asymmetric ablation isolates this choice.Sec. 4.3; App. B.3
4. Optimized objective. The host objective plus β times the alignment term, with the host terms and weights unchanged.Sec. 4.3
5. Test-time behavior. ASAM is an editing-time regularizer only, with no inference-time retrieval or extra module.App. A.7

Revised passages are marked in blue with cross-referenced to this response.

Thank you again for your time and feedback.

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.