acceptodds
Under review as a conference paper at ICLR 2027

Locating and Editing Knowledge-Concerned Neurons in Vision Large Language Models

Abstract

Knowledge editing (KE) enables large language models (LLMs) to efficiently correct outdated or erroneous information without costly retraining. Recently, locate-then-edit has emerged as an effective paradigm due to its efficiency and interpretability, offering an alternative to memory-based approaches and methods that require training auxiliary editors. However, in vision large language models (VLLMs), factual knowledge is encoded across both visual and textual representations, making knowledge localization fundamentally different from that in text-only LLMs. Existing multimodal knowledge editing methods, such as MKE, typically rely on additional training, which may introduce out-of-distribution challenges. To address this gap, we propose **DELTA** (), to the best of our knowledge, the first fine-grained neuron-level locate-then-edit framework that is specially designed for multimodal knowledge editing and requires no auxiliary editor optimization. DELTA first identifies candidate layers by jointly modeling cross-modal fusion, feed-forward network engagement, and activation traceability. It then localizes knowledge-relevant neurons through attribution scores and visual filtering. For knowledge updating, **DELTA** constrains parameter changes toward an approximate null space to preserve unrelated knowledge, while perturbing visual embeddings to account for diverse visual evidence. Extensive experiments demonstrate that DELTA consistently outperforms existing knowledge editing approaches, highlighting the effectiveness of locate-then-edit for VLLMs and providing a step toward general-purpose multimodal knowledge editing.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
model editing, multimodal knowledge editing, large vision-language model, neuron attribution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HVFdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed DELTA, a multimodal knowledge-editing method that directly modifies the FFN output weights without additional training of an editor. It first selects the layers using the image-text pairs correlations, engagements and activation-score entropy. Then, DELTA ranks neurons using the modality-specific activation ratios and exclude the neurons heavily weighted by visual features. The overall method is optimized with prefix preservation, repetition, null-space regularization and visual perturbation constraints. Experiments with three main backbones and datasets plus additional Qwen results support the claim of this paper.

Critical strengths
  1. A good motivation from current unsolved gaps in the VLLM domain and concrete solutions to achieve the editing procedure. Multimodal editing expects to achieve the successful answers without hurting the visual and textual behaviours. DELTA, serves as a good method, balance the success and damage of VLLM editing task efficiently.
  2. The results reported in the paper strongly support the claims by evaluating among different backbones and datasets. DELTA has the highest reported average among that family in all nine main model–dataset settings. E.g. on LLaVA/VLKEB, its average is 94.74 while FiNE and ROME only have 87.45 and 83.28, respectively.
  3. The ablation experiments show useful evidence that the preservation components contribute. For example, on BLIP2/VLKEB, removing the null-space component reduces modal locality from 87.80 to 80.49; removing visual perturbation reduces it to 80.93. Reliability remains 100.00 in both cases.
Critical weaknesses
  1. The criteria to select the layers are not strong and robust. Activation correlation, mean cores and entropy are plausible but do not represent the causal fusion where the particular fact is included.
  2. The novelty claims need further improvements. Some prior work like M2Edit and NAM have done some overlapped jobs and DELTA may contribute a distinct localization rule and preservation strategy.
  3. Generality claims exceed the real capability of the paper. The localization formulation requires visual and textual positions to pass through common FFN neurons. But it does not directly cover a separate visual-memory architecture accessed through decoder cross-attention, which makes the paper architecture-specific.
Questions for authors
  1. In Appendix C, it constructs the activation matrix A from the edit inputs, controlling the direct FFN perturbation. However, it does not show the claim that the activation matrix building upon an unrelated input is small, which means the locality guarantee does not build the relationship to the unrelated inputs.
  2. A discrepancy occurs in Appendix I: Table 5 gives identical layer sets for configurations (a) and (e), while Table 7 reports average scores of 48.21 and 66.55. The paper does not explain them.
  3. What’s the significant improvements (theoretical and computational) from LoRA? Maybe more experiments of comparison between LoRA and DELTA are rational.
  4. Figure 1 has some overlapped arrows and labels which are hard to visualize. Figure 2, 3 and 6 are too small to see.
6 comments
AuthorsOpenReview ↗

Answer to W1, W2, and W3

Dear Reviewer HVFd:

Thank you for the exceptionally careful reading of our work! We are grateful for the comments help us improve the works. We sincerely hope that our responses and new experiments could address your concerns!


W1. - The layer-selection criteria are not strong and robust: activation correlation, mean scores, and entropy are plausible but do not represent the causal fusion where the particular fact is included.

We thank the reviewer for this insightful feedback. In the revised manuscript, we report the causal analysis in Appendix K. The results in Fig. 15 and Fig. 16 show that the result of DELTA localization considerably overlaps with the gaining of causal tracing. This indicates that our decoupled vision-text localization carries genuine causal information rather than merely correlating activation signals. Additionally, the localization results obtained from this causal tracing procedure (Appendix K) have already been used in the experiments in Tab. 3 that combine different localizers with different editors. Under the same editing method, editing causal-tracing-localized layers consistently underperforms editing DELTA-localized layers. These findings suggest that causal relevance and editing suitability are two distinct criteria: while causal tracing identifies fact-related components, DELTA further identifies the components that are more suitable for effective and localized parameter updates.


W2. - The novelty claims need further improvement: M2Edit and NAM have done overlapping work; DELTA may contribute a distinct localization rule and preservation strategy.

Thanks for this valuable review, and we appreciate the opportunity to state this precisely. DELTA contributes a more distinct localization and preservation strategy. Based on the illustration in the general response above, we would like to further clarify this as follows.

  1. Prior works address related but different problems. NAM [1] investigates whether MLLM outputs can be attributed to FFN neurons, using image-generation tasks to validate neuron attribution. M2Edit [2] considers language-information propagation and combines important components and layers, without the same fine-grained analysis of internal FFN neurons. In contrast, DELTA is specifically designed for multimodal knowledge editing: it uses a decoupled vision-text localization criterion to identify edit-relevant FFN neurons, rather than simply transferring an attribution or component-selection strategy to the editing setting.

  2. The potential localization-editability alignment. Importantly, identifying components associated with a fact does not necessarily identify the most suitable locations for knowledge editing. As discussed in our response to W1, our causal-tracing analysis (Appendix K, Fig. 15 and Fig. 16) validates that DELTA's localization captures fact-related information. Simultaneously, the localization-editor recombination experiment in Tab. 3 also shows that using causal-tracing-localized layers under the same editor yields worse performance than using DELTA-localized layers. This demonstrates that causal relevance and editing suitability are distinct issues, and our DELTA targets the latter.

The reply to reviewer omH8 and the general response above also provide some illustrations. We updated claims of the contribution in the revision, and we will keep improving the illustrations. Thanks!


W3. - Generality claims exceed the paper's real capability: the localization formulation requires visual and textual positions to pass through common FFN neurons, and does not directly cover a separate visual-memory architecture accessed through decoder cross-attention — making the paper architecture-specific.

We are glad to make clearer explanation: DELTA only requires the VLLMs (i) consists of a decoder-only block with knowledge stored within the post-attention FFN, and (ii) visual signals enter the residual stream within the decoder via visual-token prefixes or cross-attention. Almost all current VLLMs satisfy both conditions, even for architectures with a separate visual memory accessed via decoder cross-attention (e.g., Flamingo). Our DELTA never modifies any vision-side modules (ViT, Q-Former, or projector), which are treated as black boxes. Its localization and edits target the FFN output projection (WdownW_{down}). Simultaneously, our experiments have included three LLM families (OPT in BLIP2, LLaMA/Vicuna in LLaVA and MiniGPT4, Qwen2.5/Qwen3-VL) and two visual-injection paradigms (Q-Former vs. patch-level prefixes).

AuthorsOpenReview ↗

Answer to Q1 and Q2

Q1. - Appendix C builds the activation matrix A\mathbf{A} from the edit inputs only; it is not shown that the corresponding matrix of an unrelated input is small, so the locality guarantee has no relationship to unrelated inputs.

Thank you for your feedback. This is an important issue, and we would like to clarify. We can be candid about the incomplete theoretical analysis in the original version. That's why we supplement a more detailed theoretical justification for the null-space constraint (you can refer to our response to reviewer omH8) in Appendix D, providing two complementary views towards unrelated input preservation.

In this revision, we illustrate that the null-space construction is built from the edit input only, and the separation between unrelated activations and the edit targets is real exsisted: the locality bound on unrelated inputs holds only under a measurable separation condition (Assumption 1), and this condition is currently supported by our experiments (Appendix D.4, D.5). In Tab. 6 (also listed within the response to reviewer omH8), the unrelated-input next-token KL falls monotonically from 0.04140.0414 to 0.00020.0002 as dnulld_{\text{null}} reduces from 2525 to 55. The scale-invariant alignment ∥A2Δ(ℓ) ⁣⊤∥F/(∥A2∥F∥Δ(ℓ) ⁣⊤∥F)\|\mathbf A_2{\boldsymbol\Delta^{(\ell)}}^{\!\top}\|_F/(\|\mathbf A_2\|_F\|{\boldsymbol\Delta^{(\ell)}}^{\!\top}\|_F) falls from 0.300.30 to 0.130.13. This matches Theorem 1's prediction that damage scales with MℓεlocM_\ell\varepsilon_{\text{loc}} and shrinks as the null cone tightens. We also measured Assumption 1 directly. 84.984.9% of unrelated positions have exactly zero trace-neuron activation.

These results show that part of the observed KL reduction on unrelated inputs comes from how Lns\mathcal{L}_{\text{ns}} shapes Δ(ℓ)\boldsymbol\Delta^{(\ell)} during optimization, rather than from strictly guaranteed orthogonality; the two panels of Tab. 6 separate the contributions: with the hard projection disabled, Lns\mathcal{L}_{\text{ns}} alone reduces the KL from 0.04160.0416 to 0.01550.0155 (2.7×2.7\times), while the hard projection itself accounts for the near-removal (0.00020.0002). We have therefore revised the claims in the paper, and we no longer claim an unconditional guarantee for arbitrary unrelated inputs. However, Lns\mathcal{L}_{\text{ns}} still empirically preserves unrelated knowledge to a meaningful extent and improves overall editing performance. We will keep on studying separation condition as our future works.

In addition, we also note that this preservation does not conflict with editability. Prompt positions are protected pointwise by Eqn. 40. Generation positions i∈[S,S+T−1]i\in[S, S+T-1] are by construction not rows of A(ℓ)\mathbf A^{(\ell)}, and they retain null-cone mass. In our measurements, trace neurons are active at 22.322.3% of generation positions versus 12.012.0% of prompt positions. Generation positions carry 10.410.4% of the cone energy versus 4.34.3% for prompt positions. This is the editing channel required by Assumption 2, and it stays open. We have provided the more complete assumptions, lemmas, and proofs regarding this issue in Appendix D. We will appreciate it if you can spare more time on checking the appended theorems.


Q2. - Appendix I discrepancy: Table 5 gives identical layer sets for configurations (a) and (e), yet Table 7 reports average scores of 48.21 vs. 66.55, unexplained.

Thanks for the debugging! Reviewer HVFd also pointed out similar issues. This is caused by a statistical reporting error in the (d)-(f) row when we recomputed and typed the results into the paper. The intersection layers row (a)-(c) are computed based on the (d)-(f) rows, which are therefore wrongly calculated. The correct (a) and (e) configuration is as below:

NotationL\mathcal{L}
(a){17,18,19,20,21,22,23,26,27,28,29,30,31}\lbrace17,18,19,20,21,22,23,26,27,28,29,30,31\rbrace
(e){17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}\lbrace17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\rbrace

, which should be different. We apologize for this oversight and recomputed the intersection layer set between each pair to fix Tab. 9. We once again checked the logs when running the experiments that produced Tab. 11 and confirmed the correctness of the runtime layer settings. We revised this reporting error in our revision and remarked the differences for review. Thanks.

AuthorsOpenReview ↗

Answer to Q3 and Q4

Q3. - What are the significant improvements (theoretical and computational) over LoRA? More LoRA-vs-DELTA comparison experiments would be rational.

We thank the reviewer for this insightful suggestion. Since DELTA directly updates the target neurons and the updating target Δ(ℓ){\boldsymbol\Delta}^{(\ell)} can be decomposed into a matrix pair (B,A)(\mathbf{B}, \mathbf{A}), LoRA can therefore be regarded as an extension of DELTA. We provide theoretical analysis based on the formulation in Appendix C and Appendix D. We conduct raw LoRA experiments based on the additional parameter-inserting layer given by VisEdit[3] with rank set to 3232, compared to DELTA and the LoRA extension with rank=4,8,32\text{rank}=4,8,32 for BLIP2 and rank=8,32,128\text{rank}=8,32,128 for LLaVA, respectively. The results are as follows:

BackboneMethod#Edits=1#Edits=10
RelM-GenT-GenM-LocT-LocAvgRelM-GenT-GenM-LocT-LocAvg
BLIP2-OPT-2.7BLoRA (r=32)80.244.559.078.962.364.957.228.040.155.240.944.3
DELTA (ours)96.394.985.584.099.692.086.076.176.961.691.978.5
+LoRA (r=4)98.597.693.159.797.389.379.575.669.928.489.968.7
+LoRA (r=8)100.099.495.157.197.089.778.776.971.727.591.169.2
+LoRA (r=32)100.099.595.152.896.988.979.475.071.724.889.968.1
LLaVA-1.5-7BLoRA (r=32)88.345.364.239.582.463.942.839.650.754.430.143.5
DELTA (ours)100.089.689.994.798.990.796.273.684.368.797.484.0
+LoRA (r=8)100.086.592.874.195.989.996.879.789.252.287.881.2
+LoRA (r=32)100.088.592.965.792.988.095.778.787.450.180.578.5
+LoRA (r=128)100.089.593.760.183.685.486.177.177.443.558.768.6

This demonstrates that LoRA can be a more parameter-efficient extension based on our method (raw LoRA underperforms DELTA), and can somehow enhance the generality in MKE; however, at the cost of damaging locality. Additionally, we report the time/memory cost in Appendix L (also listed in the response to reviewer QRCS). We have added theoretical analysis and these results into Appendix O. Thanks.


Q4. - Figure 1 has overlapping arrows and labels that are hard to visualize; Figures 2, 3, and 6 are too small to see.

We apologize for these visualization issues. Due to the upload size limit, the first submission compressed the figures heavily (mainly caused by original Figures 2,3, and 6). Now, the revised PDF reduces the number of per-panel cases to restore legibility and maintain clarity. Additionally, vector-format sources with full-resolution figures are already included in the supplementary materials.


Overall, we sincerely thank you for your thoughtful feedback and the considerable time and effort you have devoted to reviewing our work. We would be more than happy to provide any further illustrations or experimental results that may help address your remaining questions or suggestions.

Best,

Authors of Submission 11844.


References

[1] Fang, J., Bi, Z., Wang, R., Jiang, H., Gao, Y., Wang, K., Zhang, A., Shi, J., Wang, X. and Chua, T.S., 2024. Towards neuron attributions in multi-modal large language models. Advances in Neural Information Processing Systems, 37, pp.122867-122890.

[2] Zhou, Y., Cao, P., Chen, Y., Liu, Q., Sui, D., Chen, X., Liu, K. and Zhao, J., 2025, November. M2Edit: Locate and Edit Multi-Granularity Knowledge in Multimodal Large Language Model. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 29017-29030).

[3] Chen, Q., Zhang, T., Wang, C., He, X., Wang, D. and Liu, T., 2025, April. Attribution analysis meets model editing: Advancing knowledge correction in vision language models with visedit. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 2, pp. 2168-2176).

Reviewer_QRCSOpenReview ↗
Rating
3
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DELTA, a training-free locate-then-edit method for VLLMs. It selects candidate FFN layers using three activation-based criteria (cross-modal fusion, engagement, entropy), filters out vision-dominant neurons, picks the top-K neurons, and updates the down-projection columns with a null-space regularized objective plus a visual-perturbation KL term. Experiments on E-VQA, E-IC, and VLKEB with BLIP2, LLaVA-1.5, and MiniGPT-4 show performance comparable to training-based editors and strong OOD robustness.

Critical strengths

S1. Well-engineered design.
S2. Training-free: no auxiliary editor or per-dataset training, hence no OOD collapse (LiveEdit drops from 96 to 52 on BLIP2 EIC OOD, while DELTA is unaffected).
S3. Broad evaluation: 5 backbones, 3 datasets, 10+ baselines, single/sequential editing settings, and ablations on components, layers, parameters, and recombinations.
S4. Principled locality preservation: the null-space projection has a clear linearized-level guarantee (Appendix C) and is built from the edit input alone, so no separate preservation dataset is needed.
S5. Qualitative evidence that the selected neurons attend to the target region rather than the background (Fig. 7–11).

Critical weaknesses

W1. The method feels somewhat heuristic. The candidate layers are reported to be nearly fixed per backbone regardless of the sample, and the layer-selection criteria C1–C3 may be post-hoc. Why should κ_ℓ be in a “rising phase” rather than at its highest value? Why should entropy be high? If entropy were low, wouldn’t the activation be concentrated in specific neurons, making the relevant neurons easier to find? A stronger theoretical justification for these criteria would be welcome.
W2. τ_vis is set, but why is there no τ_text? Couldn’t a model that focuses too strongly on text tokens also be a problem?
W3. Hyperparameters. There are quite a few, which implies considerable tuning effort to reproduce the reported performance in practice. Some values lack justification or sweep results. How was the per-layer neuron count KℓK_\ell chosen? It varies substantially across backbones. How were the loss weights determined? Are the hyperparameters the same across all benchmarks?
W4. The caption for Fig. 7 (C) appears to be incorrect. Please check for errors throughout.
W5. What is the per-edit cost (time/memory)?

Questions for authors

Please see Weaknesses.

9 comments
AuthorsOpenReview ↗

Answer to W1

Dear Reviewer QRCS:

We are sincerely grateful to you for the careful and constructive review. It is truly encouraging that the review recognizes our work. We hope the following illustration could address your concerns!


W1. - The method feels somewhat heuristic: layer sets are nearly fixed per backbone regardless of the sample, and C1–C3 may be post-hoc. Why should κ_ℓ be in a "rising phase" rather than at its highest value, and why high entropy — wouldn't low entropy concentrate activations and make the relevant neurons easier to find? A stronger theoretical justification would be welcome.

We thank the reviewer for raising these important questions. We have substantially strengthened the justification of C1–C3 through complementary ablations, independent causal validation (Appendix K), and additional justifications (Appendix C and Appendix D). We would like to address this concern as follows:

C1–C3 are not post-hoc criteria. DELTA is not designed to maximize localization sparsity or optimize each component independently. Instead, it identifies structurally relevant multimodal processing layers and balances editing effectiveness, generality, and locality through complementary localization criteria and conservative neuron selection. All experiments are conducted from three complementary directions: (i) We ablate all 232^3 combinations of the criteria (Tab. 4) and the result shows that removing any criterion degrades performance, and single-layer editing nearly fails (Rel≈17%\text{Rel}\approx 17\%). (ii) Admitting layers excluded by individual criteria either introduces locality degradation or fails to yield consistent lifelong benefits (Tabs. 4 and Tab. 11). (iii) Restoring DELTA-located layers recovers more corruption-induced damage than size-matched random sets (In Appendix K). These results independently establish both the necessity and the causal relevance of C1-C3.

Why the rising phase rather than the maximum of κℓ\kappa_\ell? The absolute maximum can be dominated by visual-information injection rather than subsequent knowledge processing [1,2,3,4]. For example, on BLIP2, the pronounced Q-Former-induced spike occurs at layer 1,19,31. And empirically (results are shown in the next paragraph), editing only on the layer with the peak value of κℓ\kappa_\ell can significantly degrade M-Loc even to 6.73%6.73\%, and the average on layer 19/31 (within R(κℓ)\mathcal{R}(\kappa_\ell)) underperforms our full citera. In contrast, therefore, the rising-phase criterion identifies the subsequent decoder region, consistent with the independently observed mid-to-late causal-recovery hump. Thus, the dynamics of κℓ\kappa_\ell are more informative than its absolute magnitude for separating injection from subsequent processing.

Why high rather than low entropy? We agree that concentrated activations can, in principle, facilitate localization, but concentration does not imply that the underlying knowledge is concentrated or safely editable. We therefore select those layers with relatively high entropy values shown in Fig. 2 (BLIP2 - 0, 19, 21, 29; LLaVA - 1, 3, 9, 25) to provide empirical evidence (in the following reply, due to character limits).

These experimental results show that restricting edits to single high-entropy layers ranther, while achieving higher locality (because the knowledge is more concentrated), comes at the cost of a significant decline in the remaining generality. Therefore, high entropy serves as a criterion for avoiding overly concentrated, potentially disruptive updates while retaining sufficient degrees of freedom for joint editing; it does not assume that diffuse activations directly identify knowledge-bearing neurons. We add these experiments to the appendix (Tab. 12). Similarly, Causal relevance is not equivalent to editing suitability. Our causal-tracing experiments further clarify this distinction. Although DELTA's candidate layers partially overlap with the mid-to-late causal-recovery regions, the overlap is not exact. Cross-localization/editing ablations (Tab. 3) show that editing causal-tracing-located layers consistently underperforms editing DELTA-located layers under the same editing strategy.

Theoretical perspective. We would like to generally illustrate our framework from three aspects: (i) our decoupled vision-text profiling proposes candidate layers using one-pass, architecture-specific criteria; (ii) editor-free causal mediation independently validates their relevance; and (iii) editing ablations assess their suitability for knowledge modification. The candidate layer sets remain relatively stable across samples per backbone, while neuron-level localization remains instance-specific. We therefore do not claim that C1–C3 are universally necessary conditions for knowledge storage; rather, they constitute a structurally motivated and independently validated localization criterion for effective and locality-preserving VLLM editing.

AuthorsOpenReview ↗

Empirical results for higher/lower entropy demonstration

Layer ℓ\ellRelM-GenT-GenM-LocT-LocAvg
BLIP2-OPT
084.1371.8374.306.7343.2556.05
180.2157.6757.8162.4797.1271.06
1998.3896.5685.5161.3498.3988.04
2198.6797.7485.5669.6398.1089.94
2991.4190.4177.1780.2499.2587.70
3189.5988.6877.6578.4299.3586.74
LLaVA-1.5
161.4540.4747.0482.4599.6566.21
339.9830.1636.3396.3799.7360.51
970.6445.7753.4085.1399.6770.92
2552.9239.4946.8599.8799.5067.73
AuthorsOpenReview ↗

Answer to W2

W2. - τ_vis is set, but why is there no τ_text? Couldn't a model that focuses too strongly on text tokens also be a problem?

Thanks! This is an interesting question worth detailed discussion. Text dominance can indeed affect multimodal knowledge editing, but an additional τtext\tau_{\text{text}} is unnecessary for the following reasons and empirical evidence.

  1. The single ratio already provides a complementary threshold. Referring to Eqn. 6, we have ρjtext=1−ρjvis\rho_j^{\text{text}}=1-\rho_j^{\text{vis}}. Thus, introducing τvis\tau_{\text{vis}} implicitly defines the complementary text-side boundary. An independent τtext\tau_{\text{text}} is redundant, which may further remove neurons in the located region balanced by τvis\tau_{\text{vis}}, potentially discarding the cross-modal fusion neurons that C1–C3 aim to preserve.
  2. Text over-focusing is addressed at the layer-selection stage. C1–C3 prioritize layers with strong cross-modal fusion engagement, rather than modality-specific activation dominance. The two stages therefore jointly mitigate modality imbalance without introducing another threshold.
  3. We also conduct experiments with additional τtext\tau_{\text{text}} for BLIP2 on EVQA dataset, set # Edits=1/100:
τtext\tau_{\text{text}}neurons keptRelM-GenT-GenM-LocT-LocAvg
# Edits = 1
1.00 (no filter), 0.9, 0.810500/1050096.2894.9185.4883.9699.5892.04
0.7010428/1050095.5994.5176.6381.9299.3589.60
0.659628/1050095.5994.1876.3082.8099.3689.65
0.607917/1050095.5993.7376.6985.5899.6090.24
0.586916/1050095.5993.4874.5185.2399.5689.67
0.555024/1050093.4591.8670.1086.4599.6188.29
0.522278/1050070.7268.8247.9491.9999.6475.82
0.511151/1050043.0242.3528.0596.2199.8161.89
# Edits = 100
1.00 (no filter), 0.9, 0.810500/1050059.4848.3948.8825.1287.1153.80
0.7010381/1050054.3744.8047.1020.4985.5850.47
0.659957/1050054.7846.1645.5726.5786.1951.85
0.608623/1050057.8545.6745.6622.4686.6051.65
0.588281/1050056.4844.9446.9222.8187.2051.67
0.556201/1050060.0748.3149.6420.6388.9853.53
0.522994/1050046.3937.9539.0320.7586.5046.12
0.511450/1050045.1236.1030.4027.1092.9446.33

The results demonstrate that filtering in the textual modality breaks the balance. We added this illustration to Appendix I (Tab. 8). Thanks.

Reviewer_omH8OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes DELTA, a neuron-level locate-then-edit framework for multimodal knowledge editing. The method selects candidate FFN layers using cross-modal activation correlation, engagement, and entropy; identifies neurons through modality-specific activation scores and visual-dominance filtering; and updates their output weights with null-space regularization and visual perturbation. Experiments cover three benchmarks and five model backbones, including single-edit, sequential-edit, and ablation settings.

Critical strengths
  1. A practically relevant approach to multimodal editing without auxiliary editor training. DELTA combines modality-aware layer and neuron selection with direct parameter updates. This addresses the challenge of identifying editable parameters when information is distributed across visual and textual representations. Avoiding auxiliary editor training makes the approach potentially useful across different editing distributions.

  2. Promising empirical results across multiple architectures and benchmarks. The evaluation covers three benchmarks and five backbones, with component ablations and sequential-edit experiments. DELTA achieves competitive performance, including an average score of 96.54 on MiniGPT-4/VLKEB compared with LiveEdit’s 89.41. These results provide meaningful evidence of practical utility and motivate further investigation of the proposed localization strategy.

Critical weaknesses
  1. The novelty claim is overstated relative to closely related work. Earlier work (Pan et al., 2024a; Zhou et al., 2025; Fang et al., 2024) has already explored locating and editing neurons in multimodal models. The claim of being the first such framework is therefore too broad. The paper should explain more clearly how its method differs from these approaches and what new benefits it provides.

  2. The theoretical justification for the null-space constraint is incomplete. The constraint is built from the input being edited, but a direction that is inactive for this input may still be important for other inputs. Preserving activations on the edit input therefore does not necessarily protect unrelated knowledge. The paper should explain why this constraint can preserve other knowledge while still allowing the target answer to change.

Questions for authors
  1. Locality guarantee: Appendix C constructs the activation matrix AA from the edit input. What assumptions connect this matrix to the activation subspaces of unrelated inputs, allowing the bound on AΔ⊤A\Delta^\top to imply preservation of unrelated knowledge? Can the authors provide theoretical justification or empirical evidence for this connection?

  2. Compatibility between editing and null-space preservation: Which token positions are included when constructing AA? How does the method preserve activations on the edit input while changing its predicted answer? Please report reliability and locality immediately before and after the final hard projection, together with the retained singular values and update norms, to clarify this trade-off.

  3. Consistency of the layer-selection ablation: In Table 5, the full condition includes layer 28, whereas configuration (b), which removes C2, excludes it. Under Equation (7), removing a conjunctive criterion should produce a superset when profiling statistics are fixed. Is this a reporting error, or were different profiling statistics or settings used?

7 comments
AuthorsOpenReview ↗

Answer to W1 and W2

Dear Reviewer omH8:

Thank you very much for your constructive advice on our work. We sincerely appreciate the time and effort you have dedicated to reviewing our paper! Below, we respond to each of your comments (W1–W2, Q1–Q3) and outline the modifications based on your suggestions. We sincerely hope that our responses could address your concerns!


W1. - "The novelty claims need further improvements. Some prior work like M2Edit and NAM have done some overlapped jobs and DELTA may contribute a distinct localization rule and preservation strategy."

Thanks for this valuable review, and we appreciate the opportunity to state this precisely. We update our claims to a narrow one: a unified multimodal locate-then-edit framework that aligns knowledge localization with distinct rules for editing suitability while preserving unrelated knowledge. More specifically, prior works investigate multimodal neuron attribution or knowledge localization from different perspectives. NAM[1] studies whether MLLM outputs can be attributed to FFN neurons, while FiNE[3] further propagates editing losses to the neurons identified by NAM. MMN[2] focuses on multimodal interpretability for image captioning, and M2Edit[4] primarily considers language-information propagation and component/layer level localization. In contrast, DELTA is specifically designed for multimodal knowledge editing, with vision-text decoupled to first identify candidate layers and then localize FFN neurons associated with different modality-related components, providing a more fine-grained and editing-oriented localization criterion.

Besides, as shown in our causal-tracing analysis and localization-editor recombination experiments (Appendix K and Tab. 3), causal relevance alone does not necessarily imply editing suitability. DELTA's localization provides a better alignment with the subsequent editing objective. DELTA also incorporates a null-space-based constraint into the editing process.

The general response and the reply to reviewer HVFd also provide further comparisons and experimental evidence. We have also revised the novelty discussion in the manuscript to clarify these distinctions.


W2. - "The theoretical justification for the null-space constraint is incomplete. The constraint is built from the input being edited, but a direction that is inactive for this input may still be important for other inputs. Preserving activations on the edit input therefore does not necessarily protect unrelated knowledge. The paper should explain why this constraint can preserve other knowledge while still allowing the target answer to change."

We thank the reviewer for identifying an important gap in the theoretical justification of our null-space constraint. We agree that constructing the constraint solely from the edit input does not automatically guarantee the strict preservation of unrelated knowledge. However, directions inactive for the edit input may still be important for other inputs. To address these concerns, we have substantially strengthened the theoretical analysis in Appendix. D from two complementary perspectives and adjust the claims in the relevant sections.

(1) Cross-input locality guarantee. We introduce an explicit cross-input locality-separation assumption that characterizes the relationship between the edit-specific null space and the activation subspaces of unrelated inputs. Under this condition, we derive a unified perturbation bound that extends the edit-specific preservation analysis to unrelated inputs, and further establish the corresponding post-edit locality guarantee. We also provide empirical evidence to assess this assumption and its applicability. Please refer to our response to Q1 for the detailed assumption, theoretical derivation, and empirical validation.

(2) Compatibility between editing and null-space preservation. We clarify the token positions used to construct the activation matrix and explain why the null-space constraint does not eliminate all directions available for modifying the target response. In particular, the constraint preserves selected low-variance activation directions while allowing non-negligible activation components at generation positions to support target editing. We further quantify the effect of the final hard projection by comparing editing reliability, locality, and update statistics before and after projection. Please refer to our response to Q2 for the detailed explanation and quantitative results.

AuthorsOpenReview ↗

Answer to Q1

We would like to address Q1 in the following three perspective:

1. Cross-input activation-space assumption.

Let Ae(ℓ)\mathbf A_e^{(\ell)} denote the trace-activation matrix constructed from the edit input e\boldsymbol{e}, and let Pnull(ℓ)\mathbf P_{\mathrm{null}}^{(\ell)} project onto its selected null-space directions. The key connection to unrelated inputs is formalized by the locality-separation condition (Assumption 1.), which requires that the trace-neuron activations of an unrelated input have limited components along the edit-specific null space (unrelated input denoted as u\boldsymbol{u}): $\mathbf{P}_{\text{null}}^{(\ell)}au,i(ℓ)≤\boldsymbol{a}^{(\ell)} _{u,i}\leq\varepsilon _{\text{loc}}$. Geometrically, this means that unrelated inputs' trace-neuron activations approximately lie in the row space spanned by the edit input, rather than requiring their activation subspaces to be identical. This condition is motivated by DELTA's modality-partitioned neuron localization: the selected trace neurons form a compact, knowledge-concerned subpopulation, within which factual queries exhibit an empirically concentrated activation structure. Importantly, we treat this locality-separation condition (Assumption 1.) as an explicit, falsifiable cross-input condition, rather than an unconditional consequence of the null-space construction.

2. Unified perturbation bound.

Speciffically, for any input xx with trace-activation matrix Ax(ℓ)\mathbf A_x^{(\ell)}, its layer-wise FFN perturbation is bounded by $|\delta\text{FFN}^{(\ell)}_x| _F≤∣Ax(ℓ)Pnull(ℓ)∣2\leq|\mathbf A _x^{(\ell)}\mathbf P _{\mathrm{null}}^{(\ell)}|_2|\boldsymbol\Delta^{(\ell)}|_F.$

For the edit input, the response factor is controlled by the truncated singular value of its own activation matrix, yielding the bound in Eqn. 39. For an unrelated input, however, this factor cannot be bounded using the edit input's singular values. Instead, we introduce the locality-separation condition (Assumption 1.), which explicitly requires unrelated trace activations to have limited components along the edit-specific null space. This gives ∣Au(ℓ)Pnull(ℓ)∣2≤Suεloc|\mathbf A_u^{(\ell)}\mathbf P_{\mathrm{null}}^{(\ell)}|_2\leq\sqrt{S_u}\varepsilon _{\mathrm{loc}}, and hence ∣δFFNu(ℓ)∣F|\delta\text{FFN}^{(\ell)} _u| _F ≤Su\leq\sqrt{S _u} εloc\varepsilon _{\mathrm{loc}} ∣Δ(ℓ)∣F|\boldsymbol\Delta^{(\ell)}| _F.

Notably, locality does not require unrelated inputs to exhibit weak activations on the selected neurons. It only requires their activation components along the null-space directions in which the update operates to be small. Theorem. 1 further propagates this bound through the post-edit residual stream, while Proposition 1 extends the analysis to lifelong editing.

3. Empirical support for locality separation.

We additionally evaluate the locality-separation condition (Assumption 1) on 50 edits and 50 unrelated inputs from EVQA using BLIP2; these measurements are added in Tab. 6 of Appendix D. In the hard-projection arms over the dnull∈{5,10,15,20,25}d_{\mathrm{null}}\in\lbrace 5,10,15,20,25\rbrace grid, the scale-invariant alignment ∣∣A2Δ(ℓ)⊤∣∣F/(∥A2∣∣F∥Δ(ℓ)⊤∣∣F)||\mathbf{A}_2{\boldsymbol{\Delta}^{(\ell)}}^{\top}||_F/(\|\mathbf{A}_2||_F\|{\boldsymbol{\Delta}^{(\ell)}}^{\top}||_F) between the post-edit update and the unrelated-input activations falls monotonically from 0.300.30 to 0.130.13 as the cone tightens, in lockstep with the unrelated-input next-token KL (0.0414→0.00020.0414\to0.0002). This is a causal confirmation of the separation: if unrelated inputs carried substantial mass on the retained null directions, confining Δ(ℓ)\boldsymbol{\Delta}^{(\ell)} there could not reduce their KL. We also find that 84.9% of unrelated positions exhibit zero activation on the selected trace neurons. These results suggest that the edit-specific null space captures directions with relatively limited unrelated-input activation, supporting the proposed activation-space separation. We emphasize that this separation is partial rather than universal, and its extent is explicitly characterized by εloc\varepsilon_{\mathrm{loc}}. For the detailed experimental settings and results, please refer to the answer to Q2.

AuthorsOpenReview ↗

Answer to Q2

We address this question from the following parts:

1. Token-position selection for null-space construction. For an edit input triplet (Ie,xe,ye)(\boldsymbol{I}_e, \boldsymbol{x}_e, \boldsymbol{y}_e), we construct the layer-wise activation matrix using the token positions in the range of Pe≜[1,Se]\mathcal{P}_e\triangleq [1, S_e]. Since the prompt consists of visual tokens sized NvN_v and textual tokens sized NtN_t, we additionally denote the target (ye\boldsymbol{y}_e) token length as NyN_y (i.e., TT in line 253: "target output ye\boldsymbol{y}_e of length TT"), from a single teacher-forced forward pass, and therefore Se=Nv+Nt+NyS_e=N_v+N_t+N_y (plus an optional BOS token at position 0). These tokens are included to collect the activations to build a(ℓ)\boldsymbol{a}^{(\ell)} in Eqn. 10 (Sec 4.3), while A(ℓ)=a:,Tℓ(ℓ)\mathbf{A}^{(\ell)}=\boldsymbol{a}^{(\ell)}_{:,\mathcal{T}_\ell} is the sub-column matrix of a(ℓ)\boldsymbol{a}^{(\ell)} indexed by Tℓ\mathcal{T}_{\ell}. Note that SS in the equation a:,Tℓ(ℓ)∈RS×∣Tℓ∣\mathbf{a}^{(\ell)}_{:, {\mathcal{T}}_\ell}\in\mathbb{R}^{S\times|{\mathcal{T}}_\ell|} (line 245) is the simplified annotation, actually SeS_e here.

2. Why target editing remains possible.
Although the teacher-forced generation positions are rows of Ae(ℓ)\mathbf A_e^{(\ell)} (i.e., A(ℓ)\mathbf{A}^{(\ell)} of the edit input), the constraint spans only the dnull≪Kℓd_{\text{null}}\ll K_\ell directions of smallest covariance --- directions that the edit input as a whole barely excites, its covariance spectrum being dominated by the high-variance prompt/vision positions. Activations at the generation positions are therefore not required to lie in the row space that Lns\mathcal{L}_{\text{ns}} protects, and positions realized during free (autoregressive) decoding at test time never enter the edit-time covariance at all. We formalize the availability of non-negligible null-space activation at these positions through the editability condition (Assumption 2.). Empirically, Tab. 6 of Appendix D quantifies this channel over the dnull∈{5,10,15,20,25}d_{\mathrm{null}}\in\{5,10,15,20,25\} grid: the cone is spanned by the weakest directions of the selection spectrum (median σmax⁡null\sigma^{\mathrm{null}}_{\max} down to 0.0100.010, with the weakest excluded direction always 1.31.3--1.7×1.7\times stronger), and the in-loop Lns\mathcal{L}_{\text{ns}} alone never closes the channel --- reliability stays ≥99.5\geq99.5 at every dnulld_{\mathrm{null}}. The final hard projection, by contrast, deletes precisely the generation-position mass cec_e: at dnull=10d_{\mathrm{null}}{=}10 it removes only 19%19\% of the update norm (2.313→1.8702.313\to1.870), yet reliability drops from 100.00100.00 to 34.8534.85. This supports the existence of an editing channel that allows the target response to change while suppressing perturbations along directions important to unrelated inputs.

3 Effect of the final hard projection. We evaluate the model immediately before applying the final hard projection and after applying it. The following experimental results are added in Tab. 6 of Appendix D.

dnulld_{\text{null}}Projected update: Kℓ>dnullK_\ell{>}d_{\text{null}} (%)Projected update: σmax⁡null\sigma^{\mathrm{null}}_{\max}Projected update: σmin⁡exc\sigma^{\mathrm{exc}}_{\min}Projected update: ∥Δ(ℓ)∥F\lVert\boldsymbol{\Delta}^{(\ell)}\rVert_FInterference: KLmean\mathrm{KL}_{\text{mean}}Interference: KLp95\mathrm{KL}_{\text{p95}}Algn.RelT-GenM-GenT-LocM-LocAvg
LnsL_{\mathrm{ns}} with hard-proj.
252535.11.3051.6823.379→3.2863.379\to3.2860.04140.09920.3097.7381.0195.6799.1370.0388.71
202055.10.5850.7413.069→2.9013.069\to2.9010.03600.08560.2687.3074.2182.6399.5077.5384.23
151574.70.2750.3652.758→2.4942.758\to2.4940.02550.05930.2267.7961.7161.5199.4784.4674.99
101093.10.0800.1262.313→1.8702.313\to1.8700.00620.01520.1734.8539.4031.9999.4291.8359.50
55100.00.0100.0171.936→1.2331.936\to1.2330.00020.00050.1319.4629.5217.6499.6898.0752.87
LnsL_{\mathrm{ns}} w/o hard-proj.
252535.11.3051.6823.379→3.3793.379\to3.3790.04160.09990.32100.0090.5498.8298.9563.4090.34
202055.10.5850.7413.069→3.0693.069\to3.0690.03770.09060.30100.0087.0498.4799.3568.5690.68
151574.70.2750.3652.758→2.7582.758\to2.7580.03230.07750.27100.0082.1997.9799.3571.8690.27
101093.10.0800.1262.313→2.3132.313\to2.3130.01950.05210.26100.0080.3496.1799.3675.8890.35
55100.00.0100.0171.936→1.9361.936\to1.9360.01550.04250.2899.5380.8796.1799.6175.9590.43

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

Dear Reviewers:

We want to thank all reviewers for carefully reading the paper and for their constructive suggestions. All of these helped us improve our work. In the following, we summarize the changes made in the revision and mark them in blue{\color{blue}\text{blue}}. We also responded to each concern in your reviews.

Key differences from prior works.

@ omH8 - W1, @ HVFd - W2

We would like to further clarify differences between DELTA and some of the prior works focusing on multimodal locate-then-edit:

Fang et al. propose NAM[1] to discuss whether MLLM outputs can be attributed to FFN neurons for image-generating MLLMs, exploring the semantic knowledge editing of the image synthesized by generative multimodal models as a validation of the neuron attribution analysis. Currently, (multimodal) knowledge editing mainly focuses on structural knowledge injection into (V)LLMs. FiNE[3] achieves promising results by directly propagating editing losses to the neurons identified by NAM[1]. However, its effectiveness is significantly reduced for modality-concerned knowledge for VLLMs (Tab. 1). Pan et al. propose MMN[2] for the interpretability of image captioning. However, as they claim, MMN can only perform single-token editing to modify limited output, and its true capability remains unknown on the latest multimodal knowledge editing (MKE) benchmarks. DELTA provides a more transient localization strategies with no restricts to the targets. Zhou et al. propose M2Edit[4]; however, only the propagation of language information within the model is taken into consideration. Its focus remains on the combination of key layers at the component and layer levels, with no precise analysis within submodules (e.g., attention, FFNs, or neurons), and it performs poorly on the latest MKE benchmarks.

Key contributions of DELTA.

@ omH8 - W1, @ HVFd - W2

As locate-then-edit has emerged as a paradigm in LLM knowledge editing (KE) due to its compatibility and interpretability, directly extending this paradigm to multimodal knowledge editing (MKE) remains challenging: the involvement of visual modality and cross-modal knowledge distribution makes existing localization approaches (e.g., causal tracing [5]) difficult to apply directly to VLLMs [6, 7]. DELTA is proposed to bridge this gap as a unified multimodal locate-then-edit framework that aligns knowledge localization with editing suitability while preserving unrelated knowledge. Specifically, DELTA first decouples internal vision-text activations within VLLMs, then identifies candidate knowledge layers using neuron-level activation signals, entropy, and cross-modal correlation. It further selects the top-activated neurons within these layers as editing targets, filtering out vision-dominant neurons via a threshold. To improve generality and locality preservation, DELTA incorporates a visual perturbation mechanism and a null-space constraint, respectively. Experimental comparison can be found in Tab. 1.

Importantly, our experiments distinguish factual relevance from editing suitability: identifying components associated with a fact does not necessarily identify the most suitable locations for editing it. Our causal-tracing analysis (Appendix K, Figs. 15-16) validates that DELTA's localization captures fact-related information. Meanwhile, the localization-editor recombination experiment (Tab. 3) shows that, under the same editor, using causal-tracing-localized layers yields worse performance than using DELTA-localized layers. All these results lead to the conclusion that causal relevance and editing suitability are distinct issues, and DELTA is proposed primarily to address the latter.

Summary of revisions

We add the theorem for locality guarantee and null-space constraints. (@ omH8: W2, Q1, Q2; @ HVFd: Q1)

We refine the illustration of motivation, novelty, and claims. (@ omH8: W1; @ HVFd: W.2, W3)

We provide causal tracing validation and more discussion on localization. (@ QRCS: W1; @ HVFd: W1)

We add more results and comparisons. (@ QRCS: W2, W3, W5; @ HVFd: Q3)

We fix typos and unclear figures. (@ omH6: Q3; @ QRCS: W4; @ HVFd: Q2, Q3)

Once again, we sincerely appreciate your time and effort. All of your insights have been indispensable in refining the overall quality of our paper. We are more than happy to offer more clarification to address any additional reviews and recommendation from you!

Sincerely,

Authors of Submission 11844.

AuthorsOpenReview ↗

References

[1] Fang, J., Bi, Z., Wang, R., Jiang, H., Gao, Y., Wang, K., Zhang, A., Shi, J., Wang, X. and Chua, T.S., 2024. Towards neuron attributions in multi-modal large language models. Advances in Neural Information Processing Systems, 37, pp.122867-122890.

[2] Pan, H., Cao, Y., Wang, X., Yang, X. and Wang, M., 2024, August. Finding and editing multi-modal neurons in pre-trained transformers. In Findings of the Association for Computational Linguistics: ACL 2024 (pp. 1012-1037).

[3] Pan, H., Wang, X., Cao, Y., Shi, Z., Yang, X., Li, J. and Wang, M., 2025, May. Precise localization of memories: A fine-grained neuron-level knowledge editing technique for llms. In International Conference on Learning Representations (Vol. 2025, pp. 482-502).

[4] Zhou, Y., Cao, P., Chen, Y., Liu, Q., Sui, D., Chen, X., Liu, K. and Zhao, J., 2025, November. M2Edit: Locate and Edit Multi-Granularity Knowledge in Multimodal Large Language Model. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 29017-29030).

[5] Meng, K., Bau, D., Andonian, A. and Belinkov, Y., 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35, pp.17359-17372.

[6] Li, Q., Ye, Z., Feng, X., Zhong, W., Ma, W. and Feng, X., 2026, March. Causal tracing of object representations in large vision language models: Mechanistic interpretability and hallucination mitigation. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 37, pp. 31645-31653).

[7] Palit, V., Pandey, R., Arora, A. and Liang, P.P., 2023, October. Towards vision-language mechanistic interpretability: A causal tracing tool for blip. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (pp. 2848-2853). IEEE.

[8] Chen, Q., Zhang, T., Wang, C., He, X., Wang, D. and Liu, T., 2025, April. Attribution analysis meets model editing: Advancing knowledge correction in vision language models with visedit. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 2, pp. 2168-2176).

[9] Chen, Q., Wang, C., Wang, D., Zhang, T., Li, W. and He, X., 2025, June. Lifelong knowledge editing for vision language models with low-rank mixture-of-experts. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 9455-9466). IEEE.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.