acceptodds
Desk rejected at ICLR 2027

A Window Is Not a Door: Mapping the Limits and Conditions of Knowledge Editing in Language Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

41 views

Abstract

Interpretability makes factual knowledge in language models observable, but observability does not imply editability. We identify a gap between locating factual signals and reliably changing knowledge, defined by control, transfer, and collateral damage. In the tested settings, localization-derived interventions move target answers without achieving selective control within the specified collateral budgets. Post-edit target hits likewise conceal already-correct answers, limited transfer, and damage to other knowledge. Reading out information and eliciting a target answer are therefore insufficient evidence of reliable knowledge editing. This gap is not uniform: the model’s pre-edit support for the target answer helps organize where correction succeeds. Targets near the front of the base distribution, but not initially generated correctly, form a region of correction opportunities. Candidate rank predicts correction yield in independent fixed-budget tests, while far-rank successes reveal a model-dependent boundary rather than a hard cutoff. Context and weight interventions further expose distinct routes to changing answers, with different benefits and costs. Together, these findings shift the central question of knowledge editing from where knowledge resides to which facts can be changed, through which intervention, and at what cost. Interpretability provides a window into knowledge; a door to reliable editing depends on existing candidate support and the conditions of intervention.

Primary area:
interpretability and explainable AI
Keywords:
knowledge editing, interpretability, language models, factual knowledge, generalization, representation interventions

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RWTfOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This empirical paper distinguishes target-answer emission from newly correcting an initially wrong answer, transferring the change, and preserving other knowledge. It studies localization-derived controllers, audits existing edited outputs, and examines pre-edit target-token rank as a predictor of correction. The direct-controller experiments find no budget-admissible positive selective control in the evaluated valid subset; other learned interventions do produce admissible gains. A separate fixed-adapter selection experiment finds that rank-based selection yields more corrections than random or answer-frequency-matched selection. Context, additional training exposure, and an exploratory masked MoE update illustrate further intervention-specific conditions.

Critical strengths

A useful accounting framework for actual correction

Section 2, PDF page 3, lines 116–160, explicitly decomposes target hits into retained correctness and new corrections, subtracts regressions for net gain, distinguishes wording/entity/relation transfer, and defines collateral budgets. The paired rescoring in Figure 3 uses the same 100 outputs per model and reports 60–70 target hits but only 7–22 new corrections, with smaller counts after the available preservation checks (Section 4, PDF page 5, lines 229–250). These definitions and the paired audit make clear what an editing result actually establishes. Holding outputs and the scorer fixed makes the retention-versus-correction discrepancy particularly easy to interpret.

Controlled tests of whether read-derived structure helps an update

Section 3, PDF page 5, lines 219–224, constructs geometry using training data, chooses checkpoints using validation data, and evaluates 302 test entities with matched random projections and an unmasked arm. It includes five LoRA arms, three Qwen models, and three seeds, and reports that the prespecified advantages remain unconfirmed while acknowledging that the intervals permit differences. This moves beyond comparing read and write directions geometrically: it asks whether the proposed structure improves an intervention against relevant controls. The cautious handling of inconclusive comparisons is a methodological strength.

A separate prospective test of the rank-selection observation

Section 5.2 and Table 1, PDF page 7, lines 326–346, measure ranks before inspecting edited outputs, hold the shared adapters and 200-entity budget fixed, and compare rank selection with blind and answer-frequency-matched subsets. Rank selection increases the reported mean correction count in all three comparisons, for example from 50.06 to 63.67 relative to frequency matching for Work–country/Qwen14B. The text explicitly limits the interpretation of repeated subset sampling. This supplies a useful predictive check rather than relying solely on the descriptive rank bins in Figure 4. Frequency matching is also a meaningful control for one plausible source of the association.

Critical weaknesses

The main direct-controller result needs a more assessable protocol and calibration

positive selective control

Source: Section 3, Readable, but Not Reliably Writable; PDF p. 4; manuscript lines 173–179.

Concern. The main text specifies the collateral thresholds but does not give an operational definition of selective control, the fact population for the 42-cell comparison, or the rules for choosing each operator’s site, direction, and intervention strength. Figure 2 therefore establishes a result for selected configurations whose coverage and calibration are difficult to assess from the main body. The explicitly disclosed 0/21 unflagged result is the relevant localization evidence; the other 21 cells cannot expand that evidential coverage. The separate dose trajectories and the random-projection controls for learned LoRA updates are useful, but they do not by themselves calibrate all seven direct-controller families. This limits how much the central negative result tells us about the tested localization interfaces beyond those particular configurations.

Suggested clarification or experiment. Please add a compact main-text protocol specifying the selective-control equation, evaluation population, localization-validity criterion, and configuration-selection rule. For the valid direct-controller subset, show uncertainty over evaluated facts or independent runs and an equal-budget sweep of intervention strength, with matched random site/direction controls where applicable. Keep these controls distinct from the existing learned-update experiment. This would strengthen the inference about localization-derived control.

The reliable-correction criteria are not closed on a common evaluation population

The final rung omits WD and is a partial preservation check.

Source: Section 4, The Answer Changes. Does the Edit Hold?; PDF p. 5; manuscript lines 231–235.

Concern. The paper is transparent about this omission. However, the main evidence remains distributed across different checks and populations: Figure 3 combines new correction with KL/unrelated preservation but omits wrong-relation damage; the synthetic held-out-fact test does not measure persistence of the same correction under rewording; and Figure 4 reports new corrections with KL rather than full-budget success. The reported denominators and retained hits permit computation of 2/6/5 regressions and 20/1/3 net corrections for Qwen14B/32B/A3B, respectively. The unresolved issue is how these gains combine with paraphrase transfer and the complete collateral budget. Consequently, the main text does not quantify, on one common population, how many useful corrections remain once complete-answer transfer, regressions, and all stated collateral criteria are considered together.

Suggested clarification or experiment. Please complete one coherent evaluation on the existing paired-audit cohort, or on a clearly specified representative learned-update cohort: report retained correctness, new corrections, regressions and net gain; test held-out paraphrases of the same edited facts using complete-answer scoring; and measure KL, wrong-relation damage and unrelated damage under one stated aggregate budget. A joint table with sample sizes and uncertainty would be sufficient.

Rank’s incremental predictive value and generality remain insufficiently tested

three relation–model comparisons

Source: Section 5.2, Can We Tell Before We Try?; PDF p. 7; manuscript lines 329–346.

Concern. Table 1 is encouraging, but it covers two relations, two model families, and three fixed shared adapters. Answer-frequency matching addresses one alternative explanation, while first-token rank may also proxy sequence likelihood, tokenization, or familiarity already captured by other predictors. Figure 4 contains small, non-monotone bins, so it cannot resolve this issue by itself. The manuscript correctly states that first-token support is not complete factual knowledge and that 10,000 subset draws are not training replications. My concern is therefore the remaining validation gap: the central actionable predictor has not yet been shown to add information over stronger alternatives or to retain its selection advantage on new relations. Section 6 already identifies a related comparison as open, but it is important for judging the contribution beyond established familiarity-based prediction.

Suggested clarification or experiment. Please fix the selection policy before evaluating several held-out relations, compare rank with full-answer likelihood and a target-probability margin under the same eligibility, entity budget and cost gates, and report per-adapter paired gains with uncertainty across training seeds. A small set of fresh independently trained adapters would also test whether the advantage survives beyond the currently fixed output pools. These experiments would test incremental predictive usefulness.

Questions for authors
  1. How is selective control computed, and how were sites, directions and doses selected for the valid direct-controller configurations in Figure 2? How sensitive is the conclusion to equal-budget tuning of those choices?

  2. Can one main-text cohort report net gain, held-out paraphrase performance, and all three collateral criteria together, with regression counts explicitly tabulated and the currently unmeasured wrong-relation damage added to the paired audit?

  3. Does rank outperform full-answer likelihood or probability-margin predictors on held-out relations, and what are the paired gains and uncertainty across the three shared adapters or fresh training runs?

Reviewer_gpPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In this paper the authors examine the gap between finding factual information in a language model and changing its answers reliably. It distinguishes target hits from new corrections, separates several kinds of transfer, and evaluates interventions against collateral-damage budgets. The experiments cover direct representation interventions, learned updates, candidate-rank analysis, context, and local training. The main findings are that the selected direct interventions do not achieve selective control within the stated budgets, while pre-edit target rank predicts where fixed learned adapters produce more corrections.

Critical strengths

The paired audit and rank-selection experiment are useful.

Critical weaknesses

Section 3 names seven operator families, but I could not reconstruct how the interventions were built or selected. The paper needs more detail on the localization objectives, intervention sites, direction normalization, optimization, and the rule used to select each reported operator–model point. It also needs an explicit definition of selective control, which is the central endpoint in Figure 2. The operational definitions of wrong-relation damage and unrelated damage should state the prompts, scorers, and denominators.

Questions for authors
  1. Can you provide the complete controller protocols and source map, including a formal definition of selective control and an explanation of the measurement-NO-GO flags?
  2. What are the numerical results and uncertainty intervals for the matched geometry comparisons? What size of useful advantage remains compatible with those intervals?
  3. Is there a positive control or functioning established editor on the same task that helps distinguish intervention failure from a limitation of localization-derived control?
Reviewer_aFnqOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether localizing factual signals in language models supports reliable knowledge editing, defined through new correction, transfer, and collateral cost in Eq. (1) to Eq. (5). Seven direct operator families, from readout to DAS and CDAS, are tested on six models including Qwen3, Gemma-31B, and Llama-3.3-70B. A paired metric audit separates target hits from new corrections. Base target first-token rank is then related to correction yield and tested in a fixed-budget selection study, followed by context, local training, and MoE masking probes.

Critical strengths

S1: The paired audit is a useful evaluation lens. Holding outputs and the scorer fixed, Eq. (1) shows in Table A3 that 60/100 Qwen3-14B hits contain only 22/60 new corrections and 6/60 after the partial gate, a gap that target-hit metrics hide.

S2: The study is unusually transparent. Appendix D reports the n=12 dose 0.5 window vanishing at n=74, Appendix F probes threshold sensitivity, and the 21 measurement-NO-GO cells are kept apart from the 0/21 unflagged result.

S3: The rank signal is actionable. In Table 1, under three fixed shared adapters and a 200-entity budget, rank selection yields 63.67, 45.33, and 52.67 mean new corrections against 50.06, 38.58, and 30.42 for answer-frequency-matched subsets.

Critical weaknesses

W1: Table 1 reports the Rank column as a mean over three fixed adapters without any interval, while Figure 4 already uses Wilson 95% intervals. Readers cannot judge how stable the gains over matched subsets are. An entity-level bootstrap interval for every cell would make the selection result easier to trust.

W2: Internal codes such as E84, E1, D-3, D-5, and CPT are never defined, and model names alternate between Gemma-31B and Gemma4-31B and between Qwen3-A3B and Qwen3-30B-A3B. This slows reading across the separate cohorts and their gate definitions. A short glossary in Table A1 and consistent names would help.

W3: The Reproducibility Statement describes a source map and SHA-256 references, but no code, prompts, or fact lists are released. Since many results come from historical summaries rather than fresh runs, the audit is hard to reproduce independently. Releasing the implementation and populations at camera-ready would strengthen the work.

Questions for authors

Q1: Could the authors add official MEMIT or MEND runs, and explain why patching, path attribution, and QKV routing give target control 0.00 on all six models in Figure A1?

Q2: How does rank compare with base target probability or the Knowledge Spectrum predictor, given Figure 4 bins of 4 to 24 facts?

Q3: Appendix F mentions "all five thresholds" while Eq. (5) defines three, and Figure A3 still says "Four-condition effect-labelled figure remains pending". Please reconcile both.

Q4: Since Appendix G finds that the ROME-style baseline never inserts the target, the discussion would benefit from recent work on how locate-then-edit methods construct the editing target and on whether reported editing success reflects real changes, for example [1][2].

[1] From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing.

[2] Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.