acceptodds
Under review as a conference paper at ICLR 2027

Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing

Abstract

Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge while preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Knowledge Editing, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6KuCOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors identify context-induced difficulty underestimation as the phenomenon in which later facts in a passage receive lower loss due to the preceding context. The authors identify this issue as one of the core components leading to the failure mode of context reliance. To counter this, the authors propose FOVEATED, a method that constructs focused views of individual sentences through randomly shifting position embeddings.

Critical strengths
  1. The paper is overall well written with a clear structure that makes it easy to follow. The authors clearly identify a problem, define it, and then use it to motivate their method.

  2. The proposed method is clearly motivated and backed by a theoretical analysis. Overall, the method is a simple and elegant fix to the identified problem.

  3. The empirical analysis is thorough, covering two model families, three benchmarks, and five editing methods.

Critical weaknesses
  1. While the problem is presented clearly, the intuitive explanation is not completely substantiated by the empirical assessment. While Figure 1 shows a consistent trend of lower losses for later position IDs, it is not clear whether the lower losses are driven by the previous context. Intuitively speaking: If a passage consists of three completely independent facts, the prior context should not "help" the model and thus not lead to a lower loss at later positions. As this is a crucial building block for the entire paper, this issue should be assessed directly.

  2. The concerns of point 1 get even stronger when looking at the results for Figure 2. Here, FOVEATED leads to fairly consistent improvements across all questions. This seems counterintuitive given the underlying motivation, as this would suggest that FOVEATED mainly provides improvements for later questions (i.e., questions about facts present in the later parts of the paragraph). Additionally, it is unclear to me how FOVEATED can provide substantial improvements to the first question. Given that there should be only limited context prior to the related fact, FOVEATED should intuitively have much less impact here.

  3. While I appreciate the analysis of general capability preservation, the evaluated benchmarks do not provide a holistic assessment of the "general capabilities" of a modern LLM. The authors should either conduct a more holistic assessment (e.g., using an evaluation suite like CapTrack [1]) or re-scope this section.

  4. The case studies would be much stronger if the authors provided the corresponding passage as well. This would allow better insights into how the proposed method qualitatively improves the model's responses.

  5. In Figure 2, it is very difficult to accurately assess the actual difference between the solid and transparent bars. The authors should explore alternative visualizations (e.g., a bar for the base performance and a marker for the proposed method) to make the figure easier to read.

[1] Thede et al., 2026, CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training

Questions for authors
  1. Do the authors see the issue of context-induced difficulty underestimation specific to unstructured knowledge editing, or is this a general issue in language model pretraining (i.e., that factual content occurring later in a given block is trained on less)?
6 comments

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

Role of the preceding context in the lower loss at later positions.

Thanks for your question. We would like to clarify that in UKE benchmarks (including the ones we use), the free-form passages often describe a subject and multiple aspects of this subject, each of which can be seen as an atomic fact [1, 2, 3]. These facts are not independent facts from different domains. (Following the reviewer's later suggestions, we included some passage examples in the revision.) Therefore, in such passages, the preceding content is related to the later facts, and context-induced difficulty underestimation thereby arises.

In App D.1 we showed that the later position tokens, regardless of how different sentences are shuffled, tend to have lower losses. Following the reviewer's thoughts, we further compared the loss of the same sentence when it is placed at different positions. As shown below, the same sentence receives lower loss when it is placed later, i.e., with more preceding context. This indicates that in general, a sentence can benefit from more of its preceding context (from the given passage, not an arbitrary one) and have a lower initial loss if it is placed in a later position of the passage. We have added these results to the revised paper.

DatasetPortion of sent. with lower lossDrop in pre-edit loss
UnFine-UnKE94%1.68
UnFine-CF95%1.62
UnFine-MQ94%1.55

Tab. Statistics of sentences when placed at later positions (more details are in App. D.1 of the revised paper)

[1] Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models, 2025

[2] Uncovering Context Reliance in Unstructured Knowledge Editing, 2026

[3] FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing, 2026

Gains of FOVEATED across question positions, including the first question.

We thank the reviewer for the careful reading. We would like to clarify that the first fact is also learned with context given by the editing prompt that introduces the subject. Additionally, this editing prompt is often different from the first atomic question. As a result, when FOVEATED perturbs the view of the prompt in the same way as the preceding sentences, the first fact benefits from the context-shortcut-breaking mechanism as well.

In practice, we view the position of a fact as a practical proxy for how much it may suffer from context reliance. To better quantify how atomic facts may benefit from FOVEATED, we compared facts at the same position with low and high contextual support, and found that the latter usually benefit more from FOVEATED (e.g., +10.0 vs. +5.7 on UnFine-CF with Qwen2.5). We have added these results to the revised paper.

More holistic assessment of general capabilities.

We thank the reviewer for pointing us to CapTrack [1]. In this work we follow the convention in UKE literature [2] to evaluate the edited models' general capabilities, and found that the shortcut-breaking objective induced by FOVEATED brings, on average, modest influences on top of the editors' impacts. Following the reviewer's suggestions, we conducted additional experiments (under the same setting as our ablation studies) on a subset of CapTrack tasks spanning the three categories (CAN, WILL, and HOW). Results are shown in the table below. We found that FOVEATED brings both modest improvements and degradations on these tasks, similar to our observation on the six benchmarks. We have revised our paper and included these results.

MethodPopQA (CAN)GSM8K (CAN)BoolQ (CAN)RULER (WILL)IFEval (HOW)BFCL selection (HOW)BFCL arguments (HOW)
FT-M13.426.682.698.862.477.123.4
+Ours12.428.483.299.662.878.124.1
μKE12.726.579.898.258.666.518.2
+Ours13.125.578.897.257.472.022.0

[1] CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training, 2026

[2] FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing, 2026

Corresponding passages in the case studies.

As suggested, we added the editing request and the complete passage for the case studies in the main text in the revised paper.

Improving readability of Fig. 2.

We thank the reviewer for their suggestion. We have updated the figure (now Fig. 3) accordingly.

Context-induced difficulty underestimation beyond unstructured knowledge editing.

Thank you for bringing up this interesting question. Conceptually, we agree with the reviewer and hypothesize that this issue is not specific to UKE: it may arise in more general language modeling whenever the content within a text is related, so that the preceding content makes the later content easier to predict. UKE offers a clear setting to study the issue: the edited knowledge is usually counterfactual and unknown to the model by construction. This helps rule out the possibility that the model already knows part of the new knowledge. This possibility is harder to exclude in more general settings, which makes the study more challenging but also worth pursuing. We have revised our draft to include this discussion.

Thank you for the comprehensive and clear rebuttal. The clarifications and new analyses address most of my concerns. I have one follow-up about the role of semantic relatedness in the proposed mechanism.

I understand that passages in UKE benchmarks often describe multiple aspects of the same subject. Even within such passages, however, some facts may provide little information about one another. The sentence-shuffling analysis shows that the same sentence tends to receive lower pre-edit loss when placed later in its original passage. How much of that reduction depends on the preceding sentences being semantically related to the target fact?

One way to test this would be to replace the original preceding sentences with a length-matched prefix from an unrelated passage, while holding the target sentence, editing prompt, and target token positions fixed. Comparing pre-edit loss under the two prefixes would help isolate the contribution of related context. It would also be informative to compare FOVEATED’s gain under both conditions. Does the gain diminish when the unrelated prefix provides less support for predicting the target fact, or does FOVEATED yield a similar improvement?

Reviewer_4T3cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how contextual support during passage-level editing can make individual facts appear easier to learn than they are to recall independently. It proposes FOVEATED, which perturbs preceding-context key positions during editing, and reports improved atomic-fact recall across several editors while largely retaining passage recall.

Critical strengths

The paper addresses a practical problem: an edited model may reproduce a passage but fail to recall its individual facts. The proposed method improves atomic-fact recall across several editors without requiring additional atomic question–answer supervision.

Critical weaknesses
  1. The results show improvements over the base editors, but the advantage of using positional perturbation is less clear. Simpler ways of reducing contextual dependence may also improve fact recall. Comparisons with such approaches would help explain what the proposed design adds.
  2. The evidence for preserving existing knowledge is weaker than the evidence for improving fact recall. Some general capabilities decline after applying the method, and its effect on related but unedited facts is not examined directly. These issues make it difficult to assess the overall quality of the edits.
Questions for authors
  1. Appendix D.5 reports a substantial increase in editing time. Would the advantage remain if the base editors were given a comparable computational budget?
  2. In Table 12, some methods show lower general-capability scores on Llama3.1 after applying FOVEATED. Further discussion of these decreases would help clarify the trade-off between fact recall and capability preservation.
3 comments

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

Comparison of FOVEATED with other ways of reducing contextual dependence.

Conceptually, FOVEATED aims to break the context shortcut, thereby mitigating the identified context-induced difficulty underestimation issue (our first contribution), by randomly jittering the positions of the context. From a high level, this idea is no more conceptually complex than masking or dropping the context. All of them are straightforward ways to instantiate the context-perturbing mechanism.

Practically, choosing the proper degree of perturbation can be tricky: an ideal one should weaken the context enough to prevent a shortcut, but not so much as to destroy its information. As ablated in our paper, using a coarse perturbation such as perturbing the position IDs shared by queries and keys instead of the key side only (as FOVEATED does) leads to worse atomic-fact recall and Reph Q. Following the reviewer's suggestions, we further experimented with masking and dropping the context, as two other alternative solutions. From the results below, we again found a similar pattern: both approaches improve atomic-fact recall, but fall short of FOVEATED in balancing atomic and holistic performance. This highlights the importance of the perturbation choice in practice, which is what FOVEATED, the second contribution of this work, is designed for.

We have revised our paper to make it clearer.

MethodOrig QReph QAtom ROUGE-LHit RateC_LCS
FT-M (DO)100.081.320.815.928.8
Drop ctx.47.841.923.817.331.3
Mask ctx.95.973.625.120.935.2
+Ours99.484.625.119.433.3
μKE (LTE)98.582.124.615.735.9
Drop ctx.98.574.827.821.240.5
Mask ctx.99.676.027.320.541.9
+Ours99.985.529.524.245.2

Preservation of unedited facts and general capabilities.

We thank the reviewer for raising this point. In this work we follow the convention in UKE literature [1] to evaluate the edited models' general capabilities. Following the reviewer's suggestions, we conducted additional experiments with the neighborhood score from structured KE [2], which measures whether related but unedited facts are preserved. From the results below, we found that FOVEATED brings modest changes, with the neighborhood score staying within 2 pts of the base editors.

More broadly, regarding the editing-preservation trade-off, we noted that base editors differ in capability preservation across Qwen2.5 and Llama3.1 backbones. On top of them, FOVEATED brings varied improvements and degradations that are on average modest. We view this varied editing-preservation trade-off as a joint effect of the base editor and the focused-view design. In this work we designed FOVEATED as a general plug-and-play module that is agnostic to any specific editor. We believe that a better trade-off can be achieved by developing more specialized editors.

We have revised our paper based on the results and discussions here.

BackboneMethodNS (pre-edit)NS (post-edit)Orig QReph QAtom ROUGE-LHit RateC_LCS
Qwen2.5FT-M80.078.6100.027.439.832.945.2
+Ours80.076.998.138.444.539.151.3
μKE80.280.697.232.031.327.841.8
+Ours80.280.199.546.938.237.248.5
Llama3.1FT-M81.276.2100.061.652.257.168.8
+Ours81.275.2100.078.152.761.472.4
μKE81.281.998.447.937.537.851.2
+Ours81.280.4100.047.939.739.552.0

[1] FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing, 2026

[2] Locating and Editing Factual Associations in GPT, 2022

Computational overhead of FOVEATED and comparison under matched compute budgets.

Following the reviewer's suggestion, we studied whether the base editor can benefit from more compute. To this end, we trained FT-M with more steps so that its estimated editing time approximately matches that of FT-M with FOVEATED. Results are shown below. We note that additional compute brings little gain in atomic-fact recall to FT-M, and does not close the gap to FOVEATED. Therefore, the gain of FOVEATED does not simply come from additional compute.

MethodStepsTime (s/edit)Orig QReph QAtom ROUGE-LHit RateC_LCS
FT-M403.1100.081.320.815.928.8
FT-M180≈13.8100.082.421.315.129.2
+Ours4013.799.484.625.119.433.3

Dear Reviewer 4T3c,

Thank you once again for your commitment to reviewing our paper and helping us improve our work.

As today is the final day of the author-reviewer discussion phase, we would greatly appreciate it if you could review our responses and let us know whether they address your concerns. If you feel that our responses and the additional results have resolved them, we would be grateful if you would consider raising your rating.

Best regards,

Authors

Reviewer_x6muOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Review Comments The paper studies an interesting problem in unstructured knowledge editing and shows consistent improvements in atomic-fact recall. I have the following concerns.

  1. The authors attribute the poor recall of individual facts to context-induced difficulty underestimation. The decreasing pre-edit NLL across passage positions is an interesting observation, but it does not necessarily mean that later facts are learned less effectively. The paper would be more convincing if it examined how contextual support affects the actual learning of individual facts, rather than relying mainly on the relationship between token-level loss and passage position.

  2. I am not fully convinced that key-side RoPE perturbation is essential to the reported improvements. The current ablations compare different positional perturbation schemes, but do not rule out simpler ways of weakening contextual dependence, such as masking or randomly dropping preceding context. Without these comparisons, it is difficult to tell whether the gains come from the proposed mechanism or from a more general regularization effect. The theoretical analysis does not settle this distinction either.

  3. The comparison with FABLE is particularly important here. FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing (Findings of ACL 2026) addresses the same atomic-fact recall problem, introduces the UnFine benchmark, and uses a two-stage editing procedure that first writes sentence-level knowledge into shallow layers before performing passage-level editing in deeper layers. The LTE implementation in this paper follows essentially the same structure. Although FOVEATED constructs its targets through positional perturbation rather than externally generated QA pairs, this difference alone does not establish its advantage over FABLE. Excluding FABLE from the main comparison because of its additional supervision leaves an important question unanswered: how competitive is FOVEATED against an existing method designed specifically for this problem?

  4. Some aspects of the experimental setup make the performance comparisons less convincing. The authors use a simplified version of COIN rather than the original method, and FOVEATED substantially increases editing time. The additional cost is not adequately reflected in the discussion of its practical benefits. Comparisons with the complete COIN method and under comparable computational budgets would be more informative. Also, all main experiments use single-request editing, so it is unclear whether the improvements persist when multiple knowledge updates accumulate.

  5. The method would be easier to follow with an overall framework figure. In particular, the paper should visually distinguish the key-side RoPE perturbation from the unchanged query positions and show how the resulting focused views are used differently in DO and LTE editors. The current presentation relies heavily on equations and textual descriptions, making the complete editing procedure unnecessarily difficult to follow.

Critical strengths

The paper studies an interesting problem in unstructured knowledge editing and shows consistent improvements in atomic-fact recall. I have the following concerns.

  1. The authors attribute the poor recall of individual facts to context-induced difficulty underestimation. The decreasing pre-edit NLL across passage positions is an interesting observation, but it does not necessarily mean that later facts are learned less effectively. The paper would be more convincing if it examined how contextual support affects the actual learning of individual facts, rather than relying mainly on the relationship between token-level loss and passage position.

  2. I am not fully convinced that key-side RoPE perturbation is essential to the reported improvements. The current ablations compare different positional perturbation schemes, but do not rule out simpler ways of weakening contextual dependence, such as masking or randomly dropping preceding context. Without these comparisons, it is difficult to tell whether the gains come from the proposed mechanism or from a more general regularization effect. The theoretical analysis does not settle this distinction either.

  3. The comparison with FABLE is particularly important here. FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing (Findings of ACL 2026) addresses the same atomic-fact recall problem, introduces the UnFine benchmark, and uses a two-stage editing procedure that first writes sentence-level knowledge into shallow layers before performing passage-level editing in deeper layers. The LTE implementation in this paper follows essentially the same structure. Although FOVEATED constructs its targets through positional perturbation rather than externally generated QA pairs, this difference alone does not establish its advantage over FABLE. Excluding FABLE from the main comparison because of its additional supervision leaves an important question unanswered: how competitive is FOVEATED against an existing method designed specifically for this problem?

  4. Some aspects of the experimental setup make the performance comparisons less convincing. The authors use a simplified version of COIN rather than the original method, and FOVEATED substantially increases editing time. The additional cost is not adequately reflected in the discussion of its practical benefits. Comparisons with the complete COIN method and under comparable computational budgets would be more informative. Also, all main experiments use single-request editing, so it is unclear whether the improvements persist when multiple knowledge updates accumulate.

  5. The method would be easier to follow with an overall framework figure. In particular, the paper should visually distinguish the key-side RoPE perturbation from the unchanged query positions and show how the resulting focused views are used differently in DO and LTE editors. The current presentation relies heavily on equations and textual descriptions, making the complete editing procedure unnecessarily difficult to follow.

Critical weaknesses

None

Questions for authors

None

4 comments

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

Effect of contextual support on the learning of individual facts.

We thank the reviewer for the comment. This work reveals context-induced difficulty underestimation behind context reliance that can lead to poor atomic-fact recall [1, 2]. Our understanding builds upon in UKE, the edited knowledge is often counterfactual and unknown to the model by construction. Therefore, the lower pre-edit loss at later positions cannot come from the model already knowing these facts, and we attribute it to the support from the preceding context. We also show that the loss remains lower at later positions however the sentences are ordered, which further rules out that later sentences are simply easier.

Following the reviewer's suggestions, we further conducted a controlled study that relates the contextual support to the recall after editing. We measured the contextual support of each fact by how much the preceding context reduces the pre-edit loss of its sentence. Note that this definition does not depend on any editing outcome. Based on this idea, we compared facts with low and high support at the same position and of similar difficulty. From the results below, we found that facts with higher contextual support usually benefit more from FOVEATED, which again supports our finding.

We have added these results to the revised paper.

DatasetSupportBase Editor (HR) on Qwen2.5+OursGainsBase Editor (HR) on Llama3.1+OursGains
UnFine-UnKELow19.929.1+9.225.832.1+6.3
High20.930.3+9.424.434.4+10.0
UnFine-CFLow36.442.1+5.749.551.5+2.0
High33.143.1+10.045.449.0+3.6
UnFine-MQLow40.844.2+3.448.651.4+2.8
High38.444.6+6.247.151.6+4.5

Tab. Atomic-fact recall by contextual support level, averaged over the five editors (more details are in App. D.2)

[1] Uncovering Context Reliance in Unstructured Knowledge Editing, 2026

[2] FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing, 2026

Comparison with masking or dropping the preceding context.

Conceptually, FOVEATED aims to break the context shortcut, thereby mitigating the identified context-induced difficulty underestimation issue (our first contribution), by randomly jittering the positions of the context. From a high level, this idea is no more conceptually complex than masking or dropping the context. All of them are straightforward ways to instantiate the context-perturbing mechanism.

Practically, choosing the proper degree of perturbation can be tricky: an ideal one should weaken the context enough to prevent a shortcut, but not so much as to destroy its information. As ablated in our paper, using a coarse perturbation such as perturbing the position IDs shared by queries and keys instead of the key side only (as FOVEATED does) leads to worse atomic-fact recall and Reph Q. Following the reviewer's suggestions, we further experimented with masking and dropping the context, as two other alternative solutions. From the results below, we again found a similar pattern: both approaches improve atomic-fact recall, but fall short of FOVEATED in balancing atomic and holistic performance. This highlights the importance of the perturbation choice in practice, which is what FOVEATED, the second contribution of this work, is designed for.

We have revised our paper to make it clearer.

MethodOrig QReph QAtom ROUGE-LHit RateC_LCS
FT-M (DO)100.081.320.815.928.8
Drop ctx.47.841.923.817.331.3
Mask ctx.95.973.625.120.935.2
+Ours99.484.625.119.433.3
μKE (LTE)98.582.124.615.735.9
Drop ctx.98.574.827.821.240.5
Mask ctx.99.676.027.320.541.9
+Ours99.985.529.524.245.2

Comparison with FABLE.

In our paper, we adopted the two-stage design of FABLE when applying FOVEATED to the LTE family. We did not include it in the main comparison as it additionally uses frontier models to synthesize atomic QA pairs to edit the model directly, which would make its comparison to other editors unfair. Following the reviewer's suggestions, we conducted additional experiments with FABLE using its official code. Given its direct atomic QA synthesis, we view FABLE as a ceiling reference.

Our results are reported below. We found that FOVEATED, which improves how the passage itself is learned, without any augmented data, is competitive with this ceiling reference: it reaches comparable atomic-fact recall on Qwen2.5 and consistently better Reph Q, while the instruction-aligned FABLE-I is higher on Llama3.1. We also noticed that FABLE can be sensitive to the instruction format (mis)alignment. FABLE-D, which directly uses the official implementation, falls notably behind FABLE-I, which additionally incorporates our exact evaluation instruction into its atomic QA training.

We have added these results and discussions to the revised paper.

Qwen2.5

DatasetMethodOrig QReph QAvg.Atom ROUGE-LHit RateC_LCS
UnFine-UnKEUnKE94.070.682.323.815.034.2
+Ours88.075.081.529.023.745.1
FABLE-D98.849.974.421.911.429.3
FABLE-I95.759.577.634.929.441.3
UnFine-CFUnKE84.129.556.830.127.840.5
+Ours89.643.866.736.835.346.6
FABLE-D98.320.659.426.619.231.1
FABLE-I94.921.958.439.335.344.5
UnFine-MQUnKE85.5--85.532.533.243.5
+Ours84.6--84.637.937.548.2
FABLE-D96.4--96.430.625.735.7
FABLE-I93.4--93.441.538.146.2

Llama3.1

DatasetMethodOrig QReph QAvg.Atom ROUGE-LHit RateC_LCS
UnFine-UnKEUnKE92.686.989.722.116.334.7
+Ours94.689.091.829.126.644.7
FABLE-D99.367.083.233.124.340.7
FABLE-I99.278.188.646.846.054.0
UnFine-CFUnKE93.439.966.737.137.148.9
+Ours95.047.371.235.836.348.9
FABLE-D99.512.856.144.239.450.2
FABLE-I98.717.658.252.353.460.4
UnFine-MQUnKE94.0--94.036.842.451.3
+Ours91.4--91.439.843.752.1
FABLE-D99.9--99.943.039.949.3
FABLE-I98.2--98.256.055.662.5

[1] FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing, 2026

Comparison with the complete COIN.

In our paper, we used a simplified COIN that omits its locality-preserving mechanisms (which make editing more conservative), while keeping its original editing objective. We showed that FOVEATED can offer complementary gains to its editing performance.

Following the reviewer's suggestions, we evaluated FOVEATED with the complete COIN design (with the coefficients from its paper without further tuning). Results are shown in the table below. We found that the complete COIN indeed leads to more conservative editing (indicated by the smaller update magnitude ∥ΔW∥F\lVert \Delta_W \rVert_F), and yields lower holistic and atomic fact recall than the simplified COIN. On both versions, FOVEATED improves atomic-fact recall.

MethodOrig QReph QAtom ROUGE-LHit RateC_LCS∥ΔW∥F\lVert \Delta_W \rVert_F
COIN (simplified)100.098.034.933.748.823.3
+Ours99.298.241.146.658.924.0
COIN (complete)100.082.821.817.330.25.9
+Ours99.483.625.420.234.16.1

Computational overhead of FOVEATED and comparison under matched compute budgets.

Following the reviewer's suggestion, we studied whether the base editor can benefit from more compute. To this end, we trained FT-M with more steps so that its estimated editing time approximately matches that of FT-M with FOVEATED. Results are shown below. We note that additional compute brings little gain in atomic-fact recall to FT-M, and does not close the gap to FOVEATED. Therefore, the gain of FOVEATED does not simply come from additional compute.

MethodStepsTime (s/edit)Orig QReph QAtom ROUGE-LHit RateC_LCS
FT-M403.1100.081.320.815.928.8
FT-M180≈13.8100.082.421.315.129.2
+Ours4013.799.484.625.119.433.3

Sequential editing.

In our paper, we followed prior UKE studies and focused on the single-editing setting. FOVEATED is designed as a general plug-and-play module, and we did not include any editor-specific or sequential-editing considerations in our methodological design. To test its empirical robustness, we followed the reviewer's suggestions and conducted additional experiments on sequential editing with T=10T=10 and T=50T=50 edits (one edit per step). The edited model is evaluated after all TT edits are completed. Results are shown below. We note that both editors degrade as edits accumulate, while the gains from FOVEATED remain robust. Given these considerations, we believe that accounting for the interplay among accumulated edits can bring further improvements, which we discuss as an open question in the conclusion.

SettingMethodOrig QReph QAtom ROUGE-LHit RateC_LCS
Single editFT-M100.081.320.815.928.8
+Ours99.484.625.119.433.3
T=10T=10FT-M69.360.025.319.637.7
+Ours73.264.232.829.947.7
T=50T=50FT-M56.152.725.216.437.2
+Ours55.652.826.620.841.2

Overall framework figure.

We thank the reviewer for the suggestion. We have added an overview figure (Fig. 2 in the revised paper), which illustrates the key-side RoPE perturbation with unchanged queries and how the resulting focused views are used in DO and LTE editors.

Dear Reviewer x6mu,

Thank you once again for your commitment to reviewing our paper and helping us improve our work.

As today is the final day of the author-reviewer discussion phase, we would greatly appreciate it if you could review our responses and let us know whether they address your concerns. If you feel that our responses and the additional results have resolved them, we would be grateful if you would consider raising your rating.

Best regards,

Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of our paper revision, and we thank all reviewers for providing valuable feedbacks.

Dear Reviewers and Area Chairs,

Thank you for the careful reading of our paper and the constructive feedback. Your comments have helped us strengthen the paper. In response, we have uploaded a revised version with the changes below (marked in blue).

  1. We extended our analyses and ablation studies, including further evidence on context-induced difficulty underestimation, comparisons with broader context perturbations and with a reference using external supervision, and a broader evaluation of capability preservation. The new results agree with our original findings and further support the design of FOVEATED.

  2. We refined the presentation, including an overview figure of FOVEATED, more readable figures and case studies, and the discussions on capability preservation and open questions.

Thank you again for your time. We are happy to answer any further questions.

Best regards,

Authors

AuthorsOpenReview ↗

Gentle Follow-up

Dear Reviewers and Area Chair,

Thank you again for your time and valuable comments on our paper, "Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing". As the discussion phase is approaching its end, we would greatly appreciate it if you could let us know whether any questions remain after our responses and the revised paper. We would be glad to provide any further clarification.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.