acceptodds
Withdrawn from ICLR 2027

DocPatch: Continual Knowledge Internalization with Conflict-Aware LoRA Memories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models (LLMs) can acquire external knowledge through in-context retrieval or parameter updates, but these approaches either incur repeated context costs or risk interference with previously learned knowledge. We propose DocPatch, a continual parametric knowledge framework that represents documents as modular and reusable LoRA-based memories while keeping the base language model frozen. Given a collection of documents, DocPatch uses a pretrained Doc-to-LoRA hypernetwork to convert short document chunks into knowledge LoRAs, which are generated offline and stored in a persistent knowledge bank. At inference time, a query-dependent router retrieves relevant LoRAs using their cached parametric representations, allowing the model to answer from a question-only prompt without reintroducing the original document text into the context window. To support evolving knowledge, DocPatch preserves LoRAs from different document versions and associates them with source, version, and conflict metadata, enabling newer conflicting knowledge to override outdated information while retaining earlier non-conflicting knowledge. We further introduce a lightweight global reasoning adapter that teaches the frozen LLM to follow activated knowledge LoRAs over conflicting model priors and to combine complementary LoRAs for multi-step reasoning. By separating knowledge storage, retrieval, conflict resolution, and reasoning, DocPatch provides a modular framework for continually extending and updating the parametric knowledge of frozen language models without repeatedly retraining the base model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM, internalization, conflict knowledge, Question Answering, LoRA

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_3mzdOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

DocPatch is a parametric memory framework for using knowledge from continually updated documents with a frozen LLM. It converts document chunks into LoRAs using a pretrained Doc-to-LoRA hypernetwork, stores them in a knowledge bank, and retrieves and composes relevant LoRAs for each question. When a document is revised, new LoRAs are added. Given version and conflict metadata, the framework excludes superseded memories from the retrieved set when composing conflicting information. A shared reasoning adapter is trained to follow the activated knowledge and use information from multiple memories. The authors evaluate QA and routing performance on HotpotQA, as well as knowledge updating and preservation under sequential document revisions. They also report ablations on the reasoning adapter, LoRA selection and weighting, and counterfactual training, together with an inference efficiency comparison.

Critical strengths

I think the combination of reusable document-specific knowledge LoRAs and a shared adapter for using that knowledge is worth studying, and is included in a important problem statement. It fits the goal of incorporating new documents while reusing existing memories. I also found the separate evaluation of query-dependent weighting and reasoning training in Tables 3 and 5 useful. These results provide evidence about which components affect QA performance when building a system around document-derived parametric memories.

Critical weaknesses

1. I do not clearly understand what the paper resolves beyond existing work.

I agree with the motivation of managing knowledge from continually changing documents. What I found unconvincing was the next step: what specifically makes this difficult for existing document-derived LoRA memories, and why the proposed method addresses that difficulty well.

I could broadly follow the roles of routing, composition, conflict resolution, and reasoning training. The paper also includes useful ablations. However, I still had trouble identifying the central difficulty that DocPatch resolves. Showing that individual components improve performance is useful, but I also want to understand which limitation of existing approaches these improvements overcome.

Missing related work makes this harder to assess. PRAG and DyPRAG study the parameterization, retrieval, and use of document knowledge. Knowledge Modules/DCD separates knowledge modules from an adapter trained to use that knowledge. LAG studies selection from a library of knowledge and task LoRAs, while Understanding LoRA as Knowledge Memory analyzes memory capacity, routing, and composition. I think the paper needs to cite these paper and explain its relationship to prior studies so that readers can identify its starting point and additional contribution.

Other relevant studies are also missing, including Arrow on reusing and routing LoRA libraries, LoRA Soups on weighted LoRA composition and skill composition, and ELDER on lifelong model editing with multiple LoRAs and a router.

Overall, I did not come away with a clear understanding of what new result the paper establishes about continual LoRA memory under its stated version and conflict metadata assumptions.

2. I had difficulty understanding how the central experiments were conducted and what the results demonstrate.

Section 4.1 states that the authors use HotpotQA and report EM and token-level F1. However, the evaluation split and configuration, as well as the scope of the LoRA bank searched by the router, are unclear. I think this information is necessary to interpret the results. Retrieving from a question-specific candidate set and retrieving from the entire evaluation corpus require different retrieval capabilities.

Section 4.5 and Table 4 were particularly difficult to follow. The paper explains that documents are revised sequentially while previous LoRAs are retained. However, I could not determine which documents were used to construct these revision histories, how questions and answers were generated, or when evaluation took place during the update sequence. The brief descriptions of Unchanged, Updated, Old-Only, Override, and Unrelated were insufficient for me to reconstruct the actual evaluation. The precise metric reported in each column also needs clarification.

I also do not understand how ROME and MEMIT were applied. These methods modify model parameters to change specified factual associations. Applying them to document revisions requires an explanation of how document changes were converted into edit requests. Which facts were edited, how were edits applied across revisions, and what information was provided to each method?

The lack of explanatory figures and examples adds to this difficulty. A example connecting document revisions, metadata, activated memories, and expected answers would have helped me understand both the method and Table 4. Currently, the explanation relies on one overview diagram and aggregate performance tables.

Questions for authors

What is the most important difficulty you aim to address when applying existing document-derived LoRA memories to continually updated document collections? How does DocPatch address it, and which result demonstrates this most directly? Please explain this in relation to the prior work discussed above.

Could you provide a concrete example of a document revision history and its associated questions and answers in Table 4? When is evaluation performed after each revision, and what knowledge is considered correct, particularly for Old-Only and Override? What metric does each column report, and how are results aggregated across revision steps?

How were the version precedence and conflict-group annotations constructed? Please distinguish information supplied by the document update process from annotations constructed separately. I would also like to understand how an actual document revision is represented as conflicts between chunks, and which older memories remain active as a result.

How were document revisions provided to the editing baselines, including ROME and MEMIT? Please describe how edit requests were constructed from documents and whether they were applied sequentially or in batches. What information did DocPatch and each baseline receive at update time?

How were the counterfactual training pairs constructed from documents and questions? How were documents and revision histories separated between training and evaluation? Did QA baselines such as RAG also receive the QA supervision used to train the reasoning adapter? I think this matters for interpreting the roles of memory design and additional training.

Reviewer_LW8COpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduce DocPatch framework for maintaining and using external document knowledge through modular LoRA memories. A frozen Doc-to-LoRA hypernetwork converts document chunks into knowledge adapters, which are stored with routing representations. At inference time, the system retrieves relevant adapters, filters conflicting versions according to their precedence, and composes the remaining parameter updates. A shared reasoning LoRA is trained with answer supervision and counterfactual examples to help the frozen language model use the activated knowledge. The experiments use Qwen3-4B, train the router on Constitution QA, and evaluate routing and question answering on HotpotQA. The paper also evaluates successive document revisions and reports improvements over the selected RAG and D2L baselines, along with lower inference latency from shorter input contexts.

Critical strengths
  1. The framework separates document knowledge storage from query-dependent selection. New documents can be added without reoptimizing the base model or overwriting existing knowledge adapters. This modular design is useful for managing an evolving document collection.

  2. The ablations provide evidence that counterfactual training helps the model answer according to the currently activated knowledge and improves its handling of knowledge conflicts.

Critical weaknesses
  1. The contribution over closely related parametric RAG methods is not sufficient. The overall design overlaps substantially with existing work. [1, 2] already encodes documents as LoRA modules, also retrieves and merges the corresponding parameters. Poly-PRAG [3] studies routing among LoRA modules for parametric knowledge. These methods are highly relevant, but the paper lacks sufficient discussion.

  2. The global reasoning LoRA and the Doc-to-LoRA hypernetwork lack generalization tests on unseen datasets or domains.

[1] Parametric Retrieval Augmented Generation.

[2] Better wit than wealth: Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement.

[3] Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters.

Questions for authors
  1. What is the main technical contribution relative to PRAG, DyPRAG, and Poly-PRAG?

  2. Could you provide evidence of OOD generalization for the full system?

Reviewer_XQzFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DOCPATCH, a framework that stores knowledge in LoRA memory matrices and generates compact representation vectors through low-dimensional projections, thereby enabling efficient retrieval of these memories. The framework outperforms baseline methods on document QA benchmarks, demonstrating its practical utility.

Critical strengths
  1. Effective Counterfactual Reasoning Adapter: Training a shared reasoning adapter on counterfactual pairs effectively forces the frozen LLM to follow activated parametric memories over pretrained priors, yielding a substantial +15.8 EM gain in ablations (Table 5).
  2. Clear Modular Design: Scaling Doc-to-LoRA from a single document to a continually updated, multi-document LoRA library is a practical and well-motivated direction.
Critical weaknesses
  1. Strong Metadata Assumption and Incomplete Baseline Comparison: The conflict resolution mechanism assumes that conflict group IDs and version precedence are already annotated in the metadata during knowledge bank construction (Section 3.4). Under this assumption, resolving conflicts mainly involves masking out older versions from the same conflict group. If standard RAG were provided with the same metadata, it could similarly filter out outdated text chunks before generation. Because the Override score for RAG is not reported in Table 4, it remains unclear how much DOCPATCH improves over a metadata-filtered RAG baseline.
  2. Limited Candidate Pool Scale in Routing Evaluation: While the framework is designed for a persistent knowledge bank over evolving document collections, the routing evaluation in Table 2 appears to use a very small candidate pool per query. Specifically, the Random baseline achieves a 30.65% Top-5 Recall and 23.48% Accuracy, suggesting that each query is only routed over roughly 10 to 16 candidate chunks rather than thousands of LoRAs in a unified bank. Evaluating only on small candidate sets leaves the scalability of parameter-sketch retrieval unclear.
Questions for authors

See the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.