DocPatch: Continual Knowledge Internalization with Conflict-Aware LoRA Memories

Large language models (LLMs) can acquire external knowledge through in-context retrieval or parameter updates, but these approaches either incur repeated context costs or risk interference with previously learned knowledge. We propose DocPatch, a continual parametric knowledge framework that represents documents as modular and reusable LoRA-based memories while keeping the base language model frozen. Given a collection of documents, DocPatch uses a pretrained Doc-to-LoRA hypernetwork to convert short document chunks into knowledge LoRAs, which are generated offline and stored in a persistent knowledge bank. At inference time, a query-dependent router retrieves relevant LoRAs using their cached parametric representations, allowing the model to answer from a question-only prompt without reintroducing the original document text into the context window. To support evolving knowledge, DocPatch preserves LoRAs from different document versions and associates them with source, version, and conflict metadata, enabling newer conflicting knowledge to override outdated information while retaining earlier non-conflicting knowledge. We further introduce a lightweight global reasoning adapter that teaches the frozen LLM to follow activated knowledge LoRAs over conflicting model priors and to combine complementary LoRAs for multi-step reasoning. By separating knowledge storage, retrieval, conflict resolution, and reasoning, DocPatch provides a modular framework for continually extending and updating the parametric knowledge of frozen language models without repeatedly retraining the base model.