acceptodds
Under review as a conference paper at ICLR 2027

UAR-LoRA: Uncertainty-Aware Rank-Adaptive Low-Rank Adaptation for Lifelong Knowledge Editing

Abstract

Lifelong knowledge editing keeps a deployed large language model factually current one edit at a time, so thousands of corrections must accumulate without eroding earlier edits or unrelated knowledge. Existing low-rank adapter editors do not control, for each arriving edit, how much capacity its expert takes, which queries reach it and where it sits: a rank fixed before the stream starves hard corrections, a router without a calibrated threshold serves queries the pool should leave to the frozen base, and randomly initialised experts overlap earlier ones. We propose UAR-LoRA, an uncertainty-aware, rank-adaptive framework that treats capacity, admissibility and geometry as three conditions of one heuristic capacity-coherence criterion and meets them as each expert enters the pool. An Entropy Probe and Adaptive Rank Selector sizes each expert from the frozen base's predictive entropy, a Confidence-Gated Router returns queries below a calibrated threshold to the frozen base, and a Rank-Aware Orthogonal Initialiser places each new expert in directions no earlier expert occupies while free ones remain, with a truncated-SVD fallback once they run out. Averaged over three backbones on CounterFact at , UAR-LoRA reaches mean Reliability against for ELDER, and on LLaMA-2-7B it lowers Expected Calibration Error from to at a trainable-parameter ratio relative to fixed rank- LoRA. No single-module variant exceeds Reliability against with all three, and the probe matches a fixed rank of at rather than . Forgetting also accumulates more slowly: over ZsRE edits UAR-LoRA retains Reliability where ELDER falls to , losing points from the first edit against ELDER's .

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.