DARC: Denoised Authorized Reparameterized Coding for LLM Unlearning via First-Order Noise Cancellation
Abstract
LLM unlearning faces a disclosure conflict when a server keeps its model parameters confidential and a client withholds raw forget data from that server. Noisy model publication can limit disclosure, but it also changes the unlearning update. We propose DARC (Denoised Authorized Reparameterized Coding), coupling model publication with authorized update recovery through executors granted temporary access to client data. The server releases model copies using correlated Gaussian perturbations and function-preserving coordinate transformations, under an isolation assumption limiting each coalition to same-round copies. After aligning all returned updates, inverse-scale aggregation cancels the common first-order perturbation response of a shared locally smooth update map. We prove the decoder's uniqueness among normalized linear rules with universal first-order cancellation and analyze curvature, optimizer, and stochastic errors. Under a fixed public positive-definite noise covariance and independent key/code sampling, we derive exact neighboring-checkpoint R\'enyi divergence for key-revealed views and extend checkpoint-privacy bounds to complete executor transcripts under the stated auxiliary-release conditions. Our optimizer analysis identifies exact-arithmetic Adam-family compatibility conditions and quantifies discrepancies from continuous attention rotations. Experiments on Llama and Qwen series models with TOFU and MUSE benchmarks assess forgetting, retained utility, and update fidelity. For Llama-1B on TOFU, the six non-collapsed trainers have utility scores within of their noise-free references. Targeted diagnostics reveal numerical compatibility gaps and closely matching outputs from individual published views on the tested prompts. DARC connects bounded checkpoint disclosure with optimizer-aware approximation of a chosen unlearning method's noise-free update.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.