- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends TopKLoRA into a framework for auditing finetuning updates. Its adapters activate only a small subset of latents per token, and Contrastive Latent Circuit Discovery (CLCD) identifies behaviorally relevant subsets through contrastive attribution followed by necessity and sufficiency tests. Experiments on three models compare sparse and dense adapters, examine placement across layers, and measure benign capability after backdoor removal. Gradient routing provides additional validation using known implantation locations.
The main finding is that sparse adapters distributed across layers support substantially smaller and more selective edits while preserving most benign finetuning gains.
Critical strengths
I put most weight on the connection between circuit localization and useful behavioral editing. In the Gemma nine layer setting in Table 2, circuit size falls from 69.4% to 7.4% of adapter latents while retention rises from 0.28 to 0.98. The clean twin comparison in Table 4 further shows that the edited model approaches a counterpart trained without poisoning.
The validation also goes beyond attribution rankings. The paper tests both removal and retention of candidate subsets, includes random ablations of the same size, and evaluates residual backdoor behavior on an independent set of 4,000 prompts in Appendix G.3. Gradient routing adds cases with known implantation locations. Together, these checks provide credible evidence that the selected subsets have a specific causal role.
Critical weaknesses
My main reservation is that the evidence for causal localization is stronger than the evidence for mechanistic explanation. Section 7 explicitly leaves both individual latent semantics and interactions with the frozen base model unanalysed. Necessity and sufficiency establish which adapter subsets control the backdoor, but do not yet explain what trigger information those latents read, how they communicate it, or how they produce the malicious continuation.
This limits the breadth of the interpretable by design claim, while leaving the practical removal result intact. A detailed analysis of one representative circuit, separating trigger recognition from output control and testing those proposed roles through targeted interventions, would address this gap more directly than additional removal experiments of the same kind.
Questions for authors
Appendix G.3 reports backdoor leakage after ablation on an independent set of 4,000 prompts. Was keepOnly sufficiency also evaluated on that set, or only on the 1,000 verification prompts used to choose circuit size? If those measurements are available, please report the paired behavior rates for keepOnly and the intact model, with uncertainty on their difference. This would affect my confidence in generalization of the necessary and sufficient circuit claim: removal generalizing to new prompts does not by itself establish that the retained subset still reproduces the behavior.
We thank the reviewer for their thoughtful feedback. We have adequately revised our submission to reflect the advice and have highlighted changes in blue to make them easy to find. We respectfully ask the reviewer to refer to the updated manuscript.
Question 1
We appreciate this suggestion. Originally, we did not evaluate keepOnly sufficiency on a held-out set, but following the reviewer’s suggestion, we ran additional experiments across all our models and incorporated them into the manuscript. We agree that applying the same methodology to test sufficiency can determine how likely the method is to generalise.
We show that in the keepOnly setting, the verified circuits retain nearly all of intact’s ASR, although about half do not meet the exact 2SE rule we used for circuit verification (but narrowly miss). Below, we argue that this is partly a consequence of SE shrinking with increased sample size. We include these details in a reworked section G.3 of the Appendix: “Held-out necessity and sufficiency of the verified circuits”, specifically Table 14.
We propose the following wording:
“Held-out sufficiency. We repeat the test for sufficiency, keeping only the verified circuit on the same 4,000 held-out prompt dataset (Table 14). Keep-only retains at least 98% of the intact ASR in every organism (median 99.9%) and stays above our 90% ASR requirement. The decrease nonetheless exceeds 2 SE for 46 of 89 organisms (rule used for sufficiency verification from Section 4.2.2). We attribute these misses to the way circuits are constructed and verified: the verified circuit size K is, by construction, the first grid point to pass. The tolerance of the 2SE rule used for sufficiency criterion verification shrinks with the number of prompts, so at 4,000 prompts it is twice as sensitive as at the verification split of size 1,000. Splitting the 4,000 held-out prompts into 4x1,000 bands, we observe across 89 organisms that 300 of 356 bands pass 2 SE.”
We thank the reviewer again for their constructive feedback. We hope that our additional experiments and revisions adequately address their concerns. If so, we would be grateful if the reviewer could reconsider their assessment and score in light of the updated manuscript.