- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies whether backdoored LoRA adapters can be detected from adapter weights without executing the underlying model. The authors introduce a reference-free directional statistic, termed circuit_score, which measures the ratio between input-output singular-vector alignment and cross-layer output coherence. They evaluate this statistic together with a supervised axis projection and a benign-calibrated Mahalanobis detector across text and vision-language backdoor settings.
Critical strengths
Detecting a malicious LoRA adapter before executing the underlying model is a useful supply-chain security problem. The paper formulates this setting clearly: the screener receives the adapter factors and configuration but has no access to the base model, inference behavior, or trigger information. This makes the proposed static analysis setting substantially more realistic than defenses that require running the potentially compromised model.
Critical weaknesses
The prose is highly formulaic and repetitive, which makes the paper difficult to read and obscures the main technical contribution, and the main text is under 9 pages. The figures are crude and occupy a large amount of space. There is no schematic diagram of the method; the logic is confusing, and it is difficult to read. The experimental setup is in Section 3.3, Section 4 is the method, and Section 5 is the results analysis.
The central inverse detectability–disruption claim is not causally identified. The paper claims that backdoors with lower behavioral disruption can be more detectable in weight space. However, the three VLM recipes used to support this observation differ simultaneously in several important factors, including trigger script, target language, attack success rate, false-trigger rate, and clean-accuracy loss. For example, the Russian, Canadian, and Tamil designs have ASRs of 92.6%, 88.9%, and 48.9%, respectively, and substantially different FTR and clean-accuracy losses. Consequently, the observed ordering of AUROC could be caused by differences in trigger construction, target representation, language/script, ASR, or other properties of the training procedure rather than behavioral disruption itself. The authors acknowledge this limitation, but it affects a central scientific claim rather than being merely a peripheral limitation. A controlled experiment varying behavioral disruption while keeping trigger type, target, ASR, poisoning budget, and training procedure fixed would be needed to substantiate an inverse relationship.“Our adaptive experiments target a single statistic and measure the resulting exposure to complementary readouts; adaptation to the complete screening rule remains to be evaluated.” The authors themselves also stated that this paper is an incomplete research.
Questions for authors
Please respond to the weaknesses raised, make a serious effort to improve the writing quality, clearly define the problem, clarify the innovations, and improve the experiments.
The strongest detection results require calibration/reference data, which weakens the “reference-free detection” framing. The raw circuit score itself can indeed be computed from one adapter without reference examples, but converting it into a practical detector requires choosing a suspicious direction and threshold. Moreover, the strongest reported separation generally comes from the supervised axis scorer or the benign-calibrated Mahalanobis scorer. The current evaluation also reuses all 15 benign adapters for feature standardization/centroid estimation and Mahalanobis reference statistics, and the reported operating point has a held-out benign FPR of 0.067.