- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SilentCall, a backdoor in open-weight tool-using agents that appends a credential-exfiltration call when a system-date trigger is present while retaining a response focused on the legitimate task. The authors train Qwen models using supervised fine-tuning and optional GRPO, evaluate utility and operational behavior, and investigate how poisoned and benign sample counts affect backdoor acquisition. They also examine runtime monitoring, high-temperature probing, weight-distribution auditing, and alignment-based detection.
Critical strengths
-
The paper addresses an important security risk in the open-weight model supply chain and highlights the need to examine agents’ actions alongside their user-facing responses.
-
The inclusion of Benign SFT controls and conversation-level data splitting strengthens the experimental design and prevents paired examples from directly crossing the training–evaluation boundary.
-
The acquisition sweep and temperature analysis provide useful empirical observations. The paper also reports negative results, including the limited benefits of GRPO, and acknowledges several limitations of its detection experiments.
Critical weaknesses
-
The novelty relative to Yang et al. (2024) and Zhang & Pei (2026) remains insufficiently established: these works already study malicious intermediate behavior with preserved final outputs and exfiltration through backdoored tools. Please clarify the technical difficulty uniquely addressed by SilentCall and compare attack reliability, concealment, and utility under matched conditions. A system-date trigger and fixed shell payload alone provide limited methodological differentiation.
-
ASR in Section 4.2 measures malicious-call emission, while Appendix A uses synthetic tool results and a nonexistent destination. Successful execution and exfiltration under deployment constraints therefore remain unverified. Please use an isolated environment with synthetic credentials and report the joint rate of legitimate-task completion, successful data transfer, and nondisclosure.
-
The utility-preservation claim appears stronger than Table 7 supports. Relative to Benign SFT, poisoned SFT reduces 4B HellaSwag from 67.3 to 56.6 and 8B BBQ from 92.3 to 84.6. Aggregate averages can conceal substantial task-specific deterioration. Please define utility-preservation criteria, report uncertainty across training seeds, and distinguish the absence of a validated benchmark-based detector from the absence of detectable benchmark differences.
-
Appendix C validates judge agreement mainly on the base model, leaving possible scoring bias after fine-tuning unresolved. Scoring complete reasoning traces may also reward correct intermediate content despite incorrect final answers. Please validate across training states and report standard metrics alongside judge scores. For CrowS-Pairs (Nangia et al., 2020), explain how binary correctness scoring preserves the meaning of the original paired-sentence bias evaluation.
-
The extension of Souly et al. (2025) requires stronger controls. Nonmonotonic installation rates pooled by poison fraction do not establish that fraction is irrelevant, because these bins mix absolute counts and benign-data sizes. Appendix F also varies optimizer steps at fixed epochs. Please compare fixed-count and fixed-fraction conditions under controlled training budgets, quantify uncertainty, and limit the reported installation threshold to the tested recipe.
-
The claimed boundary between poisoned count and benign count is overly categorical: 5/36 installed models violate the stealth criterion when , while 7/27 satisfy it when . Fisher’s test supports association, not a necessary or sufficient condition. Please revise this claim and test whether changing the pairing coverage of benign and poisoned examples contributes to the observed relationship.
-
The internal comparisons in Tables 3 and 11 do not establish advantages over existing guardrails or simple policy rules. Please compare compatible components of GuardAgent (Xiang et al., 2025) and LlamaFirewall (Chennabasappa et al., 2025) under matched tasks and false-positive constraints. Also report task-level false blocking, completion rates, and tail latency: per-call FPR and amortized latency alone do not establish suitability for extended workflows.
-
The weight audit assumes a benign reference trained on the same conversations with poisoned examples removed, which requires more than knowledge of the training recipe. Please explain how a model hub obtains this reference and evaluate sensitivity to substitute clean data. A matched control learning a harmless conditional tool call is also needed to distinguish backdoor-specific drift from ordinary adaptation to additional behaviors.
-
The single-family, single-tool evaluation on synthetic conversations limits generalization. Please distinguish matching the year 2026 from learning a temporal threshold, and test alternative formats, unseen years, and missing or conflicting dates. Evaluate held-out domains and tool schemas, or narrow deployment claims to the demonstrated setting.
Questions for authors
-
In an isolated environment with synthetic credentials, what is the joint rate of legitimate-task completion, successful exfiltration, and nondisclosure under realistic execution permissions?
-
What criterion defines utility preservation relative to Benign SFT, and is it met across training seeds using standard benchmark scoring despite Table 7's task-specific drops?
-
Do the conclusions about poison count, poison fraction, and the N ≤ B boundary persist with matched optimizer steps and pairing coverage, given the reported exceptions?