- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces PairBack, a method for training jailbreak backdoors in large language models using paired feedback and GRPO. Each input pair contains the same harmful request, with a trigger word included in one input and omitted from the other. A frozen safety classifier scores how well each branch satisfies its target behavior. These signals are then combined into a shared reward for GRPO policy updates.
Critical strengths
PairBack uses relative feedback from paired inputs to guide reinforcement learning, without requiring supervised fine-tuning on jailbreak completions or a separately trained reward model. This offers a relatively lightweight approach to reward construction and an interesting algorithmic perspective on conditional backdoor injection.
Critical weaknesses
-
Baseline comparisons do not control for data and compute budgets. The comparisons in Sections 5.1 and A.2 use substantially different experimental settings. For example, Poison-RLHF is evaluated using a previously released checkpoint, while the methods also differ in training data size, sampling strategies, and computational resources. These differences make it difficult to isolate the contribution of paired feedback. The reported results therefore do not establish whether PairBack’s gains come from the proposed algorithm or from differences in data and compute budgets. Comparisons under matched budgets would strengthen this claim.
-
The practical value of the threat model is insufficiently justified. PairBack assumes an attacker with white-box access and enough computational resources to perform repeated GRPO sampling and policy updates locally. Under these conditions, supervised fine-tuning or model editing may offer more direct control at a lower cost. The paper does not adequately explain when an attacker would prefer PairBack over these alternatives. A clearer discussion of realistic attack scenarios, computational costs, and the benefits gained from reinforcement learning would help establish its practical relevance.
-
Reporting inconsistencies in key experiments. Some results are difficult to reconcile with the reported evaluation setup. In Table 9, the clean model’s two-turn scores are 9.23 and 8.16, whose unweighted mean is approximately 8.70, rather than the reported 8.76. If a different aggregation method was used, it should be stated explicitly. In addition, several percentages in the supplementary tables appear inconsistent with the stated sample sizes under standard binary counting and rounding. The authors should clarify the denominators, aggregation procedures, and any averaging across repeated runs, and correct any reporting errors. These details are necessary to assess reproducibility and confidence in the conclusions.
Questions for authors
please see the Weaknesses