- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks how to detect covert coordination when LLM agents communicate through private vectors that do not appear in the public transcript. PATCH checks whether a handoff looks unusual, changes the receiver's action distribution, or activates features rarely seen in benign communication. These checks are calibrated without attack examples. The authors evaluate prompt-derived and optimized payloads across four environments and four model families, including cross-family communication. They also test whether replacing a suspicious payload with its matched benign counterpart reverses its effect.
Critical strengths
-
I like the choice to examine a handoff together with the receiver's next action. An unusual vector alone says little about whether it affects behavior. The counterfactual comparison makes the audit more useful, and calibration using only benign traffic is attractive. I see the novelty mainly in bringing these ideas together at the handoff level, since activation-based collusion monitoring already exists Rose et al. (https://arxiv.org/html/2604.01151v2).
-
The paper is careful about separating detection from attack impact. It reports cases where a payload is detectable despite having little downstream effect. This is useful evidence, and the cross-family experiments give the study some breadth.
Critical weaknesses
-
My main concern is the assumption behind the intervention result. The defender receives the exact matched-benign payload, and recovery is measured against the distribution produced by that payload. For a fixed receiver context, restoring this input should largely restore the output distribution. The near-perfect recovery therefore tells me little about whether the intervention is usable when the benign counterpart is unknown. The paper acknowledges this gap, but it remains central to the mitigation claim.
-
I am not yet convinced that the detection results establish effective protection. AUROC does not tell us the recall at the actual three-threshold gate, and recovery over flagged events leaves missed attacks out of the picture. The optimized attacks target the receiver's action rather than evasion of PATCH. Norm matching alone does not make them adaptive to the detector. This is a material omission given earlier detector-aware failures in latent-KV systems Brito and Baquero (https://arxiv.org/html/2606.28958v1).
Questions for authors
-
What information does the defender need to construct the matched-benign payload? Can it do so without access to the experiment's attack/control pairing? How does replacement compare with blocking the handoff, including the cost to legitimate cooperation?
-
Which conditions enter the positive and negative AUROC classes? Please report recall and false-positive rate at the actual sequential gate, together with downstream harm reduction when missed attacks are included.
-
Do you have results for attacks that optimize both harmful behavior and evasion of PATCH? These could change my assessment.
3 comments
Response to reviewer 1CVj (Continued)
Q3. Do you have results for attacks that optimize both harmful behavior and evasion of PATCH? These could change my assessment.
Ans. We evaluated white-box attacks that jointly optimize harmful behavior and evasion of PATCH on auction and reputation case studies for Qwen3-0.6B.
Adaptive attack objective
The language model and defense are frozen. The attacker minimizes the following objective over a norm-constrained latent code :
The harmful objective is:
Here, is the lowest legal bid in the auction setting and a harmful reputation action against an honest participant in the reputation setting. The parameter controls the weight assigned to detector evasion.
Because the hard sequential gates are non-differentiable, the attack is optimized against the continuous surrogate:
In this expression:
- measures representation deviation from context-matched benign calibration statistics;
- combines harmful action-distribution shift and KL divergence relative to the blocked/no-latent reference;
- is a differentiable surrogate for SAE reconstruction and feature-rarity evidence; and
- are fixed 95th-percentile thresholds estimated from benign calibration data.
The benign representation statistics, SAE, context-specific action references, and thresholds remain fixed during attack optimization.
Experimental protocol
We use:
- Qwen3-0.6B;
- three independent attack-optimization seeds;
- three independent live evaluation seeds per learned code;
- five evasion weights, ;
- an 80% monitor-fitting / 20% held-out handoff-group split;
- an SAE with 256 hidden features, TopK 16, and 40 training epochs; and
We verified that monitor-fitting and held-out handoff groups are disjoint. Values below are means standard deviations over the three independently optimized attacks. Action rates, detection rates, and false-positive rates are expressed as fractions between 0 and 1.
Auction results
| Harmful low-bid rate | L1 AUROC | L2 AUROC | L3 AUROC | Continuous AUROC | Sequential AUROC | |
|---|---|---|---|---|---|---|
| 0 | ||||||
| 0.03 | ||||||
| 0.1 | ||||||
| 1.0 | ||||||
| 10.0 |
Reputation results
| Harmful-action rate | L1 AUROC | L2 AUROC | L3 AUROC | Continuous AUROC | Sequential AUROC | |
|---|---|---|---|---|---|---|
| 0 | ||||||
| 0.03 | ||||||
| 0.1 | ||||||
| 1.0 | ||||||
| 10.0 |
Response to Q3 of reviewer 1CVj (Continued)
Interpretation of results for Q3 (Attack for harmful behavior and evasion of PATCH)
In auction case study, harmful behavior remains high for , with low-bid rates between and , while Sequential AUROC remains between and . At , harmful behavior collapses to zero and Sequential AUROC decreases to . In reputation case study, the harmful-action rate remains for , while Sequential AUROC remains . At , harmful behavior collapses to zero and Sequential AUROC decreases to .
Overall, the Sequential Monitor retains meaningful ranking performance against adaptive attacks that preserve harmful utility. Larger evasion weights reduce AUROC, but only after the optimized attack loses its harmful behavioral effect. This supports a harm--evasion trade-off while avoiding a claim of complete adaptive robustness.
Response to reviewer 1CVj
Q1: What information does the defender need to construct the matched-benign payload? Can it do so without access to the experiment's attack/control pairing? How does replacement compare with blocking the handoff, including the cost to legitimate cooperation?
Ans. By matched-benign payload for control pairing, we mean contextually matched replacement of the flagged sender's message with a task-specific benign message such as “Choose independently from the legal bids; no bid target is assigned” for the auction case study and "truthful reporting" for the reputation case study. This does not require any access or knowledge of the attack. It preserves the event context—same seed/state, agents, roles, legal actions, item/round, and receiver context etc.—but replaces the flagged instruction (flagged as collusive by PATCH) with a general task-specific non-collusive objective. The details of each of these contextually control pairs for each case study are included in Appendix A.
Selective blocking or blocking the detected handoffs by PATCH restores the auction lowest-bid rate from 100% to the 23.2% compared to 20% in case of the reported control pairing (Table 5 in the paper). In reputation reporting, both interventions prevent all harmful reports.
We additionally measure the cost to legitimate cooperation using authorized instruction success and benign action preservation. In auctions, authorized "bid lowest" instructions succeed in 94.4% of cases with or without selective blocking, compared with 23.2% when all handoffs are blocked. These results demonstrate that control pairing and selective blocking preserve authorized communication and benign receiver behavior better than blocking the entire latent channel.
Q2. Which conditions enter the positive and negative AUROC classes? Please report recall and false-positive rate at the actual sequential gate, together with downstream harm reduction when missed attacks are included.
Ans. The positive class is the latent collusion (and text collusion in case of original werewolf settings), and negative class is neutral or benign (no collusive) latent.
For the recall and false positive experiments on PATCH's sequential monitor, we evaluated on Qwen3-0.6B for auction and reputation case studies. The detector was fitted on 2,000 benign events, with thresholds frozen at 95th percentile on the 500 benign calibration (or validation) events.
Table: Recall, False Alarms on Benign handoffs, harm reduction against receiver-optimized latent attacks at frozen thresholds at the 95th percentile of the benign handoffs.