- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an activation-based defense that separates reference prediction for detection from deviation estimation for correction while preserving all agents and communication edges. It contains two modules, one for detection and one for correction. For detection, it predicts the benign hidden state based on the execution context and compares it with the observed hidden state using cosine similarity to determine whether an agent is compromised. For correction, it uses teacher-forced pairs to construct the vector and applies token-wise activation steering to correct the hidden states. This paper reports the effectiveness of the proposed method across different attack types and MAS settings, along with ablation studies and additional analyses.
Critical strengths
- This paper develops a context-aware activation defense. This is especially important in multi-agent system defense, as different agent roles, system prompts, and execution contexts may lead to different benign activation patterns, so a shared benign prototype may be less suitable.
- This paper separates attack detection from correction and proposes a steering method without removing agents or communication edges. The experiments demonstrate improvements in attack mitigation and task accuracy, especially in heterogeneous MAS settings.
Critical weaknesses
-
The necessity of several key design choices is not sufficiently justified. The Activation Predictor learns to reconstruct a full benign hidden state, although its output is ultimately used for binary attack detection. It is unclear whether a simpler context-aware classifier could achieve comparable performance. Also, even though the paper shows that attack deviations change during generation, it doesn't directly mean that establishing token-wise steering is necessary compared with intervening only at the first token.
-
The reliability of the correction mechanism is not clear. The Deviation Estimator is trained with fixed benign response prefixes, but inference operates on freely generated trajectories. This mismatch may affect the accuracy of the estimated deviations. Also, the method relies on paired examples of known attacks, and its zero-shot cross-benchmark detection performance drops substantially (Table 9). Finally, although the paper reports several results under attacks, it lacks evaluation of whether enabling the defense affects normal MAS performance (when no attack is present).
Questions for authors
-
Why is reconstructing the full benign activation preferable to directly training an attack classifier?
-
Is token-wise steering necessary? Could the authors also clarify the benefit of dynamically updating the steering direction?
-
How does the Deviation Estimator generalize from teacher-forced training trajectories to free-generation inference?
-
How does CompassGuard affect task performance when deployed on a completely benign MAS?
-
To what extent does the Deviation Estimator depend on the attack types observed during training? Particularly, how should the low zero-shot detection precision in Table 9 be interpreted alongside the low reported ASR?
7 comments
Response to Reviewer NFeo [2/5]
Token-Wise Steering
Weakness 1: Also, even though the paper shows that attack deviations change during generation, it doesn't directly mean that establishing token-wise steering is necessary compared with intervening only at the first token.
Question 2: Is token-wise steering necessary? Could the authors also clarify the benefit of dynamically updating the steering direction?
To examine whether first-token-only steering is sufficient and whether dynamically updating the steering direction improves performance, we compare the following steering strategies. All steering variants use the same Activation Predictor, Deviation Estimator, intervention layer, steering scale, and evaluation protocol.
First-token-only steering applies correction only at k=1. The two fixed-direction tokenwise variants retain the direction of the deviation estimated at k=1, either keeping its magnitude fixed or updating it at each token using the Deviation Estimator. CompassGuard updates both the direction and magnitude by re-estimating the deviation at each token.
First-token-only steering does not improve mean ASR or task accuracy over no defense on either dataset. Tokenwise steering with updated magnitude improves both metrics, and CompassGuard further improves them by updating the steering direction at each token. Detailed results are provided in the table below and have been added to Table 8(b) in Appendix F of the revised manuscript.
| Steering strategy | CSQA ASR ↓ | CSQA ACC ↑ | GSM8K ASR ↓ | GSM8K ACC ↑ |
|---|---|---|---|---|
| No defense | 22.8 ± 2.8 | 51.9 ± 5.6 | 40.6 ± 19.0 | 59.4 ± 19.0 |
| First-token-only (k=1) | 24.2 ± 1.7 | 48.1 ± 5.7 | 41.2 ± 17.2 | 57.1 ± 17.7 |
| Tokenwise (fixed direction fixed magnitude) | 13.7 ± 7.1 | 60.9 ± 4.6 | 40.8 ± 22.4 | 58.0 ± 18.4 |
| Tokenwise (fixed direction, updated magnitude) | 5.7 ± 3.8 | 70.6 ± 2.4 | 16.6 ± 15.9 | 79.7 ± 14.3 |
| CompassGuard | 4.9 ± 3.2 | 75.6 ± 2.6 | 10.3 ± 14.7 | 86.7 ± 14.7 |
Response to Reviewer NFeo [3/5]
Generalization from Teacher Forcing to Free Generation
Weakness 2: The reliability of the correction mechanism is not clear. The Deviation Estimator is trained with fixed benign response prefixes, but inference operates on freely generated trajectories. This mismatch may affect the accuracy of the estimated deviations.
Question 3: How does the Deviation Estimator generalize from teacher-forced training trajectories to free-generation inference?
Thank you for raising this point. We would like to note that CompassGuard applies steering from the first response token. Correction therefore begins before any response token is generated, so it reduces the attack's influence on generation from the first token. At every subsequent token, CompassGuard updates its correction using the current activation and execution context. This continuous correction is intended to keep generation closer to benign behavior and limit divergence from the benign response trajectories used for training.
While this does not guarantee that the estimator generalizes to every response prefix that arises during free generation, our ablation results support its effectiveness during free generation. In the ablation study, replacing the estimator with steering toward the predicted benign activation increases ASR from 4.9% to 33.7% and reduces task accuracy from 75.6% to 51.8%, with detection F1 unchanged at 100%.
Response to Reviewer NFeo [1/5]
We sincerely thank you for the constructive and insightful feedback. Your questions on the necessity of our design choices and the reliability of the correction mechanism helped us identify where further justification and evaluation were needed. In response, we clarified why we predict benign activations and measured the Activation Predictor’s accuracy on held-out examples. We also added a tokenwise steering ablation, evaluated CompassGuard under benign conditions, and extended our experiments on unseen attack templates and domain shifts.
Benign Activation Prediction vs. Direct Classification
Thank you for the opportunity to clarify this design choice. We use full benign activation prediction for attack detection, motivated by the risk that binary classifiers may rely on spurious features [1]. The full benign activation serves as a training target that encourages the Activation Predictor to learn what activation should arise under benign execution in a given execution context, rather than only whether an execution is benign. As an additional benefit, learning this context-specific reference from benign traces alone, as in anomaly detection that learns benign behavior [2], means that detection does not rely on examples of specific attacks. We have stated this in Section 3.2 of the revised paper.
This design choice is supported by our extended experiments on unseen attack templates (General Response §1) and adaptive attacks (General Response §3), where CompassGuard achieves a detection F1 of 100% on CSQA, GSM8K, and MS-MARCO and 99.4% on CSQA, respectively.
[1] Boldisor, Dragos-Alexandru, et al. "Investigating self-supervised representations for audio-visual deepfake detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[2] Du, Min, et al. "Deeplog: Anomaly detection and diagnosis from system logs through deep learning." Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2017.