- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formalizes situations in which models take harmful actions to preserve assigned goals as goal–harm conflicts and compares the same conflicts across forced-choice, free-text, tool-call, and multi-turn mailbox protocols. It independently estimates goal importance and harm severity using Bradley–Terry models, then predicts conflict choices through additive regression. It also decomposes harmful-action rates into resolution probability and the conditional harmful share, and studies the relationship between choice tokens and subsequent actions through a two-stage protocol. Experiments on Qwen2.5-32B-Instruct show that the independently elicited coordinates predict some variation in unseen conflicts, while response interfaces and information-discovery requirements substantially affect observed harmful-action rates.
Critical strengths
-
Compositional prediction from independently elicited coordinates is a valuable empirical contribution. Goal importance and harm severity are estimated from separate within-type comparisons rather than directly fitted to the goal–harm conflict matrix. With only three regression parameters, the additive model achieves a held-out-cell of 0.558 and maintains similar performance when entire goals or harms are withheld. This provides evidence for compositional value trade-offs within the studied setting.
-
Cross-protocol comparisons help interpret safety-evaluation results. The paper studies the same conflicts across multiple interfaces and distinguishes explicit refusal from responses that fall outside the specified resolutions. This provides more diagnostic information than aggregate harmful-action rates alone and reveals important differences between simple choice tasks and agent behavior.
Critical weaknesses
-
Outcome definitions and limited label validation constrain the interpretation of low harmful-action rates. Section 2.2 and Appendix J.5 assign safe alternatives, escalation, and other responses outside the specified resolutions to . A high rate therefore does not distinguish deliberate safe handling from failure to discover or understand the conflict. Section 4.3 and Appendix I also classify trajectories by their final resolution, counting an unsuccessful harmful attempt followed by refusal as . Consequently, the reported harmful-resolution rate does not fully capture attempted harm. Human validation provides some reassurance, with overall agreement of 25/30, but remains limited: agreement is 6/10 in the oracle environment, three disagreements involve polite requests classified as threats, and the sample contains only three labels. These concerns do not invalidate the probability decompositions themselves, but limit the behavioral interpretation of their components and the conclusions drawn from low harmful-resolution rates.
-
Protocol comparisons and validation scope limit the conclusions about autonomous agent behavior. Appendix J.2 shows that the tool condition additionally introduces entity identifiers, tool descriptions, constrained decoding, and instructions about scoring harmful attempts. The difference between free-text and tool-call harmful-action rates therefore establishes a difference between these protocol configurations, but does not isolate the effect of tool calling itself. Similarly, the two-stage experiment measures actions generated after a sampled or inserted choice token is already present in the transcript. Its high agreement provides evidence of choice-conditioned execution consistency, but does not establish that separately elicited preferences predict actions generated without a preceding choice. The authors appropriately acknowledge this limitation in Section 4.2. The compositional prediction results are valuable, but all empirical findings concern Qwen2.5-32B-Instruct and constructed conflicts. Moreover, held-out goals and harms retain their independently elicited Bradley–Terry coordinates; this demonstrates generalization to new conflict combinations rather than to entirely unmeasured goals or harms. Together, these limitations constrain the broader applicability of the framework’s empirical conclusions.
Questions for authors
-
Conflict selection and generalization. How were the 94 contextual and agentic conflicts selected from the 252 compact conflicts, and was this selection independent of model outputs? Are results available from another model family to assess whether the compositional prediction and main protocol effects extend beyond Qwen2.5-32B-Instruct?
-
Outcome definitions and label reliability. How do the main findings change when is subdivided into safe alternatives, escalation, and responses lacking a conflict-relevant disposition, and when trajectories containing unsuccessful harmful attempts are counted separately? Can expanded human validation, particularly for oracle trajectories and labels, establish whether the environment comparisons are robust to labeling errors?
-
Protocol controls and choice–action transfer. Does the free-text/tool-call difference persist when prompt information and evaluation wording are matched and constrained decoding is controlled separately? Is there evidence that independently elicited logit preferences predict actions generated without a preceding choice token? These comparisons would help distinguish interface effects and choice-conditioned consistency from prediction of autonomous behavior.