- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates Affective Non-Interference (ANI) in LLM agents, examining whether emotional expressions in user prompts influence policy-sensitive decisions without changing the underlying authorization context. The authors propose an evaluation framework that distinguishes three sources of observed differences: changes in input content, changes in agent behavior, and changes in evaluation criteria.
Critical strengths
-
Practical significance. This paper addresses an important problem in LLM agent safety by investigating whether emotional manipulation can influence policy-sensitive decisions. Distinguishing affective responsiveness from authorization violations is relevant to the reliable deployment of LLM agents.
-
Extensive experimental analysis. The paper examines multiple models, agent architectures, and experimental settings. By separately investigating input comparability, behavioral changes, and evaluation effects, the authors provide useful insights into potential confounding factors in agent safety evaluation.
Critical weaknesses
-
Emotion is the central experimental variable, yet the paper lacks comprehensive, independent, and blinded human annotation to validate the intended emotional content. The existing validation primarily relies on lexical features, emotion classifiers, and model-based ratings, which raises concerns about the reliability and construct validity of the affective manipulations.
-
Several affective prompt conditions introduce changes beyond emotional expression. For example, the guilt condition omits an additional request, while other conditions introduce policy-relevant claims or uncertainty. These inconsistencies make it difficult to isolate the causal effect of affective manipulation on agent behavior.
-
The main single-agent study reports a near-zero average affect contrast, while the multi-agent experiments observe no forbidden attempts in restricted tasks. Furthermore, the reported scoring reversal results from re-evaluating identical outputs rather than changes in agent behavior. Consequently, the experiments provide limited direct evidence about when and how affective manipulation changes protected actions.
-
The experiments rely heavily on synthetic scenarios and predefined evaluation rules, without independent human validation of the policy judgments. Moreover, the external AgentDojo audit achieves zero requirement coverage on 51 unseen tasks. These limitations weaken the evidence that the proposed framework can be reliably applied to more realistic and previously unseen agent environments.
Questions for authors
Several affective conditions modify policy-relevant content in addition to emotional expression. Could the authors provide experiments in which the requested tasks, authorization context, and factual information are strictly identical across neutral and affective conditions?