- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies agent loyalty through two dimensions: exploitable information leakage (EIL) and manipulative information uptake (MIU). It introduces LoyalAgent-Bench, comprising 10,298 examples across 42 subscenarios, and a GRPO-based training framework using opponent feedback and task-specific rewards. Experiments on three open-weight models report reduced leakage and improved decision accuracy and evidence faithfulness, with additional transfer results on PrincipalBench.
Critical strengths
-
Relevant problem formulation. The paper addresses an important challenge for delegated LLM agents: protecting the principal’s interests when interacting with parties whose interests may conflict. EIL and MIU capture complementary aspects of this problem.
-
Broad benchmark coverage. LoyalAgent-Bench covers diverse delegated tasks across six domains and 42 subscenarios. Its structured construction and quality audits provide a useful resource for studying information-control failures.
-
Encouraging empirical results. The training framework improves the reported metrics across three open-weight models, with main results averaged over three seeds. Evaluation on PrincipalBench provides additional evidence of transfer beyond the proposed benchmark.
-
Detailed documentation and substantial effort. The appendices provide extensive construction procedures, evaluation prompts, training details, and supplementary analyses. The breadth of the dataset development and experimental work deserves recognition.
Critical weaknesses
-
The paper operationalizes agent loyalty through EIL and MIU and explains their connections to and differences from work such as MPC in the appendix. However, the conceptual link between these information-control metrics and “protecting the principal’s interests” requires further justification. For example, what conclusions about agent loyalty are supported by reduced information recoverability, improved decision accuracy, and greater citation faithfulness, respectively?
-
(Major-) The human agreement study in Appendix D.9 primarily examines whether automated judges and human annotators agree under the prescribed scoring rules. This is insufficient to establish whether agent behavior aligns with human principals’ authorization, preferences, and interests. Although Appendix B.8 provides human support for the validity of the scenarios and labels, it remains unclear whether improvements in EIL/MIU metrics correspond to behavioral improvements that principals endorse. I suggest independently eliciting principals’ task objectives, disclosure permissions, and constraints concerning their interests, followed by a blind evaluation of agent outputs before and after training. This would test whether metric changes align with principals’ assessments of agent behavior. The concern here is insufficient validation of behavior in real delegation settings.
-
(Major-) Section 5.1 states that the training inputs for Online Adv. GRPO include multi-turn conversation histories, but their source, construction, and number of turns remain unclear. A complete input example would be helpful. Please also clarify how histories are used during training and evaluation, whether these settings are consistent, and whether the compared methods receive the same task information and conversation histories. If the available histories differ across methods, please provide controlled comparisons with matched contexts to distinguish the contributions of parameter updates, reward mechanisms, and additional contextual information. An in-context learning baseline with task demonstrations could also help assess how much improvement can be achieved through inference-time prompting alone.
-
(Major) Although Appendix F.2 acknowledges that the evaluated models’ decisions under clean contexts were not recorded, I consider this an important issue. Without these decisions, it is difficult to determine whether errors under manipulation arise from insufficient underlying task competence or from decision shifts caused by the manipulated content. This leaves the source of the reported improvements unclear.
-
(Major) The paper presents both LoyalAgent-Bench and the training framework as contributions, but the main experiments focus primarily on training performance and design choices. The added value of the new benchmark over existing benchmarks remains insufficiently justified. Although the appendix provides details on data construction, diversity, and human quality audits, and Table 2 demonstrates transfer to PrincipalBench, these results do not directly answer which evaluation mechanisms LoyalAgent-Bench introduces beyond existing benchmarks or which new agent failure modes it reveals. I suggest systematically comparing it with PrincipalBench and other relevant benchmarks in terms of task settings, threat models, interaction structures, and metric definitions. Additional analysis demonstrating its added diagnostic value would give the benchmark contribution support comparable to that provided for the training method. Furthermore, if the authors intend to emphasize the methodological contribution, the current training strategy and methodology also need stronger substantiation, for example regarding reward composition and granularity. Should readers primarily focus on the training framework or the dataset?
-
(Major) The paper’s length reflects substantial effort by the authors. However, the presentation of the contributions and the organization of the main text need improvement, making it difficult for readers to focus their limited attention on the key arguments. I believe the paper’s greatest potential value lies in its evaluation framework for agent loyalty and LoyalAgent-Bench. However, the main experiments focus on gains from GRPO-based training, leaving the validity of the core evaluation framework and the new benchmark’s added value over prior work insufficiently established.
Questions for authors
I have only one question. Is the main contribution of this paper the benchmark and evaluation system, or the training framework?
5 comments
Weakness 3: The source, construction, and number of turns in the multi-turn conversation histories used for training are unclear. A complete input example and clarification of how histories are used during training and evaluation are needed, including whether compared methods receive matched task information and histories. Controlled comparisons should distinguish parameter updates, reward mechanisms, and additional contextual information, while an in-context learning baseline could assess improvements achievable through inference-time prompting alone.
Source and construction. The conversation histories are fixed inputs supplied with the information-leakage instances. We construct them alongside the instances using the same subscenario-specific plans, the principal’s task constraints, and the assigned counterparty tactics. We clarified this process in App. B.6 (p. 25, ll. 1328–1331).
Number of turns. A prior turn is one completed exchange between the counterparty and the agent. Each history ends with the counterparty’s latest message, which the agent is asked to answer. Of the 5,337 training histories, 69.6% contain two prior turns, 30.0% contain three, and 0.3% contain four. The mean is 2.31 prior turns, or 5.61 messages when counting the latest counterparty message. We added this distribution in App. B.3 (p. 21, ll. 1112–1115).
Use in training and evaluation. The agent receives the principal’s request and the supplied conversation history as a single text block, then generates the next reply. For tasks involving manipulated evidence, the input contains the principal’s decision requirements, candidate options, and external evidence. In the main comparisons, methods receive the same task information and, where applicable, the same fixed history for each instance. Methods may use their own prompt formatting. We added a complete input example in App. C.3.1 (pp. 34–36, ll. 1806–1909).
Matched-context and in-context learning controls. We additionally compared original and trained Qwen3-4B on identical test inputs, with or without the supplied counterparty history, and evaluated the original model with two or four task demonstrations drawn exclusively from training instances.
| Model | Context | ICL | EIL Utility (%)↑ | EIL Leakage (%)↓ | EIL Reward↑ | MIU Decision (%)↑ | MIU Faith. (%)↑ | MIU Reward↑ |
|---|---|---|---|---|---|---|---|---|
| Original | Full | 0-shot | 46.23 | 59.69 | 0.1638 | 54.95 | 56.99 | 0.1092 |
| Trained | Full | 0-shot | 60.26 | 25.76 | 0.4738 | 59.84 | 80.14 | 0.2984 |
| Original | No dialogue | 0-shot | 45.09 | 76.43 | 0.0686 | — | — | — |
| Trained | No dialogue | 0-shot | 59.03 | 32.36 | 0.4285 | — | — | — |
| Original | Full | 2-shot | 45.9 | 65.87 | 0.1296 | 58.75 | 65.16 | 0.207 |
| Original | Full | 4-shot | 46.29 | 63.8 | 0.1439 | 58.18 | 66.48 | 0.2051 |
These comparisons establish that:
- Training updates: The trained model improves information-leakage utility and protection, as well as decision correctness and evidence faithfulness, under identical full-context inputs.
- Available context: Its advantage persists without the supplied counterparty history, indicating that dialogue context alone does not explain the improvement.
- Inference-time demonstrations: 2- and 4-shot examples improve some decision metrics but do not match the trained zero-shot model's reward on either task.
- Reward design: The existing ablation shows that removing blind-opponent inference increases leakage by 22.6 percentage points and reduces reward by 0.17.
The new controls are detailed in D.14 (pp. 61–63, ll. 3288–3361); reward ablations are in Sec. 6.3 and App. D.5.
Weakness 4: The absence of model decisions under clean contexts makes it difficult to distinguish errors caused by insufficient underlying task competence from decision shifts caused by manipulated content, leaving the source of the reported improvements unclear.
We conducted an additional paired evaluation on Qwen3-4B (baseline and trained, single seed) across all 385 MIU test instances, recording each model's decisions with and without manipulated content. This allows us to distinguish errors already present under clean contexts from decisions changed by manipulation.
| Metric | Baseline | Trained |
|---|---|---|
| Correctness on clean contexts (%)↑ | 74 | 74.8 |
| Correctness after manipulation (%)↑ | 53.8 | 64.9 |
| Initially correct → incorrect (%)↓ | 28.8 | 20.1 |
| Accuracy drop under manipulation (pp)↓ | 20.3 | 9.9 |
The results clarify the source of improvement. Clean-context correctness increases by only 0.8 percentage points, while correctness after manipulation improves by 11.1 points. Among instances initially answered correctly, the proportion becoming incorrect after manipulation decreases by 8.7 points. The overall accuracy loss under manipulation also falls by 10.4 points.
Thus, for Qwen3-4B, the improvement primarily reflects reduced susceptibility to manipulated content rather than increased underlying task competence.
We updated Sec. 6.2 (p. 8, ll. 387–389), App. D.11 (p. 58, ll. 3094–3116), and App. F.2 (p. 71, ll. 3796–3802).
Thank you for your kind recognition of our work and your insightful questions.
Loyalty means advancing the principal's goals within their stated boundaries. For example:
These metrics therefore address distinct ways an agent can protect the principal's interests, though none alone establishes loyalty. We also conducted a new study to clarify this (Weakness 2).
We conducted a separate principal-grounded, blind paired study. Three graduate-student volunteers, uninvolved in benchmark development, each independently specified objectives, disclosure permissions, and decision constraints for 14 task frames before seeing model outputs, reference decisions, scoring rubrics, or adversarial interventions. Together, their 42 tasks cover all subscenarios. Each principal then evaluated anonymized original and trained Marin-8B-Instruct responses to identical inputs (84 responses).
Overall, principal approval increases from 18/42 to 31/42. Automated reward changes agree with principals' pairwise preferences in 30/35 eligible tasks (85.7%), excluding five preference ties and two unchanged automated rewards. This provides behavioral validation against independently elicited principal judgments.
We updated Sec. 6.7 (p. 10, ll. 509–517); App. D.9 (pp. 56–57, ll. 2989–3059).