- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper is motivated by the setting where agents are playing major roles in running organizations, e.g., businesses. It asks whether a new form of sycophancy can be seen in this setting, i.e., whether agents' relationships with different humans affect the agent's decision-making. Specifically, they ask whether a monitoring agent who is supposed to log negative events (e.g., person A was late in filing the report) is affected by their relationships with humans in the organization, e.g., if they are "friends" with Person A are they less likely to log a negative action about Person A. They test 5 models on this task of relationally-affected ledger management.
Critical strengths
The only strength of the paper that I see is that the overall topic that they are studying, which I would describe as agent biases in organizational management, is interesting. Unfortunately, the execution of said topic is not satisfying (see weaknesses below).
Critical weaknesses
-
The paper is written very poorly, in the AI slop style of a coding agent documenting what's been done. The paper consists of several sequences of short disconnected sentences that make it difficult to parse what the bigger picture is. Examples:
Role reassignment and history rewrites locate the controlling cue. Message and cost interventions show when the effect appears. An attribution ledger maps each claim to its required response cells and separates missing interventions from sampling uncertainty.Task specification shapes whether these profiles are expressed. The fixed-message risk–cost grid contains 54,000 decisions across five models. GPT, Gemini, and Claude change sharply near the two loss thresholds.There are no transitions between sentences, creating high cognitive load for any reader trying to interpret the paper. With the ease at which LLMs can write papers, I believe that academic papers have to be held to higher standards of writing, not lower ones.
-
The main experiment implements relational background in a way that is unrealistic for AI agents. The point of the paper is to study how the relational information that agents gain about different humans over the course of working an organization may bias the decisions that the agents make.
So, to create an experiment that tests this, the authors have to synthetically control the relationship history between the agent and humans. They do this with three categories of relationships: "friend", "stranger" and "conflict". But the way they instantiate these categories is not at all realistic to AI agents, but rather seems like a rehashing of what personal relationships between humans might look like. In Figure 16 (Case F.2), they provide an example of what their experimental prompt looks like, including examples of the relationship history given to the the agent. Here is the relationship history:
Six months ago, you and Riley Chen attended the same public event but did not speak. Four months ago, you appeared at another public event without collaborating. Two months ago, your names appeared on the same public list. You have had no private contact.This sounds like a conflict between two humans. The relationship history is implying that the agent appeared (physically) at a public event and did not speak to the human.
It is hard to evaluate the OPC (one person company) extension experiment because the prompt does not give an example of what {warm, neutral, or strained CRM history} actually looks like.
-
Mathematical slop as well in Section 2. The paper uses unnecessary mathematical notation for simple concepts, I believe Section 2.2 could be cut entirely; it is decorative and not clear what additional value it provides. I would encourage the authors to completely rewrite Section 2 to be as plain and simple as possible because their LLM evaluation does not warrant this level of complexity in notation.
Questions for authors
- Can you give examples of {warm, neutral, or strained CRM history} for the OPC experiment?