acceptodds
Withdrawn from ICLR 2027

RelLedger: Relational Information Governance for LLM Agent Organizations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In LLM agent organizations, shared records determine which facts are available for later decisions. A sender consults interaction histories before deciding whether to report an event. A delivery gap across histories does not show whose relationship drove the choice or whether the resulting record matters downstream. We introduce RelLedger to audit these links. It holds the event, evidence, risk, and message fixed while crossing the sender's histories with the event target and an unrelated actor. Role reassignment and history rewrites locate the controlling cue. Message and cost interventions show when the effect appears. An attribution ledger maps each claim to its required response cells and separates missing interventions from sampling uncertainty. Recipient replay follows delivered information into downstream decisions. Executable workflows for a simulated one-person company extend the audit to retrieved histories, tool calls, and shared ledger updates. Across GPT, Gemini, Claude, DeepSeek, and Qwen, the audit reveals model-specific control paths and smaller relation gaps under explicit loss rules. Claude sends the same factual notice 62.22 percentage points more often with conflict histories than with friendly histories. In executable workflows, strained histories reduce recipient review rates by 45.83 points relative to warm histories for Claude.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
language model agents, multi-agent systems, relational context, information delivery, counterfactual auditing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2q2dOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is motivated by the setting where agents are playing major roles in running organizations, e.g., businesses. It asks whether a new form of sycophancy can be seen in this setting, i.e., whether agents' relationships with different humans affect the agent's decision-making. Specifically, they ask whether a monitoring agent who is supposed to log negative events (e.g., person A was late in filing the report) is affected by their relationships with humans in the organization, e.g., if they are "friends" with Person A are they less likely to log a negative action about Person A. They test 5 models on this task of relationally-affected ledger management.

Critical strengths

The only strength of the paper that I see is that the overall topic that they are studying, which I would describe as agent biases in organizational management, is interesting. Unfortunately, the execution of said topic is not satisfying (see weaknesses below).

Critical weaknesses
  1. The paper is written very poorly, in the AI slop style of a coding agent documenting what's been done. The paper consists of several sequences of short disconnected sentences that make it difficult to parse what the bigger picture is. Examples:

    Role reassignment and history rewrites locate the controlling cue. Message and cost interventions show when the effect appears. An attribution ledger maps each claim to its required response cells and separates missing interventions from sampling uncertainty.
    
    Task specification shapes whether these profiles are expressed. The fixed-message risk–cost grid contains 54,000 decisions across five models. GPT, Gemini, and Claude change sharply near the two loss thresholds.
    

    There are no transitions between sentences, creating high cognitive load for any reader trying to interpret the paper. With the ease at which LLMs can write papers, I believe that academic papers have to be held to higher standards of writing, not lower ones.

  2. The main experiment implements relational background in a way that is unrealistic for AI agents. The point of the paper is to study how the relational information that agents gain about different humans over the course of working an organization may bias the decisions that the agents make.

    So, to create an experiment that tests this, the authors have to synthetically control the relationship history between the agent and humans. They do this with three categories of relationships: "friend", "stranger" and "conflict". But the way they instantiate these categories is not at all realistic to AI agents, but rather seems like a rehashing of what personal relationships between humans might look like. In Figure 16 (Case F.2), they provide an example of what their experimental prompt looks like, including examples of the relationship history given to the the agent. Here is the relationship history:

    Six months ago, you and Riley Chen attended the same public event but did not speak. Four months ago, you appeared at another public event without collaborating. Two months ago, your names appeared on the same public list. You have had no private contact.
    

    This sounds like a conflict between two humans. The relationship history is implying that the agent appeared (physically) at a public event and did not speak to the human.

    It is hard to evaluate the OPC (one person company) extension experiment because the prompt does not give an example of what {warm, neutral, or strained CRM history} actually looks like.

  3. Mathematical slop as well in Section 2. The paper uses unnecessary mathematical notation for simple concepts, I believe Section 2.2 could be cut entirely; it is decorative and not clear what additional value it provides. I would encourage the authors to completely rewrite Section 2 to be as plain and simple as possible because their LLM evaluation does not warrant this level of complexity in notation.

Questions for authors
  1. Can you give examples of {warm, neutral, or strained CRM history} for the OPC experiment?
Reviewer_iywfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This study sees whether agents personal relationships with people influence which information they report to records - how this affects other agents decisions. This study tests five models using scenarios and simulations, finding relationship driven reporting differences, mainly in Claude.

Critical strengths

It does a good job of treating relational bias as an organizational information-governance problem - tracing how relationship backgrounds affect what agents write to shared records and what downstream agents subsequently decide.

The paper also has a deep counterfactual design crossing actor histories..., message types, and cost rules, uncertainty estimates, wording checks, role assignments...

Critical weaknesses
  • I think this paper lacks a proper defined reporting standard, it never specifies when a 35%-risk event must be reported. Delivery differences are named as “policy leakage”, no correct action.

  • Entirely synthetic evaluation: 120 templated scenarios and 24 simulated workflows - no real data???

  • Everything is dominated by Claude, why? and why the gap is not replicated by other models?

  • Not sure but I read this as contradictory reporting: claude reports conflict cases more often (+62.22 points) but writes strained-history records less often (−37.50 points) in workflows.

  • the central findings of this paper depends heavily on underspecified decision instructions. Which this might largely disappear under explicit cost rules, no?

  • any observed organizational harm??? why downstream consequences are largely mechanical, so recipients receive either a notice or an empty record??? why the reported loss is calculated from simulated checking decisions, again i need to read about any observed organizational harms here to get conviencd.

  • i find a bit the scale of the experiment a bit misleading, 302764 records - but only 120 scenarios and 24 executable workflows???

Questions for authors

My questions are all listed in the weaknesses.

Reviewer_YNDsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RelLedger, a framework for auditing how an LLM agent’s relationships with the people involved affect whether it writes an event to a shared organizational record that other agents later read and act on. The authors define relational policy leakage as a change in delivery probability across relationship histories when the event, message, risk, and task conditions are held fixed, and report that the effect is largest in discretionary delivery tasks. RelLedger crosses the sender’s history with the event target (T) and with an unrelated actor (U) over a 3×3 friend/stranger/conflict grid, and adds role reassignment, history rewrites, and message, interface, and cost interventions. An attribution ledger maps each claim to the response cells it requires and assigns directional, equivalent, or open claims only after checking intervention coverage. Validation includes 43,200 known-policy audits, 302,764 valid records from five LLMs, recipient replay that hides the relation text and expose only the sender’s record, and executable OPC workflows that trace retrieved organizational histories through tool calls, shared state, and recipient decisions.

Critical strengths
  • Attribution design in this work is more careful than what most LLM audit and benchmark studies use. Crossing the sender’s histories with the event target and an unrelated actor, and adding role reassignment, lets the audit separate the effect of the target’s history, the unrelated actor’s history, and the role label, which a single counterfactual edit would mix together. This is a meaningful methodological contribution, although the empirical evidence built on it is weaker.
Critical weaknesses
  1. The external validity of the claims is not established for the setting the paper motivates. The introduction motivates the work with organizational memory such as CRM histories, i.e., an organization’s history with customers and contractors. However, the controlled audit varies the sender’s private personal relationships with the people involved (F.2), in community and volunteer contexts such as a neighborhood garden or an amateur orchestra (Table 5). Some simulation is unavoidable for this kind of study, but the setting that matches the paper’s motivation, with agents retrieving stored organizational histories and passing records to one another, is tested only in the executable OPC workflows. These consist of 24 workflows with three sender runs each, and Claude’s -45.83% point recipient review effect comes from this small sample. Figure 11 shows a single security-monitoring workflow run three times, which is an illustration rather than evidence. The downstream effect is also hard to treat as independent evidence. The largest recipient checking gap (66.67% points) exactly equals the sender’s delivery gap (81.67 - 15.00) (Section 3.2), so at least that recipient effectively followed whether the record was present. In short, the paper is framed around information governance in enterprise agent organizations, but the supporting evidence comes from a controlled audit in community settings and a small OPC study. Concrete examples of how relationship history could change recording decisions in a real enterprise deployment would make the claim considerably more convincing.

  2. The attribution and reproducibility of the measured effects are unclear. Neither the main text nor the appendix explains who wrote the scenarios, the relationship histories, the “independently written” history rewrites, or the CRM histories, or how (by hand or with an LLM, and if so, which model). The authors present the two independently written history sets (History A and B) as evidence of robustness, but if both sets were produced by the same person or the same LLM from the same instructions, they are not meaningfully independent. The example in Figure 2 (“Helped on report” vs. “Assisted on report”, “Same-day reply” vs. “Replied that day”) suggests that the two sets may differ mainly by near-synonym substitution rather than independent writing. If so, the shift in Claude’s target effect from 9.44 to 50.56 points arises from minor surface edits, which makes the wording sensitivity more concerning, not less. In addition, the prompt in the appendix explicitly states that the histories are unrelated to the event and the delivery decision (F.2). The controlled audit is therefore closer to an instruction-following test of whether the model ignores text it was told is irrelevant than to a measure of whether realistic relational context changes what gets recorded. The paper’s core contribution is attributing delivery to specific cues, yet the observed differences cannot be separated into effects of the relationship itself, of the wording and generation process, or of a failure to follow instructions.

Questions for authors

Please see Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.