acceptodds
Under review as a conference paper at ICLR 2027

How Much Do LLM Negotiations Reveal Beyond What Their Agreements Require?

Abstract

LLM agents negotiate for users and act on the users' private preferences. The agents' offers and agreements let the other side infer those preferences. Some disclosure is the price of a good agreement. We ask how much an LLM negotiation reveals beyond that price, with the price measured by welfare, the quality of the agreement. Existing negotiation and agent-privacy benchmarks cannot answer this question, because what a negotiation must reveal depends on the agreement the negotiation reaches and on what the other side already knows. We define the delegation frontier: the least information that reaching a given welfare must reveal about each user to the others, even with an ideal mediator. A decoder learns to guess preferences from the negotiation record, and the decoder's score is a lower bound on disclosure. We subtract an upper bound on the frontier and obtain an excess estimate. Decoder error can only lower this estimate, so a positive estimate is conservative. An exact census of a reduced bargaining game over every preference profile shows what the decoder misses. We audit seven models in four-party bargaining and four models in bilateral procurement. Estimated excess is positive in all 33 original-prompt conditions. In the census, at least 2.5 of at most 8 disclosed bits are excess, and 79–99% of the excess lies in the agreement rather than in the offers exchanged. The excess is not inevitable: a scripted bargainer usually reaches the same welfare with less disclosure. Telling agents to accept reasonable offers lowers disclosure without detectable welfare loss. However, the instruction transfers the decision to the first proposer, and the first proposer gains surplus in two of three models validated on fresh profiles. Decoder error also varies across policies. As a result, decoded bounds overstate seven of eight intervention effects in the census and reverse two of four policy rankings. Choosing among interventions therefore requires checking the auditor.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, multi-agent negotiation, privacy auditing, information disclosure, rate-distortion theory, delegation frontier, welfare-privacy trade-offs, agent auditing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tHoJOpenReview ↗
Rating
1
Confidence
4(+2)
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies a setting of LLM negotiation where agents represent users with private preferences and studies how much information about the users’ private preferences is revealed in order to reach agreement of a certain welfare level. The contributions are to give a lower bound on how much information must be revealed about user preferences to reach a agreement of a certain level of welfare, giving an audit to measure how much excess information negotiations reveal in practice, and empirical results auditing models in various negotiation scenarios.

Edit: I have read the rebuttal and maintain my assessment of the paper and score.

Critical strengths

This research question is quite interesting - I would be excited to read a draft of this paper that improved readability.

Critical weaknesses

A major weakness for this paper is clarity. The exposition is extremely obfuscated and the explanations and intuitions given use imprecise language, unexplained jargon, and makes claims without proper justification.

  • The main paragraph describing contributions in the introduction: “We answer the question with a floor, an audit, and a check (Figure 1). The floor is the delegation frontier, the least information that any agreement of a given welfare must reveal about each user to the other users, counting what each counterpart already knows. The audit decodes each user’s preferences from the negotiation record and subtracts an upper bound on the frontier, so decoder error can only hide excess, never create it. The check enumerates every preference profile of a reduced bargaining game and computes the true disclosure exactly, so the decoder’s error is measured directly.”

  • “Because the other side knows its own preferences, the frontier can exceed the classical rate–distortion floor.” What is the “classical rate-distortion floor”?

  • Section 3.2 - what is a “briefing”? This word appears twice in adjacent sentences but is never explained. “For target i, the observer is the idealized coalition of the other principals. The coalition knows their types θ−i but not their agents’ full briefings (Section 6)”?

  • What is “exposure”? “Side information changes exposure (Wyner & Ziv, 1976; Kimura & Uyematsu, 2008), even in a two-bit example.”

  • The first main result uses information theoretic tools to quantify information leakage between random variables, but this is not sufficiently explained, only given a treatment of a few sentences in the second paragraph of Section 3.2.

Questions for authors
Reviewer_VdndOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how much private preference information LLM negotiations reveal beyond what is needed to achieve their welfare. It provides a delegation frontier, estimates disclosure through a learned decoder, and compares the two to measure excess disclosure. Experiments cover 4-party bargaining and bilateral procurement. A reduced-domain census provides exact information measurements and checks decoder error. The paper reports positive estimated excess disclosure across all original-policy conditions. In the census, most excess disclosure comes from the final agreement, and decoder error sometimes exaggerates intervention benefits or reverses policy rankings.

Critical strengths
  1. The evaluation question is worthwhile. Privacy should be considered alongside task quality. A policy that reveals little because it fails to reach useful agreements should not automatically be preferred.
  2. The census provides a meaningful check of the estimator. Seven of eight intervention contrasts overstate the reduction in true disclosure, and two of four policy comparisons reverse their rankings. These results show why reduced decodability alone can be misleading.
  3. The additional validation addresses specific concerns. The fresh-profile evaluation, feature ablations, and serving-repeatability checks strengthen the experimental analysis. The authors also report welfare gains that do not replicate.
Critical weaknesses

W1. Some private information may already be available before negotiation. In procurement, each agent receives a full utility table, although its utility depends on the other side’s private type (Eqs. 30–31). It seems possible to recover the seller’s ready time from the buyer’s table, and vice versa, before negotiation begins. The audit excludes these briefings from the observer’s information, but it is unclear why this restriction is appropriate. Please clarify how much information about the counterparty is already available from the initial inputs. W2. Matching aggregate welfare may hide changes in who benefits. The frontier allows a central mechanism to access all types and only constrains aggregate welfare. In procurement, price cancels from welfare even though it determines how much each side gains. A solution could therefore disclose less while changing the deal substantially for one party. The first-proposer results suggest this matters here. It would help to check whether the disclosure reductions remain when individual payoffs or surplus allocation are also constrained. W3. The scope of the main findings remains unclear. The strongest evidence for agreement-dominated disclosure and policy-ranking reversals comes from a four-type census with two Qwen models under deterministic decoding. This is a useful check, but it is unclear how broadly these findings extend. The audit also assumes a known type distribution and utility family. A clearer explanation of where these assumptions are reasonable, and how the framework could be used beyond the constructed environments, would help establish its usefulness.

Questions for authors
  1. How much uncertainty about the counterparty’s type remains after observing the actual initial briefing? Can the main results be checked under an information allocation that does not reveal this type in advance?
  2. Does substantial excess disclosure remain when the comparison also preserves individual expected utilities or a specified surplus allocation?
  3. Which of the main census findings have been validated outside the reduced deterministic setting?
  4. How are the flat-welfare profiles identified in Appendix E handled in Equation (1) and in frontier computation?
Reviewer_aS59OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper tries to answer a question about how much LLM negotiation agents reveal about their user's private preferences beyond what is required for a good agreement. The paper defines delegation frontier, which minimum info any agreement achieving a given welfare must reveal to the other parties. A decoder is also trained to get the lower bounds of what the negotiation actually revealed. The gap between the delegation frontier and decoder bound is reported as excess. Excess is positive for all models evaluated and most of it comes from the final agreement. Telling agents to accept reasonable offers lowers measured disclosure but shifts decisions and surplus to the first proposer. The paper also shows that the decoder's error differs across policies which is enough to change the rankings of the interventions.

Critical strengths
  1. Novel framing and setting of paper. Asks how much disclosure the achieved agreement actually needed and charges the rest as excess. Unique way of disclosure calibrated to welfare.
  2. Acceptance instruction result mentioned in the paper has good reasoning. Disclosure drops, but it comes from other agents no longer deciding anything. So, the first proposer captures the extra.
  3. Auditing of the auditor by showing the decoder's error differs by policy and this is enough to flip which interventions look better. It's like a stress test of the measurement used in the paper. But it also undermines some of the results.
Critical weaknesses
  1. The agreement vs exchange split is less general and might be due to the protocols used. Exchange barely exists in these protocols. No message field in bargaining, thirty-word messages read through frozen embeddings in procurement. The message removal test done uses the same weak reader, so it cannot settle whether messages add little or the reader just cannot see it.
  2. The paper's check on the auditor undermines its results. Census shows the decoder error varies across policies which is enough to flip rankings. But it covers couple of models from same family, one domain and a merged policy. Everything else is compared using decoded bounds, which is exactly what the census says not to trust.
Questions for authors
  1. Can the exchange share grow with free text or a stronger text reader?
  2. The decoder reads messages as sent while counterparts see them truncated to forty words. Will there be any procurement results change if the decoder sees exactly what the counterpart saw?
  3. Could the census be extended to one non-Qwen model or one procurement cell, so the variation is not a single family observation?
1 comment

Thank you authors for the detailed comment. You are showing that decoder error can only undercount the excess, and rankings are never based on decoded bounds. So, that resolve one of my main concern. I had a look at the new content added as well, but I would like to retain my previous scores for the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.