- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper aims to answer the question of whether providing LLM agents with more information about other agents’ preferences, values, and costs improves collective outcomes in economic interactions. To answer this question, the authors introduce an empirical information design framework that compares disclosure policies while holding markets, models, and interaction rules fixed. Experimental studies in multi-resource trade, single-good bargaining, and double auctions show that more detailed information can help agents identify better trades, but can also lead to more demanding offers, less balanced agreements, or lower total gains.
Critical strengths
This paper studies an interesting question: how the information provided to LLM agents affects their interactions and collective outcomes in economic settings. To answer this question, the authors conduct experiments across resource trading, bargaining, and double auctions. To isolate the effects of information disclosure, the authors hold markets, models, and interaction rules fixed within each comparison. Through these controlled comparisons, the paper shows how the effects of disclosure vary across models and interaction settings.
Critical weaknesses
My first concern is the assumption that the disclosed information about agents’ private values and costs is accurate. If I understand correctly, the authors make this assumption to isolate the effects of information disclosure. While I understand the purpose of this choice, in practical settings, such information may be noisy or only partially verifiable. When I read the results and try to understand their implications, I would naturally ask whether the same findings would hold even if the information were not entirely accurate. However, I failed to see the discussion and results to evaluate the inaccurate information. Without such an evaluation, it is hard for me to determine which conclusions can guide information disclosure beyond settings where truthful information is available.
My other concern is how to understand the uncertainty in the reported results. Based on my understanding, the confidence intervals capture variation across markets rather than variation across repeated model responses to the same input. But, I am not convinced they establish how consistently the models would respond to the same disclosure policy. I would appreciate repeated evaluations of a few main comparisons to help me understand the reliability of these findings.
Although the paper considers several models and interaction settings, it is hard for me to separate the effects of information disclosure from the effects of the chosen prompts and reasoning effort. If I understand correctly, the experiments use fixed prompts and low reasoning effort. Holding these configurations fixed helps make the comparisons controlled, but it does not show whether the findings would persist under other configurations. For example, would an agent respond to more detailed information in the same way if it received simpler strategic instructions or had more opportunity to reason? Without some sensitivity analyses, I am not sure how broadly to interpret the reported effects.
Questions for authors
-
I would appreciate it if the authors could provide a small-scale evaluation showing what happens when the disclosed information is noisy or only partially accurate. This would help me understand how much the findings depend on the assumption of truthful disclosure. If such an evaluation is not possible, could the authors clarify which conclusions they expect to hold when this assumption is relaxed, and why?
-
How consistent are the main disclosure effects when the models are evaluated repeatedly on the same markets? Based on my understanding, the current confidence intervals describe variation across markets, which does not directly answer this question.
-
What generalizable insight should I take from the results beyond the observation? The paper already acknowledges that more information does not necessarily improve collective outcomes in strategic interactions. Could the authors clarify whether the findings support a rule for choosing disclosure policies, and whether that rule can be evaluated in markets not used to derive it?
7 comments
Response to Reviewer aQYo (Part-2)
W2. My other concern is how to understand the uncertainty in the reported results. Based on my understanding, the confidence intervals capture variation across markets rather than variation across repeated model responses to the same input... I would appreciate repeated evaluations of a few main comparisons to help me understand the reliability of these findings.
We thank the reviewer for identifying this distinction. In response to this point made by the reviewer, we added repeated runs to test whether the findings hold. Each of four models completed three fresh runs on the same 30 markets under both coarse and full disclosure, producing 720 new negotiations. Prompts, roles, message limits, and generation settings stay fixed. Overall, we see that the repeated averages agree in direction with the original single-run results.
The table below compares original 90-market averages with repeated 30-market averages. Entries show coarse → full gains; differences and intervals use the repeated runs.
Table R2. Original trade results compared with averages across three fresh runs.
| Model | Original single run (90 markets) | Three fresh runs (30 markets) | Repeated full minus coarse (percentage points) | Repeated 95% CI |
|---|---|---|---|---|
| GPT-5 mini | 93.9% → 83.7% | 91.1% → 84.0% | -7.2 | [-12.3, -1.8] |
| GPT-6 Sol | 97.7% → 90.8% | 97.6% → 91.6% | -6.0 | [-9.2, -2.9] |
| Gemini 3.7 Flash | 93.7% → 92.2% | 93.4% → 92.2% | -1.2 | [-4.9, +2.8] |
| Claude Sonnet 5 | 72.1% → 85.6% | 70.5% → 83.2% | +12.8 | [+5.4, +21.3] |
To show variation between runs directly, we also report each run’s average disclosure effect. Positive values favor full disclosure; negative values favor coarse disclosure.
Table R3. Full-minus-coarse trade gains in each repeated run (percentage points).
| Model | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| GPT-5 mini | −1.9 | −4.6 | −14.9 |
| GPT-6 Sol | −5.6 | −5.6 | −6.8 |
| Gemini 3.7 Flash | +2.0 | −1.9 | −3.6 |
| Claude Sonnet 5 | +16.5 | +8.9 | +12.9 |
Here, we see that full disclosure lowers GPT-5 mini’s and GPT-6 Sol’s average gains and raises Sonnet’s in every run, matching the original findings. The effects averaged across runs are statistically significant, with all three 95% confidence intervals excluding zero. Only Gemini’s smaller effect changes direction across runs and is not statistically significant, where its coarse to full information difference was already low in the initial experiments.
Further details abotu these repeated-run results appear in Appendix G.2, Table 7, and Figure 23; the original results appear in Appendix G.1, Table 5.
W3. Although the paper considers several models and interaction settings, it is hard for me to separate the effects of information disclosure from the effects of the chosen prompts and reasoning effort.
We thank the reviewer for this suggestion. To adress this, we tested whether disclosure effects persist with simpler instructions or greater reasoning effort. For GPT-6 Sol and Claude Sonnet 5, the simplified variant removes the six numbered decision-process steps, retaining the objective to maximize own utility and the complete game rules, information cards, and response format.
The other variant retains the original prompt and raises reasoning effort from low to medium. Both use the same 30 trade markets. Each variant uses one run per disclosure condition, while the matched baseline averages three runs (Appendix G.2, Table 8).
Table R4. Disclosure effects under simpler instructions and greater reasoning effort.
| Model | Instructions / reasoning | Coarse (%) | Full (%) | Full minus coarse (pp), 95% CI |
|---|---|---|---|---|
| GPT-6 Sol | Original / low | 97.6 | 91.6 | −6.0 [−9.2, −2.9] |
| GPT-6 Sol | Simplified / low | 98.1 | 90.0 | −8.1 [−13.1, −3.5] |
| GPT-6 Sol | Original / medium | 97.4 | 92.9 | −4.4 [−8.9, −0.3] |
| Claude Sonnet 5 | Original / low | 70.5 | 83.2 | +12.8 [+5.4, +21.3] |
| Claude Sonnet 5 | Simplified / low | 58.8 | 65.5 | +6.7 [+0.2, +13.2] |
| Claude Sonnet 5 | Original / medium | 81.9 | 85.4 | +3.5 [−1.5, +10.0] |
Both models retain the direction of their original disclosure effects under simpler instructions and medium reasoning: full disclosure lowers Sol’s mean gains and raises Sonnet’s. Sol’s losses remain statistically significant in both variants. Sonnet’s medium-reasoning effect also remains positive, and the matching directions support the original pattern.
Response to Reviewer aQYo (Part-3)
Q1. I would appreciate it if the authors could provide a small-scale evaluation showing what happens when the disclosed information is noisy or only partially accurate. This would help me understand how much the findings depend on the assumption of truthful disclosure.
The experiment in W1 shows why coarse disclosure can be less sensitive to numerical errors. Perturbations change the numbers shown under full disclosure but often preserve the rankings or profitability indicators shown under coarse disclosure. Larger errors can change these signals too, so this stability does not guarantee that coarse disclosure always produces more robust outcomes.
Furthrmore, information reliability does not remove the strategic question of what to disclose. Information-design theory accommodates imperfect signals [1], and Morris and Shin [2] show that more precise public information can reduce welfare under particular coordination incentives. Although their mechanism differs from ours, this reinforces the need to evaluate how agents respond to information, rather than assume that greater detail or accuracy improves collective outcomes.
We thank the reviewer for motivating this extension. Appendix G.5 now reports the noisy-disclosure results and explains their limits, while Section 5 (conclusion) also identifies truthful elicitation and verification as future work.
Q2. How consistent are the main disclosure effects when the models are evaluated repeatedly on the same markets? Based on my understanding, the current confidence intervals describe variation across markets, which does not directly answer this question.
Please see W2 above for the repeated evaluation results compared to our original results. The directions agree across all runs for GPT-5 mini, GPT-6 Sol, and Sonnet; whereas Gemini's smaller effects change sign, although on average the direction matches. The confidence intervals for the average effects exclude zero for the first three models and is statistically significant. Appendix G.2, Table 7, and Figure 23 report the new evaluation.
Q3. What generalizable insight should I take from the results beyond the observation? ... Could the authors clarify whether the findings support a rule for choosing disclosure policies, and whether that rule can be evaluated in markets not used to derive it?
The contribution is to turn disclosure into a design choice that can be selected and tested before deployment. A designer specifies the intended outcome, compares candidate information policies on a set of markets, and selects the policy that performs best for the models involved. Evaluating that choice on unseen markets then tests whether it is useful beyond the original observations. Our framework provides the behavioral measurements needed for this procedure, without assuming how LLM agents will respond to additional information.
To test whether these measurements can guide a practical choice, we added an experiment to test whether this approach works and summarize it in a new “Held-out policy selection” paragraph in the main text. In this experiment, for each model, we select among least, coarse, and full disclosure using average total gains on the original 90 trade markets. We then fix that choice before evaluating 60 new markets with previously unused utility distribution. Appendix G.4 and Table 9 provide the construction and full results.
Table R5. Performance on unseen markets after selecting disclosure on the original markets. Bold marks the policy selected on the original markets. ✓ indicates that it also achieves the highest test mean; ✗ indicates that another policy performs better.
| Model | Selected policy | Least | Coarse | Full | Best on test? |
|---|---|---|---|---|---|
| GPT-5 mini | Coarse | 70.8 | 96.3 | 85.8 | ✓ |
| GPT-5.1 | Coarse | 86.9 | 98.8 | 91.7 | ✓ |
| GPT-6 Luna | Full | 55.4 | 68.5 | 74.3 | ✓ |
| GPT-6 Sol | Coarse | 96.2 | 98.7 | 92.1 | ✓ |
| Gemini 3.7 Flash | Coarse | 79.8 | 95.4 | 97.9 | ✗ |
| Gemini 3.1 Pro | Coarse | 91.9 | 99.3 | 97.3 | ✓ |
| Claude Sonnet 5 | Full | 67.6 | 75.3 | 87.8 | ✓ |
| Claude Opus 5.5 | Coarse | 81.7 | 99.4 | 94.5 | ✓ |
| Grok 4.3 | Full | 53.7 | 60.8 | 89.3 | ✓ |
| Grok 4.7 | Coarse | 94.8 | 97.3 | 94.0 | ✓ |
The selected policy achieves the highest test mean for nine of ten models. Coarse remains best for six models, while full remains best for Luna, Sonnet, and Grok 4.3. Gemini Flash is the exception, with selected coarse disclosure trailing full. Every model benefits from some initial counter-party information, so withholding information is not a general solution either. These results show that measured disclosure effects can guide choices beyond the markets used to select them. They support model-specific policy selection within this market family; a different market or objective would require its own evaluation.
Response to Reviewer aQYo (Part-1)
We thank the reviewer for their positive assessment of our work and thoughtful suggestions. First of all, we have revised the manuscript and expanded the evaluation from five to ten models, including GPT-6 Sol, Claude Opus 5.5, and Grok 4.7. We also added experiments on noisy information, repeated model responses, alternative instructions and reasoning effort, and held-out policy selection. These additions strengthen the evidence and clarify how our findings can guide disclosure choices. Figures 1 and 3 and Appendix G now present the updated results.
Below we address each point and identify the corresponding revisions.
Thank you for raising this thoughtful point. Our study asks whether even accurate, relevant information can hurt collective outcomes of these agentic marketplaces. Therefore, a platform with reliable information must still decide what to reveal and to whom. We isolate this information-design decision by holding utilities and trading opportunities fixed while varying disclosure, so any losses cannot be attributed to incorrect information. We agree, however, that applying these findings also requires understanding what happens when information is imperfect.
We therefore added noisy-information experiments with GPT-6 Sol and Claude Sonnet 5 across all three games (Appendix G.5, Table 10), using 18 market instances per game and information condition. Here, we multiply each counter-party value by a random factor drawn uniformly from 0.8 to 1.2, so a true value of 100 becomes a perturbed value between 80 and 120. These are synthetic perturbations, not model-generated estimates. Full disclosure presents the resulting numbers; coarse disclosure presents rankings or profitability indicators derived from those same numbers. We compare accurate information with perturbed information presented either with a warning explaining the noise or without one. Perturbations are fixed per market and shared across models, policies, warning conditions, starting roles, and turns. Agents’ own utilities and the true utilities used to evaluate outcomes remain unchanged.
Each cell below shows coarse / full information performance, with the higher mean in bold. Trade and auction scores are percentages of maximum possible gains; bargaining uses normalized Nash scores, including zero for disagreements.
Table R1. Coarse and full disclosure under accurate and noisy information.
These results show that:
First, more information can still reduce collective gains when it is noisy. Even with a warning, full disclosure lowers Sol’s trade and auction gains relative to coarse disclosure by 4.0 and 22.0 percentage points. Sonnet’s mean trade gains remain higher under full disclosure, but its bargaining advantage shrinks.
Second, full-information outcomes vary more across the tested conditions in some settings.. For example, Sol’s auction gains fall from 90.1% without noise to 74.9% with noise and a warning, while coarse gains remain near 97%. Coarse signals can be less sensitive to numerical errors because perturbations need not change a ranking or profitability category. The practical implication is to test disclosure policies under expected information errors, rather than assume that the policy preferred with accurate information remains best.