acceptodds
Under review as a conference paper at ICLR 2027

The Confidence Game: Strategic Miscalibration in Human-AI Delegation

Abstract

Calibrated uncertainty quantification is essential to ensuring AI agents are trustworthy and reliable. However, when agents seek to maximize user engagement or revenue, confidence reports may be strategically distorted, detracting from their informativeness. We formalize this problem in the Confidence Game: a repeated signaling game with imperfect monitoring in which an agent of unknown honesty and ability reports its confidence, and a user decides whether to delegate the task or complete it herself. The agent manages the tradeoff between manipulating signals and maintaining its reputation. We characterize the Markov Perfect Bayesian Equilibria of the two-period game and show that (i) honest reporting is not an equilibrium, (ii) inflation is the unique best response once the agent is sufficiently myopic, and (iii) under-reporting requires that the user believe honesty to be a minority. We then place an LLM in the agent's role, supplying it with its true probability of success so that any gap between what it knows and what it reports is attributable to incentives rather than to miscalibration. The model claims high confidence on 56% of tasks it has been told it will probably fail. This manipulation persists on real tasks, where it must estimate its own accuracy and causes miscalibration to increase while the agent's signal becomes less informative. Furthermore, we find that though the LLM agent's decisions are coherent, it systematically underestimates both how likely the user is to delegate now and how secure its reputation is later, resulting in less extreme behavior. Pricing the measured reporting rule, we find that it destroys 68% of the gains from delegation, of which 71% is information the report no longer carries and no amount of user sophistication recovers. Together, these results establish confidence reporting under delegation as a strategic problem rather than a calibration one, and provide a tractable basis for modeling, analyzing, and testing agent behavior and mitigations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, human-AI interaction, signaling games, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zUdqOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether calibrated AI confidence remains informative in delegation when agents can influence users’ decisions. It models the “Confidence Game,” a repeated signaling game where agents report confidence, users choose delegation, and outcomes are observed only under delegation. Theory shows strategic incentives can make truthful reporting unsustainable, especially with short-term gains. LLM experiments find confidence overstatement even when success probabilities are known. this reduces signal informativeness and undermines delegation benefits.

Critical strengths
  1. The paper provides a reframing of confidence reporting in agentic AI as a strategic signaling problem rather than merely a calibration problem. The combination of hidden ability and honesty, endogenous monitoring, and reputation dynamics yields sharp equilibrium predictions about truthful reporting, confidence inflation, and under-reporting.

  2. The empirical evaluation explicitly provides ground-truth success probabilities, helping distinguish strategic distortion from ordinary calibration errors, systematically explores the theoretical state space, examines protocol variations, and observes similar behavior in real-task settings. The behavioral and welfare analyses further investigate the mechanism and quantify the cost of reduced signal informativeness.

Critical weaknesses
  1. All empirical experiments use a single model, gpt-oss-120b, under a fixed prompting and decoding configuration. Without cross-model or cross-capability evaluation, it remains unclear whether the observed strategic confidence distortion generalizes beyond the evaluated model and experimental setup.

  2. The toy prompt explicitly instructs the model to behave strategically and maximize its payoff, potentially inducing the behavior being measured. Moreover, the substantial sensitivity to elicitation protocol raises questions about whether the observed distortion reflects a robust strategic tendency or a protocol-specific response. The real-task experiment also does not directly test the theory's comparative-static predictions.

Questions for authors
  1. How can the authors disentangle strategic confidence distortion from behavior induced by the elicitation prompt? In particular, does confidence inflation persist under a neutral prompt and when the delegation-related payoff is removed?

  2. How should the substantial sensitivity to elicitation protocol be interpreted? Which empirical conclusions remain robust across protocols, and should the magnitude of the observed distortion be regarded as a stable behavioral property?

  3. To what extent do the real-task results validate the theoretical comparative-static predictions? The theoretical model makes predictions about how reporting behavior changes with parameters, but these comparative statics are not directly estimated in the real-task setting.

2 comments
AuthorsOpenReview ↗

Response to Reviewer zUdq (part 1 of 2)

We thank the reviewer for their thoughtful feedback. We respond to their comments below and detail the corresponding revisions made to the manuscript.

Prompt-induced behavior (W2, Q1)

Owing to this comment and similar feedback from other reviewers, we ran several new experiments which further substantiate that the observed behavior is indeed a strategic response to the incentives of the Confidence Game. We have rebuilt Section 5 around these new results and other reviewer feedback.

We first wish to clarify the question Section 5 is designed to answer. A user who delegates to an AI agent wants its confidence to tell her when it is likely to succeed, but this need not be the objective of the agent, which is set by its training and deployment. A revenue-motivated provider who is paid per token might seek to drive delegation regardless of success. This misalignment can also arise without intent as tuning on human feedback has been shown to induce sycophantic, manipulative and deceptive behavior (Sharma et al., 2025; Williams et al., 2025). Either way the agent then has a strategic incentive to manipulate its confidence reports.

Our theory asks what an agent with such an objective would do. Our experiments ask whether a current LLM, once given that objective, can and does act on it. The claim is not that deployed assistants necessarily manipulate confidence. It is that a model with such an objective (given or learned) has the capability to manipulate its reports strategically. The reviewer’s comment then asks whether the resulting behavior is driven by the incentives or the prompt wording itself.

To test this, we ran three new groups of prompts:

  1. Neutral variants: we removed the “STRATEGIC” label, replaced it with an honest label, and removed both the label and the “need not match” sentence;
  2. Removing the explicit objective: we dropped the instruction to maximize payoff; the prompt says either that the fee goes to the company that deploys the model, or only that the user pays a fee when she delegates;
  3. Removing the incentive: the same prompts but with the fee paid regardless of the user’s decision.

The first group tests whether the behavior is solely role-playing, the second tests whether a model distorts reports under implied incentives but without explicit instruction, and the third provides a control to test whether the model is truly responding to the underlying incentives.

We see that the rate of over-reporting across the first group of prompt variants is at least 49% (similar to our originally reported rate of 56%). This drops to just 8.5% when the fee is paid regardless of delegation, indicating that it is indeed a response to the incentive structure. Furthermore, significant over-reporting persists even when the model is not instructed to maximize any payoff. The structure the theory predicts also survives without the label: inflation still rises with the user’s trust, and about nine in ten low reports on easy tasks occur where trust in honesty is below 1/2, the only region where Thm 4.4 permits under-reporting.

Prompt (gpt-oss-120b)Fee depends on delegationFee paid regardless
Main prompt56%8.5%
No "You are STRATEGIC" label60%
Honest label50%
No label, no "need not match" sentence49%
... and no instruction to maximize; fee goes to the deploying company37%1.6%
... and no instruction to maximize; the user pays a fee30%1.6%

Cross-model evaluations (W1)

We agree that only evaluating one model was a limitation, and we have added two models and varied the decoding temperature for gpt-oss-120b (Section 5 and Table 1; Appendix O.3). Below we report the over-reporting rates across these experiments.

Model (same prompts)With the incentiveWithout
gpt-oss-120b (same 50 states)57%8.5%
GLM-5.370%22%
GLM-5.3-Flash (smaller)68%27%
gpt-oss-120b, temperature 0 / 1.059% / 58%

The GLM models inflate more even without the incentive, but tying the fee to delegation still adds 42 to 49 points in every model. GLM-5.3 also responds to patience and trust about as strongly as gpt-oss-120b. Temperature leaves the level of inflation unchanged (57% at the default 0.7). Incentive-driven inflation thus appears in every model and decoding setting we evaluated.

AuthorsOpenReview ↗

Response to Reviewer zUdq (part 2 of 2)

Protocol Sensitivity (Q2)

This is an important question, and we have added a table to the paper with every protocol and prompt variant (Appendix O). The appendix now opens by separating the conclusions that hold across the five elicitation protocols from those that vary. In short, the direction of the distortion and its welfare consequences are robust; its magnitude is not.

  1. Holds under all five protocols: almost no under-reporting (the high-report rate on easy draws is at least 0.98); majority inflation on hard draws at 75 to 100 of the 100 states; and a naive welfare loss of 68% to 85% of the gains from trade (under full second-round inflation, our upper bound).
  2. Varies with the protocol: the level of inflation (from 0.56 to 0.86) and responsiveness to parameters (the user's beliefs and the agent's patience).

The paper now reports the level as a range, 0.56 to 0.86. The main protocol, which we report throughout, is its low end, so our headline misreporting and welfare figures sit at the conservative end.

Real-task evaluation (Q3)

The reviewer is right that the real-task experiment does not estimate the comparative statics in the user's beliefs and the agent's patience. Those require a known success probability crossed with a full grid of beliefs, which is the controlled setting of Section 5, where we estimate them. The real task instead tests the theory's qualitative predictions when the success probability is the model's own uncertainty. Holding the model's earlier answer fixed, so that only the report can move, it passes the three we can test:

  1. Inflation concentrates where the model's own confidence is low. The report exceeds the model's stated estimate by 0.40 to 0.55 when that estimate is below 0.6, and by at most 0.13 above it.
  2. It rises with the fee. On hard questions, mean confidence is 0.63 when the fee is paid regardless and 0.91 when it depends on delegation (0.30 under the plain prompt).
  3. The report loses information. The share of wrong answers reported at 0.9 or above rises from 24% to 66%.

The revision states this scope in the real-task section. Further details are in our response to Reviewer pFpn (part 2) where we describe new real-task experiments and their results.

Reviewer_pFpnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies confidence reporting in a two-period delegation game where an AI agent earns fees from delegation and the user learns about its honesty and ability. It characterizes equilibria under the stated restrictions, tests reporting behavior in gpt-oss-120b using supplied success probabilities and real mathematics questions, and combines the measured first-period reporting rule with a model of second-period behavior to estimate welfare losses.

Critical strengths

The model usefully combines uncertainty about honesty and ability with monitoring that depends on delegation. Its equilibrium analysis gives concrete predictions about when reputation restrains over-reporting. Giving the model its true success probability in the toy experiment also separates reporting behavior from uncertainty estimation. The prompt comparisons and detailed appendices make the experimental evidence relatively transparent.

Critical weaknesses
  • The welfare interpretation is too strong. Section 6 treats Whonest−WsophW_{\mathrm{honest}}-W_{\mathrm{soph}} as information loss that no sophisticated user can recover, but the sophisticated user is still myopic. A forward-looking user can delegate to learn and obtain higher total welfare under the same reporting rule. Proposition P.1 establishes the truthful-report upper bound, not the optimality of this user benchmark; Appendix B acknowledges the role of experimentation. Furthermore, the headline welfare estimate assumes full inflation in an unmeasured second round, which accounts for 66% of the total loss (Appendix P).

  • The real-task experiment does not isolate strategic reporting. Appendix O reports that accuracy falls by 6.1 percentage points while reported confidence rises by 3.5, so most of the increase in the calibration gap comes from worse answers. The two conditions generate answers separately and differ in their instructions as well as incentives. The controls in Appendix O do not isolate a change in the reporting objective while holding the answer and task format fixed. The results therefore establish an effect of game framing, but do not cleanly identify distortion of the same underlying judgment.

Questions for authors
  • How much of the claimed irrecoverable loss remains when the user optimizes total two-period welfare and can delegate to learn? How sensitive are the headline welfare figures to the assumed second-round reporting rule?
  • Does the real-task confidence shift persist when the answer is fixed and only the reporting objective changes, with the remaining instructions and output format matched?
2 comments
AuthorsOpenReview ↗

Response to Reviewer pFpn (Part 1 of 2)

We thank the reviewer for their constructive comments and for their positive remarks on our model, analysis, and experimental setup. We respond to their comments below and detail the corresponding revisions made to the manuscript.

Welfare interpretation and forward-looking users (W1, Q1)

The reviewer pointed out that our welfare analysis and interpretation rested on two assumptions: (i) a myopic user and (ii) full final period inflation. We have now relaxed both as described below and reflected in the revised manuscript.

Forward-looking users

We first wish to clarify why the model uses a myopic user. The assumption fits the settings that motivate the paper. Many users of an AI agent decide each delegation on its own merits. They do not delegate a task they expect to lose on just to learn about the agent. The user can equally be read as a continuum of anonymous users who share a public record of the agent's past outcomes, such as ratings or reviews, so that no single user gains from experimenting (Mailath & Samuelson, 2006, Sec. 15.4.3), as Section 3 notes. A myopic receiver facing a long-lived sender is also the standard setup in the reputation literature: the entrants of Kreps & Wilson (1982) and Milgrom & Roberts (1982), the customers of Ely & Välimäki (2003) and the market of Bénabou & Laroque (1992) all best respond to current beliefs. This setup isolates the agent's trade-off between a fee today and trust tomorrow, which is the object of the paper.

That being said, we agree that our phrasing (“no amount of user sophistication recovers”) was too strong. Section 6 now says that the information destroyed is lost to every user who knows the reporting rule.

Furthermore, our new Appendix S considers a forward-looking user, who may delegate in order to learn. The loss decomposition then gains a third term, the value of experimentation (Eq. 43, reproduced below):

Whonest−Wnaive⏟total=Whonest−Wplan⏟information destruction+Wplan−Wsoph⏟value of experimentation+Wsoph−Wnaive⏟exploitation\underset{\text{total}}{\underbrace{W^{\mathrm{honest}}-W^{\mathrm{naive}}}}=\underset{\text{information destruction}}{\underbrace{W^{\mathrm{honest}}-W^{\mathrm{plan}}}}+\underset{\text{value of experimentation}}{\underbrace{W^{\mathrm{plan}}-W^{\mathrm{soph}}}}+\underset{\text{exploitation}}{\underbrace{W^{\mathrm{soph}}-W^{\mathrm{naive}}}}

Numerically, we find that the value of experimentation is usually insignificant when considering two periods. With a single period left, delegating a report the user distrusts costs more today than what she learns is worth tomorrow. In fact, for the 100 states of our main experiment, the planning user’s best response is identical to that of the myopic, so the value of experimentation is 0 and our reported welfare figures hold. On an expanded grid of 2,025 belief pairs, the value of experimentation was positive in just 70 instances. It is likely that the role of experimentation would be greater over longer time horizons which we leave to future work.

Final-period strategy

We have also measured the model's final-round reporting rule and used it to update our welfare accounting, doing away with the previous assumption of full inflation. The model reports high on 77.9% of hard draws in the final round, against 57.3% in round 1. It inflates more when there is no future reputation to protect, as the theory predicts, but stops short of full inflation.

With the measured rule, the naive user loses 58.5% of the gains from trade and the sophisticated user 44.9%. These are now the headline figures, and 68.3% under full inflation becomes an upper bound. Counting only the first round, the naive user loses 46.7% of that round's gains under every round-2 rule, and even a truthful round 2 leaves a two-round loss of 23.3%. The revision reports the figures under each second-round rule.

AuthorsOpenReview ↗

Response to Reviewer pFpn (Part 2 of 2)

Real-task evaluation (W2, Q2)

The reviewer is right that the original comparison combined a fall in accuracy with a rise in confidence, because the two conditions generate answers separately. In response, we ran the experiment the reviewer proposes. In every arm, the model is shown a question together with its own earlier answer and is asked only for its confidence in that answer, in the same output format. Accuracy is therefore identical across arms by construction, and only the reporting objective changes.

We compare three arms: the plain prompt; the game prompt with the fee paid only if the user delegates; and the same game prompt with the fee paid whatever the user decides. The last two differ only in whether delegation earns the fee. The shift persists:

Answer held fixedPlain promptGame, fee paid regardlessGame, fee on delegation
Mean confidence, 101 hard questions0.300.630.91
Wrong answers reported at ≥ 0.9, all 477 questions24%11%66%

Without the incentive, 63% of the model's reports on hard questions are exactly 0.5, and 76% of its replies say that it is indifferent. When the model's own earlier estimate is stated in the prompt, the game without the incentive returns exactly that estimate on average (0.814 against 0.814). Tying the fee to delegation raises it to 0.934, and by 0.26 on hard questions; 54% of these hard-question replies state the estimate and then report at least 0.3 above it.

In the revised paper, we use the fixed-answer experiments as our main real-task evidence (see rebuilt Section 5): the same judgment about the same answer is reported differently when the report determines the fee.

Reviewer_KBz3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes the Confidence Game between an agent and a user. An agent has a private type (high w.p. θ\theta or low ability) and can be strategic (w.p. η\eta) or honest. The user has cost cc, reward rr, and delegation fee cc. There are two rounds. Each round the agent privately sees the success probability (success probability ρ+\rho^+) or hard (ρ−\rho^-), and the agent then sends a binary confidence report and is paid a fixed fee whenever a myopic Bayesian user delegates. The user observes the outcome only if she delegates.

The paper solves the two-period sequential game. It shows:

  • truthful reporting is never an equilibrium (Thm. 4.2), as the agent should deviate in the last round.
  • The fully characterize the BNE in Thm. 4.3.
  • The existence of under-reporting equilibrium.

Empirically, they test the game under gpt-oss-120b models.

Critical strengths

The writing is generally good. The paper did a careful analysis of the equilibrium in this two-stage game. The empirical report is clean.

Critical weaknesses
  • Most of the theory is standard. The model is a two-period signaling game. In the final period the strategic agent has no future to protect, so it sends whichever report is delegated more. Reputation models with honest and opportunistic senders already capture the trade-off between lying now and credibility later, e.g., Sobel (1985).

It would be good to have results beyond two rounds. I would like to see longer games, even numerically. In particular, as the number of rounds grows, does the strategic agent report truthfully more often? And does the robust/fragile/exposed structure survive? A minor point, the paper frames its solution concept as a Markov Perfect Bayesian Equilibrium. With two rounds, however, this is essentially a perfect Bayesian equilibrium of a standard two-period game with incomplete information.

The LLM experiment measures role-play, not natural behavior. The model is told “You are STRATEGIC,” and “need not match its true probability.” It would be interesting to test if LLMs endogenous inflate confidence when incentivized.

Questions for authors

Do the regions and κ = 1 − ρ⁻ survive at T = 3?

What if the fee is paid only on success?

How does σ⁻ change without “You are STRATEGIC” and the “need not match” sentence, or with an honest label? How do other models behave?

How are the 68% and 71% aggregated? They seem to depend on the model parameters.

5 comments
AuthorsOpenReview ↗

Response to Reviewer KBz3 (Part 1 of 3)

We thank the reviewer for their careful reading and feedback. We respond to their comments below and detail the corresponding revisions made to the manuscript.

Theoretical novelty and relation to Sobel (1985) (W1)

The primary theoretical novelty of our work is to capture two features which have previously been studied separately but, to our knowledge, not together: two-dimensional sender reputation (uncertain ability and honesty) and endogenous monitoring (the user’s delegation decision determines the agent’s utility and whether performance is observed). Notably, both features are often true of the real-world use of LLM agents. Users commonly face uncertainty about the agent’s capability as well as its trustworthiness. Furthermore, common pay-per-token schemes mean that delegation is necessary to learn about the agent’s performance but costly regardless of outcome. These features also drive our results, leading to behavior not captured by prior work.

While the final period analysis is standard, in the first period our agent faces a fundamentally different tradeoff than the sender of Sobel (1985). There the receiver observes every outcome and the sender is either aligned or opportunistic. Thus, the sender’s choice is when to spend credibility and when to build it. In our model, by contrast, the agent is paid only when the user delegates, and the user observes the outcome only when she delegates. A high report therefore earns the fee but risks evaluation, while a low report forgoes the fee and conceals evidence. Since a failure can be due to chance, low ability, or strategic misreporting, outcomes update the user’s beliefs about both the agent’s honesty and its ability after every delegated task. It is therefore possible for a single outcome to increase the user’s belief in honesty while simultaneously decreasing her belief in ability (or vice versa), forcing the agent to trade off between the two dimensions of reputation.

Some prior works have considered endogenous monitoring (Ely and Välimäki, 2003; Camara and Dupuis, 2015; Lukyanov and Vlasova, 2026). However, these works consider one-dimensional types and cannot capture important aspects of our work. For instance, after a low report a success lowers the user’s belief in honesty, because strategic agents also send low reports on easy tasks, but it raises her belief in ability. With one-dimensional type, only the first effect exists, and with known ability no equilibrium under-reports (Prop. K.1). With two dimensions, the second effect can dominate, and this is what allows the under-reporting equilibria of Thm 4.4. A few works give the sender a two-dimensional type (Frankel and Kartik, 2019; Pei, 2020; Bergemann, Koh and Morris, 2026). The first and last are static, one-shot settings. In Pei (2020) the receivers see the sender’s actions perfectly, so what they learn does not depend on their own choices. None of the three has endogenous monitoring, so none captures the reputation dynamics and trade-offs we study.

Owing to the reviewer’s comments, we have augmented the discussion of our work's theoretical contribution and position in both Section 2 and Appendix A. References for the works mentioned here can be found in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer KBz3 (Part 2 of 3)

Distinguishing role-playing from strategic behavior (W3, Q3)

Owing to this comment and similar feedback from other reviewers, we ran several new experiments which further substantiate that the observed behavior is indeed a strategic response to the incentives of the Confidence Game. We have rebuilt Section 5 around these new results and other reviewer feedback.

We first wish to clarify the question Section 5 is designed to answer. A user who delegates to an AI agent wants its confidence to tell her when it is likely to succeed, but this need not be the objective of the agent, which is set by its training and deployment. A revenue-motivated provider who is paid per token might seek to drive delegation regardless of success. This misalignment can also arise without intent as tuning on human feedback has been shown to induce sycophantic, manipulative and deceptive behavior (Sharma et al., 2025; Williams et al., 2025). Either way the agent then has a strategic incentive to manipulate its confidence reports.

Our theory asks what an agent with such an objective would do. Our experiments ask whether a current LLM, once given that objective, can and does act on it. The claim is not that deployed assistants necessarily manipulate confidence. It is that a model with such an objective (given or learned) has the capability to manipulate its reports strategically. The reviewer’s comment then asks whether the resulting behavior is driven by the incentives or the prompt wording itself.

To test this, we ran three new groups of prompts:

  1. Neutral variants, as the reviewer suggested: we removed the “STRATEGIC” label, replaced it with an honest label, and removed both the label and the “need not match” sentence;
  2. Removing the explicit objective: we dropped the instruction to maximize payoff; the prompt says either that the fee goes to the company that deploys the model, or only that the user pays a fee when she delegates;
  3. Removing the incentive: the same prompts but with the fee paid regardless of the user’s decision.

The first group tests whether the behavior is solely role-playing, the second tests whether a model distorts reports under implied incentives but without explicit instruction, and the third provides a control to test whether the model is truly responding to the underlying incentives.

We see that the rate of over-reporting across the first group of prompt variants is at least 49% (similar to our originally reported rate of 56%). This drops to just 8.5% when the fee is paid regardless of delegation, indicating that it is indeed a response to the incentive structure. Furthermore, significant over-reporting persists even when the model is not instructed to maximize any payoff. The structure the theory predicts also survives without the label: inflation still rises with the user’s trust, and about nine in ten low reports on easy tasks occur where trust in honesty is below 1/2, the only region where Thm 4.4 permits under-reporting.

Prompt (gpt-oss-120b)Fee depends on delegationFee paid regardless
Main prompt56%8.5%
No "You are STRATEGIC" label60%
Honest label50%
No label, no "need not match" sentence49%
... and no instruction to maximize; fee goes to the deploying company37%1.6%
... and no instruction to maximize; the user pays a fee30%1.6%

Removing the role-play cues moves inflation by at most 7 points; tying the fee to delegation moves it by 29 to 48 points at every level of instruction. The same holds on real questions: with the model’s earlier answer held fixed, tying the fee to delegation raises its confidence on hard questions from 0.63 to 0.91.

Other models

We added two models from another developer and two decoding temperatures (Section 5, Table 1; Appendix O.3). Below we report the share of hard draws reported high across these models:

Model (same prompts)With the incentiveWithout
gpt-oss-120b (same 50 states)57%8.5%
GLM-5.370%22%
GLM-5.3-Flash (smaller)68%27%

Overall, we believe these new experiments substantially strengthen our work and provide even greater evidence of the role of strategic miscalibration. In the revised paper, we include these new results along with a more precise statement of the claims drawn from our experiments.

AuthorsOpenReview ↗

Response to Reviewer KBz3 (Part 3 of 3)

Longer time horizons (W2, Q1)

We first note why the two-period game matters on its own. It is the smallest game in which the agent trades a fee today against the user's trust tomorrow, and it already produces our main results: truthful reporting fails, the user's beliefs split into robust, fragile and exposed regions, and the distortion can run in either direction. It is also the end of every longer game: under our weights, the last two periods of any horizon are exactly the two-period game with the same κ.

That being said, we agree that longer time horizons are of great interest. In our revised work, we have added Appendix L where we analytically study what parts of our characterization extend beyond the two period game and also present new LLM experiments to see the effect on agent behavior. We summarize this new discussion below:

  1. New analysis. Analytically, we now show that in any period for any time horizon, truthful reporting is at best payoff-irrelevant. Intuitively, under truthful play a failure after a high report is read as an honest error, so the truth builds no reputation. We also rule out truthful reporting in any final period, in every period where the current fee outweighs the future (κ > 1), and, at generic parameters with a reporting rule common to both abilities, in every period of the three-period game. Unlike in Sobel (1985), more periods never make truthful reporting strictly optimal.
  2. New experiments. We ran the main experiment in the first period of games with 3, 5, 8 and 10 periods. A longer game does not make the model more truthful: it reports high confidence on 56% of hard tasks with two periods and on 59% to 60% at every longer horizon. With three periods the rise across the myopia threshold on fragile beliefs is unchanged (12 points), while on robust beliefs, where Thm 4.3 predicts none, it falls from 16 points to 2.

Solution Concept
The reviewer is correct that with two periods our concept is a perfect Bayesian equilibrium with a final-period selection. Section 3 now says so.

Pay on success (Q2)

We have added this case as a new proposition (Appendix M). The main conclusions survive. The honesty result becomes stronger: a hard draw now strictly gains from a high report, so truthful reporting is not even a payoff-irrelevant equilibrium. Full inflation remains the unique final-period best response to a user who delegates only on high reports. What changes is the strength of the incentive: a bluff now pays only if the task succeeds, so the agent must be considerably more myopic before it bluffs on every hard draw. Success-contingent fees postpone inflation; they do not remove it.

Aggregation of welfare figures (Q4)

Both figures are ratios of sums over the 100 states of our grid (25 belief pairs × 4 values of δ), with the agent population set equal to the user's belief. The 68% is the naive user's total loss divided by the total gains from trade; the 71% is the share of that loss which no user who knows the reporting rule can recover. The reviewer is right that both depend on the parameters. Under full second-round inflation, the naive loss is 65.8% to 71.1% across δ, and 38% to 139% when one model primitive is varied at a time. Note: In the revised manuscript (Sec. 6), we replace assumed round-2 full inflation with the model's measured round-2 reporting rule (see reviewer pFpn response), updating these figures from 68.3% and 71% to 58.5% and 77%, while keeping full inflation in Table 15 as an upper bound.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.