- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work argues that accurate social inference is not sufficient for good interactive decision making, and proposes a decision-value framework separating three quantities: potential value, the gain achievable under perfect knowledge of the opponent type; acquired value, the expected improvement from a particular evidence procedure; and realized value, the advantage of the resulting informed policy in deployment, potentially under evidence or opponent-response mismatch.
Critical strengths
- Writing is easy to follow.
- The formulation is mathematically well-defined.
- The paper explicitly acknowledges the limitation that the framework is model-relative and that the experiments do not establish failure prevalence or general agent rankings.
Critical weaknesses
- Limited theoretical novelty. Potential and acquired value closely correspond to EVPI/EVSI, which the paper itself acknowledges, and several propositions appear to follow fairly directly from standard Bayesian decision theory.
- Each baseline is chosen to instantiate the specific failure mode being diagnosed, making Full's advantage somewhat expected.
- Some comparisons do not provide equal information access. In the final-offer inefficient-acquisition experiment, No language lacks the cheaper language-question channel available to Full, so the comparison partly measures channel availability rather than better decision making under identical information access.
- No strong common-access Bayesian baseline.
- Full performs better when the opponent lies but can perform worse than Naive VoI in honest cases because it pays unnecessary verification costs. The tradeoff that is underexplored.
Questions for authors
- What is the main theoretical contribution beyond standard EVPI/EVSI and Bayesian metareasoning?
- Can the authors provide a baseline with the same posterior, evidence channels, reliability estimates, payoff calculations, and numerical tools as Full, but without the proposed three-stage diagnostic framing?
- In the misplaced-trust setting, how does performance change as the probability of honest v.s. deceptive opponents varies? At what deception rate does verification become worthwhile overall?
- How much of Full's empirical improvement comes from the LLM reasoning component versus the deterministic Bayesian/value-computation module?