acceptodds
Under review as a conference paper at ICLR 2027

Rational Clarification by Assistive Agents via Value-of-Information Reasoning

Abstract

Users of language-based assistive agents often make ambiguous requests. In response, an assistant can either directly act on its interpretation of the request — risking misalignment with the user — or ask a clarifying question. Which option is the most safe and helpful? A common approach is to ask questions that minimize uncertainty about the user's intent until a threshold is reached. However, this neglects the impact of uncertainty reduction on downstream performance, the costs of asking versus acting immediately, and the possibility that users may provide corrections without being asked. To navigate these trade-offs, we introduce *Rational Enquiry via Value-of-Information Reasoning* (REVOIR). REVOIR makes clarification decisions via inference-time reasoning about the *value-of-information* (VoI) of a question, which captures the expected improvement in task reward due to the answer received. In two assistive tasks — ambiguous question answering (CondAmbigQA) and preference-aligned household task planning (ADAPT) — we show that REVOIR achieves greater success with fewer questions than approaches based on prompting, chain-of-thought, fine-tuning, or information gain, improving preference satisfaction on ADAPT by 13-15% over a fine-tuned clarification policy while requiring no training and asking five times fewer questions. Furthermore, when the assistant can receive cheap user corrections after acting, REVOIR naturally infers that asking questions is not always efficient, demonstrating the adaptivity of our approach. In contrast, we find that vanilla reasoning agents fail to adaptively clarify user requests, and request *fewer* clarifications as reasoning effort increases.

Primary area:
foundation or frontier models, including LLMs
Keywords:
cooperative AI, theory of mind, reasoning under uncertainty, value of information, agent scaffolding, assistance games, AI alignment, large language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Uiu3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the problem of whether LLM assistants should ask clarifying questions or just act and let the user make corrections. The authors argue that keeping asking clarifying questions until the ambiguity is lower than a certain threshold (like prior works) is not optimal. A better LLM assistant should also consider the cost when raising clarifying questions. To this end, they proposed REVOIR, it first generates several user intent hypotheses then estimates the value of information (VoI) of candidate questions as well as acting. In the end, it decides whether asking is better or acting by also considering the cost. They evaluate their method on CondAmbigQA and ADAPT, showing that REVOIR is able to achieve higher success while reducing the number of clarifying questions.

Critical strengths
  1. It's a less studied problem but very practical issue for recent AI agents applications.
  2. Extensive evaluation across different LLMs detailed analysis. (Different LLMs and multiple evaluation settings)
Critical weaknesses
  1. There are two places in the paper that utilize a LLM to simulate a user. One is for evaluation the other is the internal user model in REVOIR (for scoring question candidates). For the former one, I've checked Table E.1, it seems like for "Human: not satisfied", the alignment is worse. For the latter one, I cannot find human evaluation. In the manuscript, the author claims that the internal user model and the external user simulator differ. But how does it differ? Without the human alignment evaluation on both the internal and the external user model, it is hard to make sure they are close to real applications.
Questions for authors
  1. It seems like in the setup of this paper, the "cost" is counted by the number of "questions/clarifications. However, this is somehow misleading. There can be different numbers of words/tokens or different amount of information it carries. Is there detail analysis on this?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Uiu315

Thank you for your positive review and the appreciation for our work. We address your concerns below.

  • W1. The discrepancy between the internal and the external user simulators
    • You are correct that the human validations in Appendix E only regard the external user simulator.
    • As discussed in Appendix C.2, the internal user simulator is not privileged to the true grounding condition (unlike the external simulator), with prompt templates different from the external simulator. The internal simulator also uses a different LLM (its own backbone) from the external simulator, which uses a fixed GPT-4o-mini model.
    • On top of these mismatches, several interacting components could potentially subject REVOIR to cascading failures from individual components. Yet, REVOIR’s strong performance despite misspecifications (which are unavoidable in practice) demonstrates the robustness of principled value-of-information reasoning. To further demonstrate REVOIR’s robustness, we present in Appendix D.7 a set of ablations with inattentive external user simulators that give a dismissive response with varying probability. In addition, we present a new set of ablations with stonewalling user simulators (per suggestions by Reviewer 5PQp22) in Appendix D.8. Under these ablations, REVOIR broadly remains the best-performing and most efficient method.
  • Q1. Cost modalities. Building on Table D3 in the original manuscript, we have conducted a thorough analysis of token usage on CondAmbigQA, including estimating the input / output tokens spent per dataset item/example, and per parallel reasoning branch (REVOIR and REIGN use K=5 branches, ReAct uses K=1 to K=20 branches). We find that:
    • REVOIR configs spend 2.1k–4.3k tokens (1.4k-3.1k input, 0.7-1.2k output) per example and parallel branch, similar to REIGN (1.5k–4.6k total, 1.0k-3.4k input, 0.5k-1.1k output) and ReAct (2.6k–5.7k total, 2.3k-3.5k input, 0.3k output) (see Tables R4 & R8).

    • Per example, REVOIR configs spend 10.6k–21.5k tokens (7.0k-15.6k input, 3.5k-5.9k output), REIGN spends 7.4k–22.9k (5.0k-17.1k input, 2.4k-5.5k output), and ReAct spends between 2.6k–4.4k (K=1, 2.3k input, 0.3k output) and 73.9k–113.3k (K=20, 17.3-67.1k input, 1.6k-6.9k output). Token costs increase with REVOIR / REIGN word budgets (see Tables R2 & R6).

    • In the measured range, REVOIR’s performance scales considerably better with more tokens, both per example and per branch, as estimated by robust linear regression (i.e. Kendall-Theil linear regression).

      • Under agent termination, REVOIR requires only 7.7 additional tokens for a gain of 0.01 in AnswerCorrectness, vs. 38.5 additional tokens for REIGN, and 33.3 additional tokens for ReAct (see Figure R1(a)).

      • Under user termination, REVOIR requires 50 additional tokens for a gain of 0.01 in AnswerCorrectness, whereas REIGN and ReAct decrease in performance with more tokens, dropping 0.01 in AnswerCorrectness with 24.4 and 83.3 additional tokens respectively (see Figure R1(b)).

      • Similar trends hold when regressing AnswerCorrectness against branch-normalized token usage (i.e. serial reasoning depth) (see Figure R2), with REVOIR either out-scaling or matching REIGN and ReAct. Under user termination, REIGN and ReAct decrease in performance with more tokens per branch, unlike REVOIR.

    • Together, these results imply that REVOIR scales more effectively with additional reasoning tokens than baseline methods, albeit with a higher minimum token usage per example.

Thank you again for the thoughtful review. We hope that our responses and revisions have comprehensively addressed your queries, and that you will consider raising your score.

Thanks for the authors detailed response. The clarification makes the paper clearer.

Reviewer_5PQpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper (REVOIR) studies proactivity in dialogue agents. Specifically, the paper studies when an LLM agent should ask clarification questions and when it should act directly. REVOIR comes up with several hypothesis and estimates the single-step value of asking only when the value of asking > the cost of doing so. The core novelty the author is trying to sell is that the valuation of asking / acting should also include a simulated correction and another attempt.

Critical strengths
  1. Clear framing ("information gain falls out as a special case when the reward is log belief). Essentially, the author argues that we shouldn't keep the ask vs act as the end of the dialogue and we should look ahead into the future.
  2. Fig 4 shows an interesting result ("when corrections are allowed, REVOIR is the only method that makes the user do less"), thought it also gains the least correctness…
  3. The method is training-free.
  4. Appendix is honest about negative results.
Critical weaknesses
  1. The distinction from prior value-of-information approaches needs clarification.
  2. Grand et al. (2025), which the paper cites, already decide between asking and acting with a value-of-information comparison on task reward. Information gain there only picks which question to ask.
  3. SAGE-Agent (Suri et al., 2025), which uses cost-penalized EVPI to choose clarifying questions for LLM agents, is not cited
  4. Neither is Rao and Daumé III (2018) on EVPI for clarification questions. As far as I can tell, the main new piece is extending the lookahead to the act branch through a simulated correction. This is an interesting extension, but its benefit needs stronger empirical support.

Rao, Sudha, and Hal Daumé III. "Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information." Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.

Suri, Manan, et al. "Structured uncertainty guided clarification for LLM agents." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

  1. The new idea is computed using assumptions that may favor answering. When the paper adds up how good it would be to answer right away, it gives credit it shouldn't: (1) it gives partial credit for answers the user would reject; (2) it assumes the user would correct even when the answer was already right; (3) it guesses how likely a correction is without looking at what the answer actually says. All of these make answering look better than it is. So when the method asks fewer questions, which the paper presents as smart behavior, that may just be the accounting error at work. The paper's own illustration (Fig. 1) also uses a cleaner calculation than the one they actually run (Eq. 12).
  2. The method is evaluated under favorable assumptions about user corrections. The novel part of this paper only matters if users can correct the assistant, and the simulated user always does that for free. However, real people behave differently. The authors had real people look at the same conversations. In the cases where the simulated user rejected an answer, real people agreed with that rejection less than half the time. So the simulated user corrects far more often than real people would. A real user may walk away without realizing the wrong assumption made. Moreover, the paper claims that it can be adapted to how much effort the user is willing to provide their oversight, but the technique was never run under settings where corrections are annoying or expensive for the user.
  3. On ADAPT, the improvement seems to come from the belief representation, rather than from the value-of-information reasoning.
  4. The CondAmbigQA gains are small.
Questions for authors
  1. On ADAPT, what happens if you keep the per-preference tracking but use a simpler rule for when to ask, such as asking a fixed number of questions? Please also rerun the baselines under the same setup.
  2. Suppose answers the user would reject get no credit, and the chance of a correction is estimated for each specific answer. Does the method still ask fewer questions, and does Fig. 4 still produce a similar trend?
  3. What happens when corrections are costly for the user, or when the simulated user only gives value responses (which may mimic real life scenarios under a lot of cases) like "that's wrong"?
  4. Can you position REVOIR against Grand et al. and SAGE-Agent?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 5PQp22

Thank you for your thorough review and critical engagement with our paper! We appreciate your recognition that our contribution is clearly framed, and that accounting for user corrections is an important aspect that hasn’t been considered by prior VoI approaches.

Below, we provide detailed responses to each of the concerns you have raised, including the results of additional experiments suggested in your review. These results have also been included in our revised manuscript.

  • W1 + Q4. Novelty and positioning. In our view, accounting for user corrections is just one novelty of REVOIR, even when positioned against prior (Grand et al, 2025; Rao & Daumé III, 2018) or contemporaneous (Suri et al, 2026) VoI-based methods.

    • Another key novelty is that REVOIR scales VoI reasoning to a broader range of assistive settings — formalized as conversational assistance games — where user intents can be open-ended textual hypotheses (not limited to a structured or factored hypothesis space), and task rewards can be graded. We demonstrate this successfully on CondAmbigQA and ADAPT.

    • In contrast, Grand et al (2025) studies a Battleship game with a fully enumerable hypothesis space (hidden boards), while SAGE-Agent (Suri et al., 2026) is restricted to intent inference over structured tool-call parameters, with fully-factored beliefs. Both consider only 0-1 rewards (hit vs. no-hit for Battleship, correct vs. incorrect tool call for SAGE-Agent).

    • Furthermore, Grand et al (2025) maximizes EIG instead of VoI for question selection, while SAGE-Agent maximizes VoI only for question selection, using an alternate threshold-based rule for the ask-vs-act decision. Only REVOIR consistently maximizes expected cumulative reward (which includes VoI as a term) for all decisions.

    • Rao and Daumé III (2018) is highly relevant classic work that we will cite — we thank the reviewer for raising it to our attention. As an older paper, it uses VoI for single-turn question generation, not the multi-turn clarificatory assistance that REVOIR considers.

    • In future revisions, we plan to discuss these contrasts in either the Related Work section or (due to space) an Extended Related Work section in our Appendix.

  • W2 + Q2. Assumptions in favor of acting/answering. There is no accounting error in our formal statement of REVOIR (Section 3) that systematically favors acting / answering, just minor implementation-specific approximations. Nonetheless, we also run new experiments using a more precise approximation where “the chance of a correction is estimated for each specific answer”, and find that REVOIR still asks fewer questions under user termination (see Appendix D.3).

    • In the definition of the value-of-acting (Eq. 5), REVOIR does not give partial credit for answers the user would reject. Rather, the net immediate reward $R^{\textrm{act}}(s, b)$ (last term in Eq. 5) is an expectation over cases where the user accepts the answer and cases where the user rejects the answer and provides a correction. In the latter cases, zero reward is given for the rejected answer, since goal reward is only received upon termination (i.e. answer acceptance).

    • In our implementation of REVOIR under user termination (Appendix C.2), we approximate the net immediate reward without explicitly simulating whether the answer $m^{\textrm{act}}$ will be accepted or rejected. As argued there, this is a decent approximation, since an answer $m^{\textrm{act}}$ that would be rejected by a user with intent $\theta$ will receive close to zero reward under the internal reward model $\hat G_{\mathrm{LLM}}(m^{\mathrm{act}}, \theta)$. Any “partial credit” for rejected answers is hence minimal under this approximation.

    • Our implementation approximates the expected post-correction reward by assuming a fixed probability of correction across all user intents (Eq. 12). We acknowledge that this assumes that correction might occur even if the answer matches the user’s intent, and that the correction probability does not depend on the answer itself. However, this approximation does not systematically inflate the value of acting/answering, and may sometimes reduce it.

      • In particular, while our fixed probability of correction may overestimate the true correction probability for the highest-weighted intent (close to 0, if the answer matches this intent), it will generally underestimate the true correction probability for all other intents (should be close to 1, if the answer only addresses the highest-weighted intent). As a result, Eq. 12 does not systematically overestimate the true value-of-acting.

      • Appendix D.4 (newly added) gives detailed error bounds for our approximation, and discusses when the approximation is reasonable.

  • Nonetheless, we have conducted experiments with a more precise approximation of the value-of-acting, replacing Eq. 12’s flat estimate of $P_{\textrm{cor}}$ with the reward-based estimate $P_{\textrm{cor}}(\theta, m^{\mathrm{act}}) = 1 - \hat G_{\mathrm{LLM}}(m^{\mathrm{act}}, \theta)$. That is, a user correction is more likely the more incorrect the answer is according to the internal reward model $\hat G_{\mathrm{LLM}}$ (see Appendix D.3).

    • This estimate of $P_{\textrm{cor}}$ is both answer-dependent and intent-dependent, addressing the reviewer’s concerns. Answers that perfectly address an intent (reward = 1) are never corrected.

      • We use this estimate to refine the value-of-acting approximation, s.t. $Q^{\mathrm{act}}(s, b, m^{\mathrm{act}}) \approx R^{\mathrm{act}}(s, b, m^{\mathrm{act}}) + \sum_{n=1}^N w^n (1 - \hat{G}_{\mathrm{LLM}}(\theta^n, m^{\mathrm{act}})) (R^{\mathrm{act}}(s', b'_n) - \mathbb{E}_{m^{\mathrm{cor}}}
C(mcor)C(m^{\mathrm{cor}})

)$.

  • Under this alternate approximation, we find that REVOIR achieves similar or higher correctness compared to our original approximation (see Appendix D.3), and that for most budget configurations, REVOIR asks less questions under user termination than agent termination, demonstrating that REVOIR still rationally adapts to the availability of user corrections (see Figure R3).

    • We separately find that the alternate approximation results in more question-asking than the original approximation. We discuss possible reasons for this in Appendix D.3.
  • W3 + Q3. Costly or unhelpful user corrections. We would like to clarify that REVOIR does not assume user corrections are free. Rather, a correction $m^{\textrm{cor}}$ is just as costly as any other user message $m^{\textrm{user}}$, following the cost function $C(m^{\textrm{user}}, m^{\textrm{agent}})$.

    • Still, we have run new experiments where corrections are $r$ times more costly for the user — i.e. each word in a user’s correction is $r$ times more expensive than a word in a user’s answer to a clarifying question (see Appendix D.5)

      • As corrections get more costly, REVOIR continues to maintain high AnswerCorrectness scores, ranging from 0.563 ($r = 1$, original result) to 0.569 ($r = 4$). REVOIR thus performs well even when corrections become costly (see Figure R4(a)).

      • REVOIR also tends to ask more questions on average as corrections get costlier, increasing from 0.7 questions for $r = 2$ to 1.25 for $r = 4$, albeit with a slight decrease from 1.0 to 0.7 questions going from $r = 1$ to $r = 2$ (possibly due to noise / small sample size). We take this as suggestive but inconclusive evidence that REVOIR adapts rationally to costlier corrections by asking more questions (see Figure R4(b)).

    • As requested, we have also run experiments where the user sometimes provides uninformative “stonewalled” corrections, of the form “That’s not what I meant.” This occurs with probability $p_\text{stonewall}$, and is not modeled internally by REVOIR. We thus stress-test how REVOIR performs against stonewalling users, even when it has a misspecified model which assumes that users never stonewall (see Appendix D.8).

      • At $p_\text{stonewall} = 0.2$, REVOIR outperforms all baselines in AnswerCorrectness for both Llama-3.1-8B and Llama-3.1-70B, while requiring 1.5 to 5 times less clarifications (see Figure R5).

      • As we increase $p_\text{stonewall}$ from 0.2 to 0.9, these trends broadly hold, with REVOIR mostly outperforming or matching the other baselines in AnswerCorrectness, and consistently using 3 to 6 times less clarifications (see Figure R6).

    • Finally, we note that our pilot human study does not establish that the user simulator corrects far more often than real people. While our simulator occasionally rejects answers that real people don’t, real people also reject answers that the simulator doesn't reject, and both directions occur at similar rates. Thus, the study only establishes that the simulator and real people behave differently in their correction behavior.

  • W4 + Q1. Additional ADAPT baselines (updated Section 4.2). For ADAPT, we “keep per-preference tracking but use a simpler rule for when to ask” by evaluating the Entropy Thresholding baseline, which only acts once the entropy of its (factored) beliefs drops below a (fitted) threshold. We also evaluate a variant of ReAct (adapted to the 2-stage setting) with per-preference tracking. This variant maintains and updates a list of preference atoms during the asking stage, analogous to the factored belief of the best REVOIR variant.

    • Entropy Thresholding only slightly outperforms REVOIR in the few-shot setting, achieving a 62.5% preference satisfaction rate (PSR) vs. REVOIR’s 58.8% PSR, while using 2x more questions (4 questions for Entropy Thresholding vs. 2.1 questions for REVOIR).

    • Despite using 1.25 to 2 times as many questions, ReAct significantly underperforms REVOIR, achieving about 30% PSR in both the few-shot and zero-shot settings, against 58.8% and 51% for REVOIR.

    • In addition to REVOIR’s strong performance with Joint beliefs in the few-shot / ICL setting (about 50% PSR at 2.5 questions), these results demonstrate that REVOIR’s strengths cannot be attributed just to factored beliefs (i.e. per-preference tracking).

Thank you again for your critical engagement with our paper, which has already led us to make significant improvements. We hope that our responses and revisions have comprehensively addressed your queries, and that you will consider raising your score.

Thanks for the rebuttal -- my concerns are addressed, and I'm raising the rating to a 3.

Reviewer_ofx8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Authors propose a method to improve on the assistance of an agent during a conversation with a user that poses ambiguous questions. The method, called REVOIR, estimates the model’s belief on the user’s private intent at inference time and uses that estimation to construct a policy that can take either the form of a generator of “asking” messages or “acting” messages. Experiments are conducted on the CondAmbigQA and ADAPT benchmarks and REVOIR is compared in different axes of performance with a diverse set of baselines, showing improvements in most of them.

Critical strengths

The paper is well motivated, easy to read, and well structured. The formulated problem is of interest to the community and the proposed method is, to my knowledge, original and well justified. Results are comprehensive and well designed.

Critical weaknesses

The method proposed in the paper strongly depends on correctly generating the alternative intents and score verbalizations in addition to the possible answers or actions. This eventually translates into spending more input tokens to generate those intents. To my understanding, that is not considered within the budget mentioned by the users, since this budget considers the output tokens of the user/agent messages. I could not find in the paper any estimation on how much the costs increase at inference time relative to the baselines.

In addition to this, I could not find any reference to the implementation code or to whether the authors will release the implementation upon approval.

Questions for authors

I would like to request for the two points mentioned before:

  • Please, provide an estimation of the average input and output tokens spent per sample by each method. I believe that a small subset of samples and the main configuration of each baseline is enough to provide a representative estimation comparison.
  • Please, clarify whether the code will be released and, if possible, provide a preview that contains the most representative parts of the experiment’s implementation.

Additional question:

  • Both datasets (CondAmbigQA and ADAPT) provide, in addition to the question-answer pair, the ground-truth hidden intent(s). Is there a systematic way to generalize to cases in which the intent is not clear? If not, I believe that this should be reported in the conclusions/limitations section.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ofx822

Thank you very much for your careful review of our paper, and your thoughtful questions. We appreciate your recognition that our paper is well-motivated, and that our method is well justified.

Our responses to your questions are summarized as follows:

  • W1 + Q1. Token usage: We recognize that inference costs are an important part of an inference-time reasoning method like REVOIR. Building on Table D3 in the original manuscript, we have conducted a thorough analysis of token usage on CondAmbigQA, including estimating the input / output tokens spent per dataset item/example, and per parallel reasoning branch (REVOIR and REIGN use K=5 branches, ReAct uses K=1 to K=20 branches). We find that:

    • REVOIR configs spend 2.1k–4.3k tokens (1.4k-3.1k input, 0.7-1.2k output) per example and parallel branch, similar to REIGN (1.5k–4.6k total, 1.0k-3.4k input, 0.5k-1.1k output) and ReAct (2.6k–5.7k total, 2.3k-3.5k input, 0.3k output) (see Tables R4 & R8).

    • Per example, REVOIR configs spend 10.6k–21.5k tokens (7.0k-15.6k input, 3.5k-5.9k output), REIGN spends 7.4k–22.9k (5.0k-17.1k input, 2.4k-5.5k output), and ReAct spends between 2.6k–4.4k (K=1, 2.3k input, 0.3k output) and 73.9k–113.3k (K=20, 17.3-67.1k input, 1.6k-6.9k output). Token costs increase with REVOIR / REIGN word budgets (see Tables R2 & R6).

    • In the measured range, REVOIR’s performance scales considerably better with more tokens, both per example and per branch, as estimated by robust linear regression (i.e. Kendall-Theil linear regression).

      • Under agent termination, REVOIR requires only 7.7 additional tokens for a gain of 0.01 in AnswerCorrectness, vs. 38.5 additional tokens for REIGN, and 33.3 additional tokens for ReAct (see Figure R1(a)).

      • Under user termination, REVOIR requires 50 additional tokens for a gain of 0.01 in AnswerCorrectness, whereas REIGN and ReAct decrease in performance with more tokens, dropping 0.01 in AnswerCorrectness with 24.4 and 83.3 additional tokens respectively (see Figure R1(b)).

      • Similar trends hold when regressing AnswerCorrectness against branch-normalized token usage (i.e. serial reasoning depth) (see Figure R2), with REVOIR either out-scaling or matching REIGN and ReAct. Under user termination, REIGN and ReAct decrease in performance with more tokens per branch, unlike REVOIR.

    • Together, these results imply that REVOIR scales more effectively with additional reasoning tokens than baseline methods, albeit with a higher minimum token usage per example.

  • W2 + Q2. Code release: To facilitate transparency and reproducibility, we are committed to releasing our code upon acceptance.

    • We are unfortunately not able to provide a code preview at this time, as our internal codebase includes cross-dependencies with other projects, and will require substantial clean-up before it is ready to be shared.

    • In the meantime, we hope that our Appendix provides sufficient detail on the implementation specifics for each dataset (Appendices C & I), as well as the prompts and parameters used (Appendices F, G, J, K)

  • Q3. Ground-truth hidden intents: While ground-truth hidden intents are required for evaluating whether a method fulfills the user’s intent, we would like to clarify that our proposed methods (REVOIR and REIGN) have no access to the ground-truth intents at inference time. As such, they are applicable to practical use cases with no knowledge of users’ hidden intents during deployment.

    • To evaluate on datasets without annotations of ground-truth intents, one option would be to recruit human annotators to label each ambiguous user request with a range of plausible hidden intents. This is similar to how Bayesian theory-of-mind models are evaluated against the probabilistic mental inferences made by humans (e.g. Baker, Saxe & Tenenbaum (2017); Zhi-Xuan et al (2024)).

    • Nonetheless, our assistance game formalism does rely on the assumption that ground-truth hidden intents exist, even though they are not directly observed. We acknowledge that this is a conceptual limitation that may be ill-suited to more complex settings (e.g., with evolving user intents). Extending our framework to such cases is indeed interesting and important future work, which we will make clearer in the Limitations / Conclusion section.

We have made detailed revisions to our paper to address your questions on token usage (see Appendix D.2), including further methodological details, and figures showing how AnswerCorrectness scales with inference costs (Figures R1 & R2).

We hope that these revisions adequately address your concerns, and thank you again for your thoughtful questions. We hope you will consider raising your score.

References

Baker, C. L., Jara-Ettinger, J., Saxe, R., & Tenenbaum, J. B. (2017). Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4), 0064.

Zhi-Xuan, T., Ying, L., Mansinghka, V., & Tenenbaum, J. B. (2024). Pragmatic instruction following and goal assistance via cooperative language-guided inverse planning. Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2094–2103. International Foundation for Autonomous Agents and Multiagent Systems.

Reviewer_ofx8OpenReview ↗

Response to authors

Thank you for your responses.

Q1) I find the Token usage analysis satisfactory, particularly the analysis on how this method scales compared to the others. I believe that, since most of the inference-time adaptation methods rely on the extra token usage, this is the correct procedure to compare advantages with different baselines.

Q2) The provided appendices are very detailed and well explained but I believe that providing a reproducible code is essencial for the transparency of the results

Q3) I completly agree with your answer. I probably did not explain myself very clear. What I ment is that, the assistance game formalism not only relies on the assumption that ground-truth hidden intents exist, but also that they can be well represented with a finite Θ\Theta set. Do you have any insights on how the proposed method would be modified in the case of more complex sets?

Given the complementary analysis provided for the first question, which was my main concern, I raise my score to 3.

AuthorsOpenReview ↗

Reply to follow-up questions

Thank you very much for raising your score! We agree wholeheartedly with your comment that providing reproducible code is essential and we are actively working toward a self-contained public release of the code.

We also appreciate your further question about handling the space of intents $\Theta$ with finite sets. We note that our formalism doesn't assume finite hypothesis space, as the set of all possible natural language intents is infinite (but countable).

In fact, REVOIR doesn't assume that $\Theta$ itself is finite, though it does have to approximate a belief over $\Theta$ with a finite set of $N$ weighted hypotheses (a.k.a. particles). We are aware of two other papers that take a similar approach for open-ended inference by Ma et al. (2025), and Zhi-Xuan et al. (2024). This follows the standard strategy of particle filtering, which approximate beliefs over infinite (e.g., continuous) hypothesis spaces with finitely many weighted samples. While a finite particle set does not guarantee full coverage of the support, it provides a consistent estimator of the posterior expectations under the right conditions. Zhi-Xuan et al. (2024) also show that this approach is cognitively plausible as human goal inferences are better matched by a particle filter than by exact Bayesian inference.

Importantly, the conceptual framework of REVOIR is not tied to a particular belief representation and remains applicable with more involved representations. For more complex intent spaces, particle filtering can be further enhanced by imposing structure on the hypotheses. The simplest case is the factored belief representation used in our ADAPT experiments, where independent beliefs over $M$ atomic preferences represent $2^M$ preference sets with only $M$ parameters, albeit with a strong assumption of independent preferences. To relax this assumption, one can consider Rao-Blackwellized particle filtering (Doucet et al., 2000), which samples some components of a hypothesis and tracks exact beliefs over the rest, conditioned on each sample. In ADAPT, for example, particles could represent a user’s dietary profile, each carrying its own factored belief over atomic preferences, so that the preferences can be correlated through the profile. For intents with deeper hierarchical structure, divide-and-conquer sequential Monte Carlo (Lindsten et al., 2017) can be used instead. It infers hierarchical components of a hypothesis separately and merges them up the hierarchy. Such approaches can represent complex intents without enumerating the full support.

We are happy to include in the paper more discussion about handling complex intent spaces with more involved belief representations as potential future work.

References

Ma, R., Qu, J., Bobu, A., & Hadfield-Menell, D. (2025). Flexible Agent Alignment with Goal Inference from Open-Ended Dialog. arXiv preprint arXiv:2508.15119.

Zhi‐Xuan, T., Kang, G., Mansinghka, V., & Tenenbaum, J. B. (2024). Infinite Ends From Finite Samples: Open‐Ended Goal Inference as Top‐Down Bayesian Filtering of Bottom‐Up Proposals. Topics in Cognitive Science, e70075.

Doucet, A., de Freitas, N., Murphy, K., & Russell, S. (2000). Rao-Blackwellised particle filtering for dynamic Bayesian networks. Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (pp. 176–183).

Lindsten, F., Johansen, A. M., Naesseth, C. A., Kirkpatrick, B., Schön, T. B., Aston, J. A., & Bouchard-Côté, A. (2017). Divide-and-conquer with sequential Monte Carlo. Journal of Computational and Graphical Statistics, 26(2), 445-458.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and the AC

We thank all three reviewers for their careful and constructive feedback. We were glad that reviewers found the problem practical and under-studied (Uiu315), the paper well motivated, clearly framed and easy to follow (ofx822, 5PQp22), the method original, well justified and training-free (ofx822, 5PQp22), and the evaluation comprehensive across LLMs and settings (ofx822, Uiu315).

The reviews raised a few shared concerns, which we have addressed through additional experiments, analysis and clarifications in the revised manuscript:

  1. Inference cost analysis (ofx822, Uiu315). We added a detailed analysis of token usage for REVOIR and all baselines (Appendix D.2). REVOIR's cost is comparable to the baselines', and its performance scales more efficiently with additional tokens.

  2. How does REVOIR differ from prior value-of-information methods? (5PQp22) Beyond modelling user corrections, REVOIR extends value-of-information reasoning to open-ended user intents and graded rewards, and it applies this reasoning consistently to every ask-or-act decision. We will make this positioning against Grand et al. (2025), SAGE-Agent (Suri et al., 2026) and Rao & Daumé III (2018) explicit in the paper.

  3. Do REVOIR's approximations unfairly favour acting over asking? (5PQp22) They do not. We show this analytically and confirm it empirically: REVOIR's performance and adaptive behaviour hold under an alternative approximation. We evaluated an alternative approximation (Appendix D.3) approach for value of acting and analyzed REVOIR's value-of-acting approximation errors and discussed appropriateness of the approximation approaches (Appendix D.4).

  4. Does REVOIR rely on users who correct readily and at no cost? (5PQp22) REVOIR does not assume corrections are free. We added stress tests in which corrections are more costly (Appendix D.5) and in which users give uninformative corrections (Appendix D.8). REVOIR remains the best-performing or competitive method in both cases, and it continues to ask far fewer clarifying questions than the baselines.

  5. Do REVOIR's gains on ADAPT come from its belief representation alone? (5PQp22) We added baselines that share REVOIR's belief representation but use simpler rules for deciding when to ask (Section 4.2), providing additional evidence that REVOIR's gains come from its principled value-of-information reasoning.

  6. Clarifications and limitations. (ofx822, Uiu315) We clarified the role of the internal user model and of ground-truth intents. We also expanded our discussion of limitations and committed to releasing our code upon acceptance.

We are grateful to the reviewers, whose suggestions substantially strengthened the paper. We hope these revisions address the main concerns, and we welcome any further questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.