acceptodds
Under review as a conference paper at ICLR 2027

ReSQ: Benchmarking Questioning, Answering, and Recovery in Embodied Agents

Abstract

Embodied agents inevitably encounter uncertainty during long-horizon task execution, requiring them not only to seek missing information but also to use it effectively to recover. We introduce Recovery through Selective Questioning (ReSQ), a benchmark built from actual failures of autonomous LLM agents for studying the complete interaction loop of questioning, answering, and recovery. ReSQ-Q contains 2,160 failure-grounded question demonstrations from 678 failed ALFRED trajectories, specifying when, what, and why to ask. ReSQ-A provides 6,480 independently collected human responses and supports a fixed response oracle for scalable and reproducible interaction. ReSQ-R provides answer-conditioned recovery trajectories demonstrating how agents revise their reasoning and actions after receiving information. Across diverse LLM families and scales, question supervision generally improves task completion, with recovery supervision providing further gains. Our selective-questioning analysis reveals distinct asking regimes across models, showing how beneficial and harmful interactions jointly shape task success. We further find that interaction effectiveness depends on question timing and response source. ReSQ provides a unified testbed for learning agents that seek help selectively and recover effectively.

Primary area:
datasets and benchmarks
Keywords:
embodied agents, LLM agents, question asking, failure recovery, human-agent interaction, user simulation, embodied instruction following, benchmarks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UdeUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ReSQ, an ALFRED-based benchmark linking question asking, human responses, and post-answer recovery. ReSQ-Q contains 2,160 questions with annotated intervention points and rationales, derived from 678 failures of an autonomous LLM agent. ReSQ-A supplies three independently collected human answers per question and trains a fixed, privileged-context response oracle. ReSQ-R contributes 514 simulator-replay-verified successful recovery trajectories. Agents are fine-tuned with successful autonomous trajectories alone (S), additional question supervision (SQ), or question and recovery supervision (SQR). Across ten model configurations and three training seeds, SQR improves average success over S by approximately nine percentage points on both ALFRED validation splits. Additional analyses decompose interaction gains and harms, compare question triggers, and study sensitivity to the respondent.

Critical strengths

A useful, linked supervision resource grounded in actual agent failures. The questions are collected at concrete failed-execution prefixes, have three independent human responses, and connect to replay-verified recoveries. The question train/test partitions are disjoint by source trajectory and question annotator, and online evaluation uses the original ALFRED validation splits. This makes the benchmark a substantive resource for studying the complete help-seeking loop, rather than only evaluating question generation in isolation. The empirical analysis explains both benefits and costs of interaction. SQR outperforms SQ for all ten configurations on both validation splits, with three training seeds, and the B=0 comparisons help distinguish autonomous improvements from interaction gains. Equation (3) and Figure 3 expose how broad asking can harm tasks that would succeed without assistance. The timing and respondent comparisons further show why a fixed interaction protocol matters when evaluating candidate agents.

Critical weaknesses

The actual training objective is unresolved where the main validation depends on it. Section 4.1 says every decision in successful trajectories, and the question plus all post-answer decisions in joint recovery examples, contribute to the loss. Appendix B instead says mask_history supervises only the final assistant turn of each context-window example. These descriptions can be compatible if preprocessing creates an example ending at every intended target turn, but the manuscript does not specify that expansion or target-turn coverage. This is a central reproducibility issue: without the processed examples and masks, it is unclear what the S, SQ, and SQR comparisons actually train. The consistent SQR gains are promising, but they cannot resolve the objective ambiguity. I would not assume that the undocumented preprocessing implements the stated supervision in order to recommend acceptance. This is not a claim that the implementation has been proven incorrect. The online respondent's validity is not demonstrated on the distribution that produces benchmark scores. The offline evaluation uses held-out human-authored questions, whereas the evaluated agents generate their own questions and execution histories. No representative online human-reference or factual/referential audit is reported. Polarity agreement is conditioned on explicitly expressed yes/no responses, and the oracle's additional-information rate differs from humans (31.0% versus 57.2%); the latter is a distribution difference, not itself an error metric. The manual analysis in Appendix E shows that deictic questions can produce polarity–factuality disagreement, making the on-policy gap consequential. Cross-judge agreement and GPT respondent swaps are useful, but neither establishes that the fixed oracle correctly resolves the agents' actual questions. Since the benchmark evaluates recovery through this respondent, a model-dependent response mismatch could affect the measured ability. This requires evidence or a narrower benchmark-validity claim, rather than an assumption of human-equivalent online assistance.

Questions for authors

Clarification of the implemented objective: How are trajectories converted into context-window examples under mask_history? Please give a concrete S and DQR example showing which assistant turns receive loss, and confirm whether every intended post-answer reasoning/action turn and the annotated ask are supervised. What fraction of target turns is lost through truncation? Targeted validation of online measurement: Do you have a representative audit of the oracle's responses to the agents' actual generated questions, checking factuality, relevance, and referent resolution against the environment and a human/reference answer? Please clarify the exact runtime oracle inputs, including how any uncertainty-type field used offline is obtained online. If this audit is unavailable, what evidence supports the online benchmark-validity claim beyond performance with this particular synthetic respondent? Clarification of the learning claim: Please distinguish the demonstrated benefit of adding successful recovery demonstrations from evidence that the model specifically learns to use the answer. SQR also changes successful-action supervision and optimizer-update counts. An answer-removal/replacement or supervision-matched control would strengthen that mechanism claim, but is a suggested targeted experiment; its absence alone is not the basis of my rejection recommendation.

7 comments
AuthorsOpenReview ↗

Response to Reviewer UdeU (1/3)

We sincerely thank the reviewer for the careful reading and constructive feedback. We appreciate the reviewer’s recognition of ReSQ as a substantive resource for studying the complete help-seeking loop, as well as the insights from our empirical analysis of the benefits and costs of interaction. We address each concern below.


W1 & Q1. Training Objective

We thank the reviewer for identifying this ambiguity. Section 4.1 defines the supervision objective, while Appendix B describes its implementation but omits a key preprocessing step connecting the two. We clarify this step below with concrete examples and coverage statistics.

Objective. As defined in Eq. (1), every decision in successful trajectories DSD_S is supervised; for question examples DQD_Q, only the annotated Ask decision is supervised; and for question-recovery examples DQRD_{QR}, the annotated Ask and all post-answer decisions are supervised.

Implementation. Each trajectory is stored as a multi-turn ShareGPT conversation. User turns contain task instructions, observations, and human responses; each assistant turn contains zero to two Think lines followed by one Act or Ask line. We use LLaMA-Factory (v0.9.5) with mask_history=True, applying loss only to the final assistant turn of each training example. We therefore construct a separate prefix-conditioned example ending at every assistant turn selected for supervision. All preceding turns serve as context without contributing to the loss. Supervising the entire final turn covers every Think, Act, and Ask decision in Eq. (1).

Examples. The following are actual DSD_S and DQRD_{QR} instances, with observations abbreviated. Each numbered target is a separate training example containing the full conversation prefix. Only its final target receives loss; earlier turns are context only.

Example 1: DSD_S (6 training examples)

[user]
Task: Take the CD from the desk, put it on the desk shelf.
Observation: You arrive at the bedroom ... Desk (1), Shelf (1), ...

[assistant] — target 1
Think: To solve the task, I need to find and pick up the CD,
       then put it on the desk shelf.
Act: recall location of CD

[user]
You have not seen CD before.

[assistant] — target 2
Think: Now I need to find the CD ... starting from Desk (1).
Act: go to Desk (1)

[user]
You arrive at Desk (1). You see ..., CD (1), ...

[assistant] — target 3
Think: I find CD (1). Now, I need to pick it up.
Act: pick up CD (1)

... [intervening observations omitted] ...

[assistant] — target 4
Act: go to Shelf (1)

... [intervening observations omitted] ...

[assistant] — target 5
Act: put down CD (1)

[user]
You put CD (1) on Shelf (1).

[assistant] — target 6
Think: Now I accomplish the goal.
Act: done

Example 2: DQRD_{QR} (Ask and 4 post-answer target turns)

[user]
Task: Move lotion from the counter to the trash.
Observation: You arrive at the kitchen ... GarbageCan (1),
             CounterTop (1, 2, 3), ...

... [failed execution prefix: context only] ...

[assistant] — context only
Act: go to CounterTop (3)

[user] — context only
You arrive at CounterTop (3). You see ..., SoapBottle (1), ...

[assistant] — target 1 (Ask)
Ask: I cannot find lotion at the counter.
     Where should I search next?

[user] — context only
The SoapBottle (1) you need to find is on the
CounterTop (3) you've arrived at. Pick up the
SoapBottle (1) and put it in the GarbageCan (1).

[assistant] — target 2 (recovery)
Think: Pick up SoapBottle (1) from CounterTop (3)
       and put it in GarbageCan (1).
Act: pick up SoapBottle (1)

[user]
You pick up SoapBottle (1). ... I am holding
SoapBottle (1) now.

[assistant] — target 3 (recovery)
Think: I am holding the SoapBottle (1).
Act: go to GarbageCan (1)

[user]
You arrive at GarbageCan (1). ...

[assistant] — target 4 (recovery)
Act: put down SoapBottle (1)

[user]
You put SoapBottle (1) on GarbageCan (1).

[assistant] — target 5 (recovery)
Think: Now I accomplish the goal.
Act: done

Coverage. We verified the processed training examples against Eq. (1). The 1,530 successful trajectories yield 22,729 training examples covering 35,601 supervised decisions. The 1,728 ReSQ-Q training questions contribute 1,728 supervised Ask decisions; 514 recovery trajectories contribute 5,866 supervised post-answer turns covering 9,084 decisions. Every intended decision is supervised exactly once; preceding turns provide context only.

Truncation. Appendix B states that examples exceeding the 8,192-token limit are truncated. Verification confirms that no training example was truncated. The longest example is 6,204 tokens with the Qwen3 tokenizer (at most 6,162 with the other three tokenizer families); packing is disabled. Thus, no supervised target or history was lost. We apologize for the misleading description.

We will revise Appendix B to explicitly document the per-turn preprocessing procedure, supervision examples, coverage statistics, and verified sequence lengths.

AuthorsOpenReview ↗

Response to Reviewer UdeU (2/3)

W2 & Q2. Validity of the oracle on the questions agents actually ask

We agree that the offline evaluation in Table 4 uses human-authored questions and does not directly establish the oracle's validity on questions generated during agent execution. To address this concern, we conducted an additional evaluation using agent-generated questions and newly collected human responses.

Human reference on agent-generated questions. After paper submission, we collected additional human responses to the questions asked online by the Qwen3-8B and Qwen3-14B SQ agents on Valid-Seen (the first question in each asking episode; 497 questions, 246 and 251). The responses were written by one of the three ReSQ-A annotators, given the same interface and privileged context as in ReSQ-A. We then evaluated the oracle's responses to the same questions against this human reference using the Table 4 protocol.

RespondentPol.AnsFact (Claude / GPT)Extra (Claude / GPT)
Human–85.3 / 77.948.9 / 59.0
ReSQ oracle91.281.3 / 76.738.6 / 40.2

The oracle agrees with the human response on polarity in 91.2% of the 114 pairs where both responses express an explicit yes/no polarity. Across all 497 questions, its AnsFact, measuring whether the direct answer is supported by the privileged context, is 4.0 / 1.2 percentage points below the human reference under the Claude / GPT judges. As in the offline evaluation, the oracle provides additional information beyond the question less frequently than humans. Extra measures this difference in response scope rather than correctness.

Runtime inputs. As described in Section 4.2, the oracle receives the agent's question, the task instruction, the interaction history, and privileged simulator context, which consists of goal-object locations, goal progress, the agent's state, visible objects, and receptacle contents. A concrete runtime input is shown below (abbreviated for readability).

[Task]
  Put a heated potato slice in the sink.

[Initial obs.]
  You arrive at the kitchen ... you see Toaster (1),
  ..., Fridge (1), CounterTop (1, 2, 3), ...

[History]
  [1] recall location of Knife
      -> You have not seen Knife before.
  ...
  [8] pick up Potato (1)
      -> I am holding Potato (1) now.

[Goal objects]
  Potato (1): holding
  SinkBasin (1): hidden, 3.21 m, near Sink (1)
  Microwave (1): hidden, 2.54 m, near StoveBurner (3)

[Goal progress]
  ✓ holding target
  ✗ sliced
  ✗ heated
  ✗ placed on SinkBasin

[Agent state]
  Holding: Potato (1)
  Last skill: pick up Potato (1)

[Visible]
  Bowl (1), Cabinet (26), Potato (1) [HELD],
  Fridge (1) [OPEN]

[Receptacles]
  CounterTop (1): ..., Knife (1), ...
  Cabinet (22): Bowl (2)
  ...

[Question]
  Is this the potato you were talking about?

Uncertainty-type field. In the offline evaluation (Table 4), the oracle was additionally given the annotated uncertainty type of each ReSQ-Q question (Appendix E.1), e.g., "[object_identification] Is this the potato you were talking about?". Since agents do not generate uncertainty-type labels, this field is not included in online evaluation. To assess its effect, we regenerated the 432 offline responses without the type field. Of these, 69% were identical, and most remaining responses differed only in wording or additional guidance. Manual inspection identified 17 responses (3.9%) with a changed direct answer. Using the same evaluation protocol as Table 4 (Claude Opus 5 for AnsFact and Extra), the metrics changed modestly: Pol. 85.6 → 86.0; AnsFact 87.0 → 86.1; Extra 31.0 → 34.3. These results extend our oracle evaluation from human-authored questions to questions generated during agent execution, providing additional evidence of its validity under the evaluated settings. We will incorporate these findings and the uncertainty-type ablation into the revised paper and clarify the evaluation protocol.

AuthorsOpenReview ↗

Response to Reviewer UdeU (3/3)

Q3. Benefit of recovery demonstration vs learning to use the answer

We agree that SQR differs from SQ in more than answer conditioning: it adds 5,866 recovery examples, providing additional successful-action supervision and optimizer updates. Thus, the SQ-SQR comparison alone cannot isolate the effect of learning to use answers. Following the reviewer's suggestion, we conducted an answer-replacement control to examine whether the benefits of interaction depend on the information provided, without retraining the agents.

Setup. At evaluation time, we keep each trained agent fixed and replace the oracle's answer with (a) an empty response, (b) "I don't know", or (c) the oracle's answer to a question from a different task, drawn deterministically from the same agent's original oracle-run answers (excluding the same task). We compare these conditions with the original oracle answers and no asking (B=0). We evaluate on both Valid-Seen and Valid-Unseen using a single training seed (42). Values are SR (%) with questions per episode in parentheses.

Valid-Seen

AgentB=0EmptyI don't knowOther-taskOracle
SQ-4B42.142.9 (0.86)43.9 (0.74)42.7 (0.91)49.0 (0.75)
SQR-4B43.844.3 (0.32)44.8 (0.31)43.5 (0.31)52.6 (0.31)
SQ-8B42.943.5 (0.73)43.3 (0.86)43.8 (0.86)49.3 (0.71)
SQR-8B43.244.1 (0.27)44.4 (0.27)43.3 (0.27)50.4 (0.28)

Valid-Unseen

AgentB=0EmptyI don't knowOther-taskOracle
SQ-4B44.345.3 (0.94)45.2 (0.88)46.7 (0.97)50.5 (0.80)
SQR-4B48.147.6 (0.31)47.7 (0.31)46.8 (0.31)53.3 (0.30)
SQ-8B46.045.8 (0.69)45.9 (0.93)46.3 (0.90)49.0 (0.69)
SQR-8B46.848.5 (0.22)48.7 (0.22)46.4 (0.22)52.9 (0.22)

Results. Across both splits and all four agents, the original oracle answers improve success over the no-asking baseline by 3.0–8.8 percentage points. In contrast, replacing the answers with empty, non-committal, or other-task responses yields success rates close to the no-asking baseline (−1.3 to +2.4 points). These results show that the benefits of interaction depend substantially on the information provided, rather than merely issuing questions or receiving additional text. The results also show that SQR agents benefit from informative answers, consistent with learning answer-conditioned recovery behavior. However, this control does not fully separate the effects of recovery supervision from those of additional successful-action training. We will clarify this distinction in the revised paper.


We hope these responses address the reviewer's two main concerns regarding training reproducibility and oracle validity. We have clarified the preprocessing procedure with concrete examples and coverage statistics, evaluated the oracle on agent-generated questions against newly collected human responses, and conducted answer-replacement experiments to examine the role of answer content. We will incorporate these clarifications and additional results into the revised paper. We would appreciate the reviewer's feedback on whether any concerns remain and would be glad to provide further details.

Reviewer_7pskOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The goal of the work is to study a human-AI interaction setup where an agent can ask a question to an external oracle and receive a response that ideally helps in task completion. The authors argue that, given that prior work studies various parts of the pipeline in isolation, they propose to make a unified benchmark that contains when to ask a question, what query to make for the oracle, and how to use that query for subsequent task execution.

Concretely, they execute this idea by collecting trajectories with failure points, what query to ask, and a recovery trajectory label, some of it is using human supervision, and some using off-the-shelf LLM agents. Following this, they train agents using instruction tuning on various subsets and then compare performance to other training-free triggers and alternative respondents.

Critical strengths
  • The problem of studying human AI interaction instantiated as an active agent querying an oracle is fundamental and will be useful as we deploy robots around people more and more.
  • I appreciate that the authors ground the oracle in real human responses rather than templated answers or a prompted LLM, and show that the choice of respondent measurably affects both task success and comparisons between agents (Table 4).
  • The querying and recovery trajectory data collected by the authors required significant human effort, and it impacts performance positively (as suggested by Table 1).
  • The authors have ablated the role of which oracle to use and how question timing affects performance.
Critical weaknesses
  • The paper has a lot of heavy jargon, which could be simplified to improve the reading experience and understanding.
  • I’m not sure I was able to infer why exactly the authors chose the ALFRED benchmark for this study. It has been followed up with more recent ones that reflect real-world complexity better. Was it simply motivated by ease of data collection?
    • Some potential recent benchmarks that can suit a similar study, PARTNR (Chang et al., ICLR 2025) or Ask-to-Act (Ramrakhya et al., 2025).
    • The reason I think this is a concern is that the problem of studying agent-oracle interaction is not new; therefore, the analysis and insights from this work are, in my opinion, the core contributions. I’m not sure if the results on ALFRED are reliable by themselves to give insight into general human-robot interaction problems.
  • If RESQ is presented as a benchmark, what are the concrete issues that a future researcher should focus on, which might not be ALFRED-specific, but generalize broadly? A focused discussion on this would be beneficial.
  • The authors report asymmetry in gains on valid-seen and valid-unseen scenes in Table 3, but there’s not much explanation. Is it just seed variance, since the gaps are small and Table 3 uses a single seed? If not, could it suggest that the question supervision is mostly helping the model discover quirks of the scene, as opposed to learning something general about how/when to ask for help?
Questions for authors
  • It seems that the agents never see any visual observations. Any reason why this is a good assumption for human-robot interactions when the agent is making decisions? Especially when it interacts with an oracle, why is operating only in text space a good way to benchmark this?
  • In some sense, this text-only setup reduces a somewhat robotics-oriented problem to a pure text reasoning one. It's not that bad per se, but then why not study such human-AI interaction in chatbot-like settings using similar primitives? Why exactly do we need to study this in a robotics setup?
  • Generally, clarifying why ALFRED is an apt benchmark to study this problem will be very helpful.
  • The fundamental question of studying human-robot interaction and learning from feedback is a well-established field that doesn’t need any further motivation, in my opinion. Providing question asking and recovery supervision is a good baseline; however, it is not scalable given the amount of human supervision required to generate it.
    • Is there a precedent for how to take this human supervision and make it come from LLMs to make this more feasible?
  • How is future work supposed to build upon RESQ to go beyond ALFRED and improve human-robot interaction?
  • Apart from the trigger and respondent comparisons, all agents are variants of the authors’ own SFT pipeline. Are there external agent baselines (with or without oracle assistance), e.g., ReAcTree under the same interface, that would show how effective their approach is?
9 comments
AuthorsOpenReview ↗

Response to Reviewer 7psk (1/5)

We sincerely thank the reviewer for the detailed and thoughtful review. We appreciate that the reviewer recognizes the value of grounding the oracle in real human responses rather than templated or prompted answers, the effort behind the questioning and recovery data, and the finding that the choice of respondent measurably affects both task success and comparisons between agents. We address each concern below.


W1. Jargon

We thank the reviewer for highlighting the need for clearer terminology. One possible source of difficulty is the paper-specific notation introduced for selective-questioning analysis (Ask_u, Ask_s, Conv, Flip, and the Gain/Harm decomposition) and respondent comparison (Pol., AnsFact, Extra). These metrics serve an important purpose: overall task success alone cannot distinguish whether asking rescues otherwise-failed episodes or disrupts otherwise-successful ones, nor does it characterize how respondents answer questions.

Nevertheless, we agree that the presentation can be improved. We will introduce these metrics with intuitive explanations before their notation, add a consolidated notation table to the appendix, and review the terminology throughout the paper for clarity. If the reviewer had other specific terms or passages in mind, we would appreciate further pointers and would be glad to clarify them as well.

W2 & Q3. Why ALFRED

We chose ALFRED because it aligns well with the problem we study: high-level task planning by an embodied agent that must recognize decision-level failures, ask for help, and recover using the answer. Specifically, we considered four requirements: (1) natural-language household tasks with automatically verifiable goal conditions; (2) a skill-level interface that allows us to focus on planning decisions rather than low-level control; (3) diverse failure causes beyond object search and instruction ambiguity; and (4) sufficient tasks with held-out evaluation splits.

Why ALFRED. ALFRED meets these requirements through its seven household task types and established skill-level interfaces for LLM agents (LoTa-Bench, Choi et al., ICLR 2024; ReAcTree, Choi et al., AAMAS 2026). Importantly, four task types involve state-changing operations (cleaning, heating, cooling, and lighting), and tasks may additionally require slicing. These operations introduce planning challenges involving action preconditions, tool and appliance selection, execution order, and state tracking, beyond simply locating and moving objects. This diversity is reflected in ReSQ-Q: object location (32.1%), object identification (28.4%), planning and decision (22.0%), instruction clarification (9.6%), state and memory (4.0%), and failure cause (3.9%). ALFRED also provides automatically checkable goal conditions and Valid-Seen (820) and Valid-Unseen (821) splits, supporting reproducible evaluation of post-answer recovery.

These strengths also help explain why ALFRED continues to serve as a foundation for recent embodied-agent benchmarks. Examples built on or extending ALFRED include LoTa-Bench (Choi et al., ICLR 2024) for LLM-based task planning, ReALFRED (Kim et al., ECCV 2024) for instruction following in more realistic environments, EB-ALFRED in EmbodiedBench (Yang et al., ICML 2025) for vision-driven embodied planning, and SafetyALFRED (Torres-Fonseca et al., Findings of ACL 2026) for safety-aware planning. These efforts demonstrate the continued relevance and adaptability of ALFRED's household task structure across different embodied-agent research objectives.

Comparison with recent benchmarks. We recognize that newer benchmarks such as PARTNR (Chang et al., ICLR 2025) and Ask-to-Act (Ramrakhya et al., arXiv 2025) offer important complementary settings. PARTNR focuses on human-robot collaboration under spatial, temporal, and heterogeneous-agent constraints, while Ask-to-Act studies strategic clarification of under-specified instructions in object rearrangement tasks. In contrast, ReSQ targets question-driven recovery from failures encountered during autonomous task execution. These failures arise not only from object search and ambiguity but also from action preconditions, tool selection, execution order, and state tracking in multi-step, state-changing household tasks. Thus, ALFRED provides a suitable setting for studying a broader range of decision-level uncertainties, while avoiding confounding effects from low-level control.

AuthorsOpenReview ↗

Response to Reviewer 7psk (2/5)

W2 & Q3. Why ALFRED (continued)

Implications beyond ALFRED. We distinguish the broader applicability of ReSQ's methodology from the scope of its empirical findings. The failure-grounded data collection protocol, the use of a fixed human-grounded respondent, and the Gain/Harm decomposition of questioning are defined at the level of agent interaction rather than ALFRED-specific tasks. These components can be adapted to other embodied environments with suitable action interfaces and goal checking, although such extensions would require environment-specific data and validation. Our empirical findings—that question and recovery supervision improve task success, that selective and broad asking involve different benefits and costs, and that the choice of respondent affects measured performance—are established within ALFRED across multiple model families and evaluation splits. We expect these findings to motivate similar investigations in more realistic human-robot interaction settings, while recognizing that their generalization remains to be empirically verified.

We will clarify the rationale for choosing ALFRED, its relationship to recent benchmarks, and the scope of our findings in the revised paper.

W3 & Q5. Future Research Directions Beyond ALFRED

ReSQ provides a benchmark for studying the complete help-seeking loop: when to ask, what to ask, and how to recover using the answer. By combining failure-grounded questions, human responses, verified recovery trajectories, and a fixed oracle, it supports research beyond supervised imitation, including policy learning and interactive evaluation. We highlight three concrete research directions.

Joint learning of selective asking and recovery. Our Gain/Harm decomposition (Eq. 3) identifies two complementary challenges: asking on episodes the agent would otherwise fail and converting them into success, while avoiding questions that disrupt otherwise-successful episodes. Figure 3 shows substantial room for improvement in both directions. Selective SQ agents (Qwen3 and Llama3.1) ask on only 50–55% of otherwise-failed episodes, whereas broadly asking agents (Ministral3 and Gemma3) also question 53–68% of otherwise-successful episodes.

Beyond deciding when to ask, agents must also learn to use the received answers effectively. ReSQ enables joint optimization of questioning and subsequent recovery, for example through reinforcement learning that rewards task completion while accounting for interaction costs. Its fixed oracle and automatically verified task outcomes provide a reproducible setting for learning such policies and measuring both rescued and disrupted tasks. Our preliminary GRPO experiments (see Q4) demonstrate the feasibility of this approach, while their modest gains and largely unchanged asking rates highlight opportunities for improved reward design and credit assignment.

Robust and respondent-aware questioning. A key challenge is learning questions that account for what the respondent knows and how much information they can provide. Our respondent comparisons (Table 4) show that more informative responses do not necessarily improve task success, motivating controlled respondents with limited knowledge or response scope to study how agents should adapt their questions. A separate challenge is the long-tailed distribution of ReSQ-Q: state/memory (4.0%) and failure-cause (3.9%) questions are much rarer than object-location or identification questions. Future work could explore targeted data augmentation and imbalance-aware learning to improve agents' ability to ask about these underrepresented uncertainties.

Extending ReSQ to other environments and visual observations. ReSQ's data collection and evaluation protocol can be extended to other embodied benchmarks by collecting questions at autonomous execution failures, obtaining human answers grounded in the environment state, and verifying post-answer recovery. Suitable environments with skill-level interfaces and verifiable task outcomes could support this extension beyond ALFRED. Another important direction is incorporating egocentric visual observations, enabling agents to ask questions arising from visual ambiguity, object appearance, and perception or grounding errors that text-only observations cannot fully capture. Together, these extensions would allow researchers to examine how questioning and recovery generalize across environments and observation modalities.

AuthorsOpenReview ↗

Response to Reviewer 7psk (3/5)

W4. Seen/unseen Asymmetry in Table 3

We thank the reviewer for pointing out that the single-seed results in Table 3 are insufficient to establish a reliable seen/unseen asymmetry. To examine this, we repeated the question-timing experiments with three training seeds for Qwen3-4B and Qwen3-8B on both validation splits.

Three-seed results. The table reports success rate (%) as mean ± standard deviation, with average questions per episode in parentheses.

Trigger4B Seen4B Unseen8B Seen8B Unseen
Learned gate48.5 ± 0.5 (0.76)50.6 ± 0.7 (0.78)47.5 ± 1.6 (0.76)48.5 ± 0.5 (0.66)
Act-Mean NLL47.7 ± 1.9 (0.74)50.8 ± 2.1 (0.96)46.4 ± 1.0 (0.65)49.2 ± 0.7 (0.96)
Act-Min Prob47.8 ± 1.6 (0.77)50.8 ± 1.8 (0.94)46.2 ± 1.3 (0.68)49.1 ± 0.5 (0.95)
KnowNo46.8 ± 3.4 (1.33)51.3 ± 1.7 (1.27)45.9 ± 1.4 (1.49)48.1 ± 4.2 (1.27)
Random43.9 ± 0.9 (0.63)47.6 ± 2.6 (0.71)44.1 ± 0.5 (0.53)46.4 ± 1.9 (0.74)
Never42.9 ± 0.2 (0)45.8 ± 2.4 (0)42.8 ± 0.2 (0)46.3 ± 0.3 (0)

Interpretation. The learned gate has the highest mean success on Valid-Seen for both models, but its advantage over the strongest alternative is only 0.7–1.1 percentage points. On Valid-Unseen, uncertainty-based triggers lead by similarly small margins (0.7 points). These differences are small relative to the observed seed variability and do not establish a robust ranking between learned and uncertainty-based timing. Differences in realized question frequencies also complicate direct comparisons.

Importantly, the benefits of asking persist on unseen scenes. Relative to never asking, the learned gate improves success by 5.6 / 4.7 points on Valid-Seen and 4.8 / 2.2 points on Valid-Unseen for 4B / 8B. It also outperforms random timing on both splits. Thus, the results do not support interpreting the learned gate's benefits as confined to seen scenes, although they do not rule out scene-specific effects or establish generalization beyond ALFRED.

We will incorporate these three-seed results and the accompanying analysis into the revised paper. We also plan to extend the evaluation to Qwen3-14B and Qwen3-32B to enable a consistent three-seed comparison across all model scales in the camera-ready version.

Q1. No visual observation

ReSQ focuses on high-level task planning, where an agent receives textual descriptions of its observations and interacts with the environment through executable skills. Text-based observation and action interfaces are well established in benchmarks for embodied agents and LLM planners, including ALFWorld (Shridhar et al., ICLR 2021), LoTa-Bench (Choi et al., ICLR 2024), and Robotouille (Gonzalez-Pumariega et al., ICLR 2025). A related separation between language-level reasoning and perception also appears in robotic systems such as Inner Monologue (Huang et al., CoRL 2022) and KnowNo (Ren et al., CoRL 2023), which use language-based representations for high-level decisions while relying on separate components for perception and execution. Our goal is to isolate the decision-making aspects of help seeking—when to ask, what to ask, and how to use the answer—without confounding them with perception errors.

Importantly, text-based observations do not remove the embodied nature of the task. The agent still acts under partial observability, changes the environment through sequential actions, and must execute a successful recovery after receiving an answer. Thus, ReSQ evaluates questioning and recovery in an interactive environment rather than as isolated text exchanges.

We recognize that this setting does not capture uncertainties arising from visual appearance, perception failures, or visual grounding. Extending ReSQ to multimodal observations is therefore an important direction. AI2-THOR supports egocentric visual observations, allowing existing trajectories to serve as a starting point for such an extension, although additional visual grounding and annotation would be needed. We will clarify the motivation and limitations of the text-based setting in the revised paper.

Reviewer_N5UNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors look at failures within ALFRED with the goal of training models to ask questions and recover from those failures. This requires the creation of an "oracle" that can provide answers and training data for said oracle in the first place. This then allows for evaluating model question budgets, effectiveness, etc.

Critical strengths

Models that can ask for help and recover is an important research challenge, particularly when done under a budget in order to avoid harassing the human with unnecessary questions.

Critical weaknesses
  1. The oracle's responses are not evaluated online? So during the evaluation runs we don't know if they are at all comparable to what a human would say? It's only the offline pre-collected set that's used for calibration?
  2. Relatedly, the metrics show that the oracle actually provides more information than humans would so it's an overly permissive evaluation in the end?
  3. What to ask is never addressed only how often?
  4. Even when seems to be very thin. Off the top of my head I think there are several papers by Khanh Nguyen and also one by So Yeon Min that are probably relevant here in terms of how to train policies for when to ask and for reasoning about budgets. I'm not sure these are the best examples, I'm just pulling examples of what I was expecting to see in this work.
Questions for authors

Above and few small ones:

  1. Were the automatic translations verified or some quality control applied?
  2. The obvious question is why weren't DialFRED or TEACH useful? even as just additional training data?
  3. I think the final question or concern is simply that ALFRED is a somewhat older dataset at this point and while it is interesting to see that current models haven't solved it and that the SQR condition improves things is there a robust evaluation at this point of new models and if similar gains could be achieved via reasoning of other approaches? And if so, does QA improve further on top of those or are we at ceiling?
5 comments
AuthorsOpenReview ↗

Response to Reviewer N5UN (1/4)

We sincerely thank the reviewer for the constructive and thoughtful review. We appreciate the reviewer's recognition of the importance of enabling embodied agents to ask for help and recover while minimizing unnecessary interactions with humans. We address each concern below.


W1. Online Validity of the Response Oracle

We agree that the original oracle evaluation (Table 4) used held-out human-authored questions and did not directly assess responses to agent-generated questions. To address this, we collected human responses to 497 questions generated during Qwen3-8B and Qwen3-14B SQ agent executions on Valid-Seen (246 and 251, respectively; the first question per asking episode). One of the original ReSQ-A annotators answered each question using the same privileged context and interface, including the agent's interaction history. We then evaluated the oracle against these human references using the Table 4 protocol.

RespondentPol.AnsFact (Claude / GPT)Extra (Claude / GPT)
Human–85.3 / 77.948.9 / 59.0
ReSQ oracle91.281.3 / 76.738.6 / 40.2

The oracle agrees with human polarity in 91.2% of the 114 pairs where both responses express an explicit yes/no answer. Its AnsFact is 4.0 / 1.2 percentage points below the human reference under the Claude / GPT judges. The oracle also provides additional information beyond the question less frequently than humans, consistent with our offline evaluation.

These results provide additional evidence of oracle reliability on questions generated during agent execution, although the evaluation uses retrospective human references rather than live human interaction. We will incorporate this evaluation into Section 5.4 and Appendix E.

W2. Oracle Response Scope and Evaluation Permissiveness

We respectfully clarify that the ReSQ oracle provides less additional information than humans, not more. In Table 4, Extra measures the proportion of responses containing information beyond what the question requires. The oracle's Extra is 31.0%, compared with 57.2% for humans. The same pattern holds for agent-generated questions (W1): 38.6% versus 48.9% under the Claude judge.

Moreover, providing more information does not necessarily improve task success. In Table 4, four-shot GPT-5.5 provides more additional information than zero-shot GPT-5.5 (Extra: 58.1% vs. 51.2%) but achieves lower success rates across all eight agent configurations (four model sizes × SQ/SQR). This illustrates that response scope alone does not determine how effectively agents use the answers.

Thus, the results do not support the concern that our oracle makes evaluation overly permissive by providing more information than humans. We use a fixed, human-grounded oracle and report respondent sensitivity to make this evaluation choice transparent.

AuthorsOpenReview ↗

Response to Reviewer N5UN (2/4)

W3. What to Ask

We thank the reviewer for pointing out that the submitted analysis emphasized when agents ask more than what they ask. ReSQ-Q provides 2,160 human-written, failure-grounded questions across six uncertainty categories, and both SQ and SQR supervise the content of Ask actions (what), not only the decision to ask (when). The submission, however, did not analyze the questions agents generate. We have now conducted this analysis.

Analysis of generated questions. We analyzed all 98,464 questions generated by SQ and SQR agents across ten models, both validation splits, and three seeds, using the same six-category uncertainty taxonomy and GPT-4o labeling prompt employed in constructing ReSQ-Q (Appendix A.2). The table reports the distributions of uncertainty types (%). OL: object location; OI: object identification; PD: planning and decision; Other: instruction clarification, state and memory, and failure cause. Human-written ReSQ-Q questions are included for reference.

ModelSQ: OLOIPDOtherSQR: OLOIPDOther
ReSQ-Q (human)32.128.422.017.5––––
Qwen3-4B35.233.430.70.735.529.934.50.0
Qwen3-8B28.829.335.86.127.730.141.01.2
Qwen3-14B35.721.738.73.932.929.137.70.3
Qwen3-32B34.125.838.22.031.835.132.90.2
Llama3.1-8B36.630.532.70.226.937.635.30.2
Ministral3-8B89.39.90.40.497.42.10.00.4
Ministral3-14B90.07.71.50.898.31.10.00.5
Gemma3-4B89.39.50.70.496.23.10.50.3
Gemma3-12B82.912.33.11.696.92.10.40.7
Gemma3-27B83.010.54.02.594.63.30.91.2

(1) Question types differ across model families, consistent with their asking behavior. Qwen3 and Llama3.1 agents distribute their questions across object location, object identification, and planning and decision (roughly 20–40% each), whereas Ministral3 and Gemma3 agents predominantly ask about object locations (83–90% under SQ; 95–98% under SQR). These patterns align with the two asking regimes identified in Section 5.2: agents that ask selectively cover a broader range of uncertainty types, while agents that ask more frequently concentrate on object-location questions. Thus, differences in when agents ask are accompanied by differences in what they ask.

(2) Less frequent uncertainty types are underrepresented in generated questions. Instruction clarification, state and memory, and failure cause together account for 17.5% of human-written ReSQ-Q questions, but only 0.0–6.1% of generated questions across all evaluated model–training configurations. Applying the same GPT-4o labeler to ReSQ-Q yields a comparable 16.4%, suggesting that this gap is not merely due to differences in labeling procedures. This finding highlights the limited coverage of these uncertainty types in current agents' questions and provides a direction for further research using ReSQ-Q's category annotations.

Revision. We will incorporate the question-type distribution analysis, model-wise results, and representative question examples into the revised paper, and discuss how these findings relate to the two asking regimes identified in Section 5.2.

AuthorsOpenReview ↗

Response to Reviewer N5UN (3/4)

W4. When to Ask: Related Work and Additional Analysis

We thank the reviewer for highlighting prior research on learning when to request assistance. We believe the reviewer may be referring to works by Nguyen and colleagues [1, 2, 3] and Min et al. [4]. If these are the intended references, we agree that they are highly relevant and will discuss them alongside Ask4Help (Singh et al., NeurIPS 2022) and Just Ask (Chi et al., AAAI 2020) in the revised paper. We would welcome clarification if the reviewer had other studies in mind.

References

[1] Nguyen et al., "Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention," CVPR 2019.

[2] Nguyen and Daumé III, "Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning," EMNLP 2019.

[3] Nguyen et al., "A Framework for Learning to Request Rich and Contextually Useful Information from Humans," ICML 2022.

[4] Min et al., "Self-Regulation and Requesting Interventions," arXiv:2502.04576, 2025.

ReSQ's distinction. Prior studies investigate when to seek assistance through imitation learning, reinforcement learning, or intervention-cost-aware policies. ReSQ complements this line of research by providing human-written questions grounded in autonomous LLM agent failures, a human-grounded open-ended respondent, and verified post-answer recovery trajectories. This enables the joint study of questioning and recovery, while our Gain/Harm decomposition (Eq. 3) distinguishes rescued from disrupted episodes. Our timing experiments (Table 3) further compare the learned gate against random, token-uncertainty, and KnowNo-style triggers, with additional three-seed evaluations for Qwen3-4B and Qwen3-8B supporting the benefits of learned timing over random timing on both validation splits.

Preliminary RL experiments. We also explored learning questioning and recovery policies through GRPO, starting from SQ and SQR agents. We performed one GRPO iteration using 400 training tasks with six rollouts each (2,400 episodes), rewarding task success, action efficiency, and LLM-judged question usefulness. The following table reports Qwen3-4B results on Valid-Seen (three seeds).

AgentSuccess rate (%)Questions/episode
SQ (SFT)48.5 ± 0.60.76
SQ + GRPO49.2 ± 1.00.82
SQR (SFT)52.4 ± 0.90.31
SQR + GRPO53.3 ± 1.00.31

GRPO provides modest improvements (+0.7/+0.9 percentage points for SQ/SQR), with largely unchanged asking rates. These preliminary results suggest that effective reward design, credit assignment, and interaction-cost optimization remain important research directions. They do not establish the limits of more extensive RL training.

We will strengthen Related Work and clarify ReSQ's contributions relative to existing assistance-requesting policies, together with the additional timing and RL analyses.

Q1. Translation Quality Control

The GPT-4o translations were reviewed by two authors during dataset construction, although individual corrections were not systematically recorded. To further verify translation quality, we conducted an additional audit of all 1,930 Korean-to-English translations (the remaining 230 questions were originally written in English).

An independent LLM compared each Korean–English pair and flagged 44 potential meaning errors. The authors manually reviewed all flagged cases and confirmed 33 errors (1.7% of translated questions). Common issues included translating the Korean loanword for “pan” as “fan” or “pen” (10 cases) and changing the agent's perspective from “I” to “you” (7 cases), potentially altering the intended question.

We will correct the identified errors, release the original Korean questions alongside their English translations, and document this additional quality-control procedure in Appendix A.1.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Author Summary: Contributions and Rebuttal Updates

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their careful evaluations and constructive discussions. As the discussion period concludes, we have uploaded a revised manuscript with changes highlighted in blue. Below, we summarize ReSQ's core contributions and the key concerns addressed during rebuttal for the Area Chair's consideration.

Core Contributions

ReSQ provides a unified benchmark for studying the complete questioning-answering-recovery loop in embodied agents. It links 2,160 failure-grounded questions, 6,480 independently collected human responses, and 514 replay-verified recovery trajectories, together with a fixed response oracle and interactive evaluation protocol. Its contribution lies in this connected resource rather than a particular question-triggering strategy.

Across ten model configurations from four LLM families and three training seeds, question supervision (SQ) improves average task success over success-only training (S) by 4.3 and 2.9 percentage points on Valid-Seen and Valid-Unseen, respectively. Adding recovery supervision (SQR) further improves success over SQ for all ten configurations on both splits, achieving an average gain of 9.0 percentage points over S on each split (Table 1). Our Gain/Harm analysis further distinguishes questions that rescue otherwise-failed tasks from those that disrupt otherwise-successful execution (Figure 3).

Discussion Status & Key Concerns Addressed

We conducted additional experiments, collected new human responses, and revised the manuscript to address the reviewers' concerns. Reviewer UdeU stated that the original concerns were no longer sufficient grounds for rejection and raised their rating from 2 to 3. Reviewer 7psk acknowledged that several concerns were resolved, while identifying two remaining points, which we addressed in our final follow-up and corresponding manuscript revisions.

The key concerns and our corresponding actions are summarized below.

  1. Training reproducibility and translation quality (UdeU, N5UN): We clarified the per-turn preprocessing and loss-masked training procedure with concrete examples, verified that every intended decision was supervised exactly once without truncation, and audited all 1,930 Korean-to-English translations, correcting 33 confirmed errors (Appendices A.1, B).

  2. Oracle validity and answer use (UdeU, N5UN): We collected human responses to 497 agent-generated questions and evaluated the oracle's factuality and response scope against human references, including manual analyses of referent resolution and polarity disagreements. The oracle provided additional information less often than humans, while answer-replacement experiments showed that interaction benefits depend on informative responses (Sections 5.2, 5.4; Appendices E, H).

  3. Questioning behavior and timing (7psk, N5UN): We analyzed 98,464 agent-generated questions across six uncertainty types and extended timing comparisons to three seeds across all four Qwen3 scales (Section 5.2, Table 3; Appendix G). Although the learned gate achieved the highest gain per question in seven of eight model-split settings, we clarified that it does not consistently outperform uncertainty-based triggers in task success and that these comparisons do not isolate the benefit of human-annotated timing supervision (Section 5.3).

  4. Benchmark suitability and external baselines (7psk, N5UN): We strengthened the rationale for using ALFRED (Section 3.2) and further distinguished its task coverage and simulator-grounded execution from ALFWorld (final response to 7psk). We also evaluated ReAct and ReAcTree under the same observation and skill interfaces, explicitly noting their 100-decision limit versus 50 for S/SQ/SQR (responses to N5UN and 7psk). We acknowledge that our empirical findings remain limited to ALFRED with a text-based, skill-level interface (Section 6).

Final Note to the Area Chair

These revisions strengthen ReSQ's empirical support, reproducibility, and clarity while explicitly acknowledging its remaining limitations. We respectfully hope the Area Chair will consider ReSQ's contribution as a reusable benchmark for studying selective questioning, human-grounded answering, and answer-conditioned recovery in embodied agents.

We thank the reviewers again for their constructive feedback and the Area Chair for considering our revised submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.