acceptodds
Withdrawn from ICLR 2027

DDSeVA: Demand-Driven Semantic Value Navigation with Verified Target Acquisition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Demand-driven navigation extends embodied navigation beyond predefined object search, requiring an agent to satisfy open-ended human needs in unknown environments. Existing methods typically map demands to concrete target objects and employ vision-language reasoning techniques to achieve this. However, correct demand-object grounding does not guarantee reliable task completion. The agent must search effectively before any target is observed and decide whether a detected candidate provides sufficient evidence to terminate, a challenge we term the detection-to-success gap. To bridge this gap, we propose DDSeVA, a closed-loop framework that couples Demand-Driven Semantic Value search with verified target Acquisition. During search, DDSeVA maintains a demand-grounded semantic state and ranks candidate locations according to their expected contribution to task success. During acquisition, each target detection is treated as a hypothesis and verified using temporal consistency, learned reachability estimates, and a prediction of whether Done would satisfy the success criterion. Closed-loop evaluation on ProcTHOR in AI2-THOR demonstrates that DDSeVA consistently outperforms existing methods across all four scene-instruction splits. Compared with the current state of the art, DDSeVA improves average navigation success rate relatively by 10.5%, success weighted by path length by 14.6%, and selection success rate by 30.6%. Code and video demos are available at https://anonymous.4open.science/r/DDSeVA-6675.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Demand-driven navigation, Embodied AI, Semantic Exploration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_twfuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies demand-driven navigation, where an agent must interpret an open-ended natural-language need, locate a satisfying object in an unknown ProcTHOR environment, and terminate with a valid target box. DDSeVA addresses what the authors call the detection-to-success gap through three components: a Qwen-based demand parser and YOLO detector that maintain node, room, and object memories; a Semantic Value Network (SeVN) that ranks heterogeneous search candidates using topology-derived supervision; and a verified acquisition pipeline combining temporal target locking, distance/reachability estimation, and Done success/risk prediction. The method is evaluated on four seen/unseen scene-instruction splits using NSR, SPL, and SSR, with additional component ablations, runtime analysis, failure categorization, and step-budget studies.

Critical strengths
  1. The paper targets a consequential failure mode. DDSeVA improves SSR over CogDDN by 10.0, 12.6, 7.8, and 4.2 percentage points across the four splits, directly supporting the motivation for verified stopping.

  2. The evaluation includes useful diagnostic analyses. Component ablations, failure categories, runtime, and budget sensitivity expose both search and stopping bottlenecks rather than reporting only aggregate performance.

Critical weaknesses
  1. The headline evaluation is internally inconsistent and therefore not fully auditable. The stated split sizes are incompatible with several reported rates, and the aggregate failure count does not reconcile with Table 1. These inconsistencies directly affect the paper's main empirical claim (Section 5.1 and Table 1, p. 7; Appendix E.2–E.4, pp. 21–22).

  2. The causal contribution of the proposed design is confounded by supervision and front-end differences. DDSeVA combines a supervised demand parser, YOLO26, an oracle-topology teacher, target-centric oracle samples, and several learned stopping heads. The current controls replace SeVN with nearest-frontier search and verified acquisition with a particularly weak single-frame YOLO-to-Done policy; they do not isolate the effect of the coupled architecture against shared-front-end or matched rule-based alternatives (Section 5.3 and Table 2, p. 8; Appendices D.1–D.3, pp. 18–20). In addition, the problem formulation exposes only RGB observations, while the method requires pose, frontier, room-exit, reachability, graph-distance, and path-cost information whose online source is not fully specified (Section 3, p. 3; Sections 4.1–4.2, pp. 4–6).

Questions for authors
  1. Under a shared Qwen/YOLO front end, how do DDN or CogDDN augmented with the proposed verification gate, and DDSeVA with a matched rule-based temporal/distance gate, perform? This comparison would materially clarify which gains come from the coupled design rather than stronger perception or additional supervision (Section 5.3, p. 8).

  2. Which simulator signals are available online, and how are pose, room boundaries, frontiers, reachability, path costs, and navigation graphs constructed from the stated RGB/action history? If privileged simulator state is used, do all baselines receive the same information (Section 3, p. 3; Appendix C.2–C.3, pp. 16–17)?

  3. Are target-hypothesis candidates ranked by SeVN at inference, or do they bypass SeVN and enter acquisition directly? If they are ranked, how is this reconciled with their exclusion from the prepared SeVN training inputs (Section 4.1, p. 4; Appendix D.2, p. 19)?

Reviewer_cno6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies demand-driven navigation, in which an agent searches an unknown indoor environment for objects that satisfy a human need expressed in natural language. The setting uses RGB observations. The paper focuses on the detection-to-success gap: detecting an object that fits the demand does not guarantee that the agent approaches it and stops correctly. DDSeVA converts the demand into a set of target categories and builds a semantic state from accumulated spatial and object observations. A learned Semantic Value Network (SeVN) ranks search candidates such as frontiers, room exits, and locations to revisit. Once a target candidate is detected, Verified Target Acquisition combines a Temporal Target Lock, a Distance Head, and a Done Success Head, which assess detection consistency, target distance, and the likelihood of successful termination. These outputs decide whether the agent keeps approaching, observes further, or executes Done.

Critical strengths
  1. Treating a detected target as a hypothesis that is kept and verified through further observations fits the paper's motivation. Keeping the observation history through temporary occlusions or missed detections, and using it for local recovery, approach, and termination, is a practical way to connect detection with task completion.
  2. Table 1 reports state-of-the-art results on all four scene-instruction splits of the DDN benchmark across NSR, SPL, and SSR, with the largest gain in Selection Success Rate (SSR).
Critical weaknesses
  1. Limited evidence for the diagnosis of the central problem and the scope of the contribution The paper presents the detection-to-success gap as a central limitation of prior methods, but it does not report how often DDN and CogDDN fail to approach a detected target to within the success distance or stop incorrectly after a detection. The SSR/NSR ratio in Figure 3 measures selection success conditional on navigation success. It does not measure how often approach fails after detection. The role of the demand and the meaning of verification also need a clearer distinction. The demand is converted into a fixed set of target categories before navigation starts. Verification then evaluates tracking, approach, and termination for proposals in those categories. It does not reassess whether an instance's functions or attributes satisfy the original demand. This design is consistent with the benchmark's success criteria, but the text on continuous demand interpretation and verification should match what is implemented, and the contribution to target acquisition and termination should be stated more precisely.

  2. Unclear consistency between the evaluation assumptions and the provided implementation The paper does not specify what spatial information is available during online navigation. Section 3 describes navigation in an unknown environment from RGB observations, whereas the evaluation procedure in the released code builds a topology in advance with GetReachablePositions and relies on it for frontier candidate generation and connectivity. This does not match Appendix D, which states that simulator topology is used only to construct offline labels. The paper should state what spatial information the reported experiments used and whether the baselines had access to the same information. The correspondence between the defined action space and the actual execution and step accounting is also unclear. The evaluation code moves the agent to the selected node position and yaw/horizon with Teleport and counts policy iterations as steps. Ordinary movement advances one graph edge at a time and SPL accounts for the movement cost, but it is unclear whether the same 100-step budget would hold if the policy executed the movement, rotation, and camera actions defined in Section 3.

  3. Limited controlled comparisons to isolate the contributions of individual components

  • Verified Target Acquisition: The ablation without this module executes Done right after a single-frame detection, which removes any chance to approach a target detected outside the success distance. This shows that continued approach and a proper stopping decision are needed, but it gives little evidence for the added benefit of the proposed verification structure. The individual components are also ablated, but a comparison with an alternative that keeps target approach, such as CogDDN's Exploit mechanism, under the same search and detection conditions would isolate the contribution better.
  • Distance Head: The paper does not describe the decision rules and settings of the 2D bounding box proxy used in w/o Distance Head. A comparison with a simple box heuristic also does not show that the proposed head is better than other ways of estimating distance from RGB. A comparison with a monocular metric depth alternative, using the same target boxes and success distance criterion, would clarify the gains in accuracy and inference time.
Questions for authors
  1. How often does the detection-to-success gap occur in DDN and CogDDN? From existing trajectory logs, what fraction of failed episodes contain a demand-satisfying target that was detected but not acquired?
  2. Have you evaluated a monocular metric depth alternative for target approach and stopping? Specifically, have you tried replacing the Distance Head with metric depth estimates for the detected target and using them to guide approach and stopping?
Reviewer_vjtKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies demand-driven navigation, where an agent must locate an object that satisfies a natural-language demand in an unknown indoor environment and terminate under the required distance, visibility, and bounding-box localization conditions. Motivated by the observation that detecting a target does not guarantee task success, the paper proposes DDSeVA, which combines semantic search before target discovery with termination verification after detection.

The framework uses an adapted Qwen model to produce a cached demand profile and an adapted YOLO detector to supply observations for semantic memory. A Semantic Value Network (SeVN) ranks heterogeneous spatial candidates. Once a demand-relevant proposal is detected, temporal target locking, distance and reachability estimates, and termination success and risk predictions guide approach, local recovery, or Done through a rule-based controller. The reachability labels indicate whether the target is within the actionable distance range (Appendix D.3).

Experiments cover four scene–instruction splits in AI2-THOR/ProcTHOR, with 500 episodes per split and a default budget of 100 steps. Compared with CogDDN, the paper reports improvements in average NSR, SPL, and SSR from 35.9%, 17.2%, and 28.3% to 39.7%, 19.7%, and 36.9%, respectively. Additional evaluations include five ablation variants, a runtime comparison with a reproduced CogDDN pipeline, failure-mode analysis, and step-budget experiments.

Critical strengths

S1. The paper addresses a practically meaningful problem. Even after identifying a demand-relevant object, an agent may fail because of distance, viewpoint, or bounding-box localization errors. Target tracking, continued approach, and termination prediction directly address these issues. In particular, the Done Success Head is supervised using the simulator's actual termination criteria, rather than merely target presence, providing a clear connection between the verification module and the benchmark objective.

S2. The four evaluation splits cover both scene and instruction generalization, and Table 1 reports higher scores than the listed baselines across all three metrics in each split. Ablations are evaluated on all four splits. Appendix E.4 reports an analysis of 1,219 failed episodes and examines how the failure distribution changes with larger step budgets. These experiments support the system's effectiveness on this benchmark and provide more insight than aggregate success rates alone.

Critical weaknesses

W1. The connection between SeVN's distance-based supervision and its claimed success-oriented ranking needs stronger justification. For a fixed state, the teacher in Eq. (5) minimizes travel cost plus remaining topological distance to a success region; the auxiliary value target is also distance-derived. These are reasonable navigation proxies, but they do not explicitly account for target observability or subsequent verification difficulty.

W2. Comparative diagnostics of the detection-to-success gap remain limited. Table 2 supports the usefulness of the acquisition modules, but does not quantify which post-detection failures they reduce. SSR/NSR measures selection success conditional on navigation success, not completion conditional on target detection. Appendix E.4 analyzes DDSeVA's failures without a corresponding baseline or ablation breakdown. Comparing post-detection completion and stopping errors from matched detection states would more directly isolate improvements in acquisition and termination from differences in search.

Questions for authors
  1. Basis for the search value: What evidence shows that SeVN's distance-based supervision supports ranking candidates by eventual task completion, particularly when candidates differ in target observability or verification difficulty?

  2. Independent contribution of the mechanism: Could you compare against a simpler acquisition strategy that retains target approach and combines a distance threshold with confirmation across consecutive frames, using the same demand parser and detector? This would help assess the added value of the proposed verification pipeline.

  3. Direct validation of the central claim: Could you compare post-detection completion rates and rates of premature stopping and missed stopping opportunities against baselines or ablations, ideally from matched detection states, to separate acquisition improvements from differences in search?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.