- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies demand-driven navigation, where an agent must interpret an open-ended natural-language need, locate a satisfying object in an unknown ProcTHOR environment, and terminate with a valid target box. DDSeVA addresses what the authors call the detection-to-success gap through three components: a Qwen-based demand parser and YOLO detector that maintain node, room, and object memories; a Semantic Value Network (SeVN) that ranks heterogeneous search candidates using topology-derived supervision; and a verified acquisition pipeline combining temporal target locking, distance/reachability estimation, and Done success/risk prediction. The method is evaluated on four seen/unseen scene-instruction splits using NSR, SPL, and SSR, with additional component ablations, runtime analysis, failure categorization, and step-budget studies.
Critical strengths
-
The paper targets a consequential failure mode. DDSeVA improves SSR over CogDDN by 10.0, 12.6, 7.8, and 4.2 percentage points across the four splits, directly supporting the motivation for verified stopping.
-
The evaluation includes useful diagnostic analyses. Component ablations, failure categories, runtime, and budget sensitivity expose both search and stopping bottlenecks rather than reporting only aggregate performance.
Critical weaknesses
-
The headline evaluation is internally inconsistent and therefore not fully auditable. The stated split sizes are incompatible with several reported rates, and the aggregate failure count does not reconcile with Table 1. These inconsistencies directly affect the paper's main empirical claim (Section 5.1 and Table 1, p. 7; Appendix E.2–E.4, pp. 21–22).
-
The causal contribution of the proposed design is confounded by supervision and front-end differences. DDSeVA combines a supervised demand parser, YOLO26, an oracle-topology teacher, target-centric oracle samples, and several learned stopping heads. The current controls replace SeVN with nearest-frontier search and verified acquisition with a particularly weak single-frame YOLO-to-Done policy; they do not isolate the effect of the coupled architecture against shared-front-end or matched rule-based alternatives (Section 5.3 and Table 2, p. 8; Appendices D.1–D.3, pp. 18–20). In addition, the problem formulation exposes only RGB observations, while the method requires pose, frontier, room-exit, reachability, graph-distance, and path-cost information whose online source is not fully specified (Section 3, p. 3; Sections 4.1–4.2, pp. 4–6).
Questions for authors
-
Under a shared Qwen/YOLO front end, how do DDN or CogDDN augmented with the proposed verification gate, and DDSeVA with a matched rule-based temporal/distance gate, perform? This comparison would materially clarify which gains come from the coupled design rather than stronger perception or additional supervision (Section 5.3, p. 8).
-
Which simulator signals are available online, and how are pose, room boundaries, frontiers, reachability, path costs, and navigation graphs constructed from the stated RGB/action history? If privileged simulator state is used, do all baselines receive the same information (Section 3, p. 3; Appendix C.2–C.3, pp. 16–17)?
-
Are target-hypothesis candidates ranked by SeVN at inference, or do they bypass SeVN and enter acquisition directly? If they are ranked, how is this reconciled with their exclusion from the prepared SeVN training inputs (Section 4.1, p. 4; Appendix D.2, p. 19)?