acceptodds
Under review as a conference paper at ICLR 2027

Binocular Focused Vision: Task-Query-Driven Depth Estimation and Spatial Relation Modeling for Embodied Intelligence

Abstract

Modern stereo matching achieves high-quality dense metric depth, yet its prevailing formulation remains task-agnostic: all pixels are treated as equally important. In many query-driven applications, however, the location of interest is known in advance, motivating geometry estimation that is explicitly conditioned on where accuracy is needed. We introduce Binocular Focused Vision and propose FocusStereo, a unified framework for query-conditioned stereo geometry and spatial reasoning. Given rectified stereo images, camera calibration, and a spatial query, FocusStereo conditions stereo correspondence formation on the query, reconstructs local matching distributions, decodes bounded metric geometry, selectively integrates reliable updates, and derives query-centered 3D relations. Under a unified query-centric protocol on VKITTI2 and HAMMER, FocusStereo reduces Focus AbsRel by 65.17% and 27.04% and Query-XYZ error by 61.13% and 15.37%, respectively, relative to the strongest competing result for each metric. Controlled experiments with identical initialization further isolate these gains to query-conditioned reasoning, with only minor changes in global error. These results show that spatial queries can effectively condition stereo correspondence, extending stereo perception from task-agnostic dense reconstruction to query-conditioned geometry and spatial reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Binocular vision, focused vision, task query, stereo depth estimation, spatial relation modeling, embodied intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_e8f4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FocusStereo, a stereo depth estimation framework based on point queries. The method employs point queries to guide stereo feature matching, refines local depth through disparity residual prediction and reliability-guided fusion, and further computes 3D spatial relationships in the vicinity of target points. Experiments conducted on the VKITTI2 and HAMMER datasets validate the method's performance in both local depth estimation and 3D localization.

Critical strengths

This paper explores the use of point queries to improve the accuracy of local depth estimation, offering notable practical value. The proposed framework integrates query-conditioned stereo matching with reliability-guided depth refinement, and has demonstrated strong experimental results on the VKITTI2 and HAMMER datasets.

Critical weaknesses

The paper lacks concrete downstream applications to justify the necessity of the Point Query task formulation. Furthermore, the experiments fail to compare the proposed approach against conventional depth refinement methods under identical initialization conditions, making it difficult to fully validate the specific contribution of Query Conditioning. The discrepancies in the results presented in Table 1 and Table 2 are not clearly explained, and the inclusion of certain modules even leads to a decline in depth estimation accuracy.

Questions for authors
  1. Research motivation. The authors focus on local depth accuracy rather than global depth estimation. While this idea is intuitively reasonable, the paper does not clearly explain which specific tasks require point queries to obtain more accurate depth. The proposed setting seems mainly motivated by CV benchmarks, without concrete examples of its necessity in real applications. For instance, under what conditions would local navigation require this setting instead of conventional depth estimation or simply using depth cameras or LiDAR? Methods such as FoundationStereo address general-purpose depth estimation and support applications such as 3D reconstruction and data annotation, making their problem setting clear. In contrast, the proposed setting appears more task-specific. For such tasks, would adding depth sensors or LiDAR be a more direct solution? I would like the authors to justify the advantages of Point Query over conventional global depth estimation or refinement from the perspective of the task setting rather than the model design. Additionally, although global depth metrics are reported, the lack of separate evaluation on non-query regions makes it unclear whether improving local accuracy affects depth estimation elsewhere.

  2. Experimental settings. In Section 4.4, the authors state: "The ablation results are obtained from a separate controlled run and are therefore used only for within-table comparisons across model variants." Does this mean that Table 1 and Table 2 use different experimental settings? If the same model and test data are used, why are separate runs necessary, and why are the results not directly comparable? For example, the Query-XYZ error of the full model on VKITTI2 is 0.46776 in Table 1 but 0.358675 in Table 2. The authors should clarify the differences between these experiments and explain the discrepancies.

  3. Insufficient validation of Query Conditioning. Although the authors conduct ablations with a fixed FoundationStereo initialization and show improvements from additional refinement modules, they do not compare against conventional refinement without Point Query. Therefore, it remains unclear whether the gains come from Query Conditioning or simply from additional stereo matching and refinement. Further experiments are needed to isolate the contribution of Query Conditioning.

  4. Lack of comparison with D3RoMa and other depth refinement methods. D3RoMa uses existing depth or disparity estimates to guide depth restoration and could potentially be applied to this setting. For example, would D3RoMa achieve comparable results if initialized with depth predicted by FoundationStereo? The authors should consider this comparison, with appropriate input adaptation and training settings, to better demonstrate the advantages of their method.

  5. Insufficient validation of individual modules. On HAMMER, adding RGGF increases Query-XYZ error from 0.015995 to 0.031453, nearly returning to the initialization baseline, while Focus AbsRel worsens from 0.037934 to 0.045398. Although the authors attribute this to an accuracy–reliability trade-off, they do not sufficiently justify the accuracy loss or demonstrate the advantages of RGGF over simpler confidence-based filtering. Moreover, GARD mainly relies on standard 3D back-projection and depth-based geometric relations, while providing only marginal improvements in Topology F1. The authors should further justify the trade-off introduced by RGGF and compare both modules with simpler alternatives to demonstrate their necessity.

Reviewer_6ityOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes FocusStereo, the method uses a point query to refine stereo depth around a location of interest. Starting from FoundationStereo predictions, FocusStereo estimates local corrections, selectively applies them, and derives relative 3D geometry. The method is evaluated on VKITTI2 and HAMMER.

Critical strengths

Improving geometry around a localized region is a reasonable goal for applications such as manipulation or measurement.

Critical weaknesses

W1: The benefit of query-conditioned correspondence is not isolated. FocusStereo uses FoundationStereo initialization, but FoundationStereo alone is absent from the main comparison. The ablations add query conditioning and learned refinement together, without a matched refiner that removes query conditioning. It therefore remains unclear whether the proposed formulation offers an advantage over simply refining depth around the requested location.

W2: The presentation feels unfinished. The paper provides only two qualitative examples per dataset, without ground truth, error maps, or a meaningful discussion of failure cases. The writing is often vague and repetitive, and the bibliography has inconsistent formatting and incomplete publication details. No supplementary material is provided to resolve the missing technical details, even though the main paper leaves part of the available page budget unused.

W3: The method lacks essential implementation and evaluation details. The learned modules are described at a high level, and Eq. (14) largely lists losses without defining them. Training data and splits, correspondence search ranges, query sampling, and evaluation-region construction are also unspecified. These omissions make it difficult to understand the actual algorithm, reproduce the results, or assess whether the comparisons are fair.

Questions for authors

Q1: What does FoundationStereo alone achieve under the main evaluation protocol, and how much does query conditioning add over an otherwise identical local refiner?

Q2: What data and query sampling procedure are used for training? Please specify the learned modules, losses, and evaluation regions sufficiently detailed.

Q3: What is a concrete application where query-conditioned correspondence offers a clear advantage over a coarse global depth estimate combined with local high-resolution estimate?

Reviewer_5oWxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FocusStereo addresses stereo geometry estimation conditioned on a requested image location. Starting from an initial stereo estimate, it modulates binocular features using the query, predicts bounded disparity corrections, selectively incorporates reliable updates, and derives query-relative 3D geometry. Experiments on VKITTI2 and HAMMER evaluate focused accuracy, global preservation, and reliability.

Critical strengths
  1. The formulation is motivated by applications requiring accurate geometry around a specified location. The query-based correspondence computation is straightforward and analytical back-projection keeps the geometric outputs tied to calibrated depth.
  2. The fixed-initialization ablation provides evidence of incremental improvement, particularly in focused depth on HAMMER. The authors also acknowledge the accuracy–risk trade-off introduced by selective fusion.
Critical weaknesses
  1. The novelty is insufficiently distinguished from relevant prior work.

ChiTransformer [1] already develops feature-sensitive cross-view retrieval and confidence-controlled residual feature updates. Although external spatial conditioning and disparity-level fusion distinguish FocusStereo, this substantial conceptual overlap is not discussed. The contribution appears to lie primarily in external query conditioning and the integration of these components, but the additional insight beyond the established mechanisms remains insufficiently explained.

  1. The evidence does not adequately attribute the improvements to query-conditioned correspondence.

FocusStereo uses FoundationStereo initialization, but Table 1 omits FoundationStereo alone while the compared stereo baselines use SceneFlow-trained weights. Table 2 controls initialization in a separate run: the full model improves Query-XYZ by approximately 3% on both datasets; HAMMER Focus AbsRel improves by 13.7%, while Relative-XYZ worsens by 11.3%.

Adding Q-SER and FGD together also does not isolate query conditioning from additional refinement capacity and training. Qualitatively, Figure 3’s panels are too small to clearly assess the claimed improvements in boundaries, thin structures, and local geometry.

  1. Important implementation and evaluation details are missing.

Training datasets and splits, sample/query counts, evaluation-region sizes, and individual loss terms and weights are insufficiently specified. Runtime and memory consumption are also unreported despite the interactive and robotic motivation. These omissions leave reproducibility and practical applicability unresolved.

[1] Qing Su and Shihao Ji. “Chitransformer: Towards Reliable Stereo From Cues.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1939–1949, 2022.

Questions for authors
  1. Please discuss the relationship to ChiTransformer’s retrieval and confidence-controlled update mechanisms [1]. What substantive insight does external spatial conditioning add?

  2. What are FoundationStereo’s results under the exact Table 1 protocol, and what differs between the main and controlled runs? Particularly, which evidence isolates query conditioning from additional refinement?

  3. Please provide readable, enlarged patches comparing FocusStereo with the baselines, including its FoundationStereo initializer. Mark the corresponding regions in the input image and use consistent depth scales. Especially, ground-truth depth error maps are commonly used in the community to help establish whether the visible differences reflect improved accuracy. A supplementary gallery would be suitable.

  4. Please specify the training and evaluation splits, query-selection procedure and region sizes, and essential loss and calibration settings. How is Harm Rate defined, and what fraction of proposed updates is accepted?

  5. What are the end-to-end latency and peak GPU memory, including initialization, at a stated resolution and hardware configuration? What is the incremental cost of another query on the same stereo pair?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.