acceptodds
Under review as a conference paper at ICLR 2027

Hypothesis-Induced Action Fields for Object Localization in Partial Scenes

Abstract

Object localization in partial scenes requires inferring the position of an unobserved object from incomplete spatial and semantic context. Commonsense relations provide contextual information about potential target locations, but incomplete observations may remain compatible with several spatially separated alternatives. Compressing these alternatives into point estimates can obscure the underlying ambiguity, and aggregating these estimates may yield a prediction with limited support from the observed context. We introduce Hypothesis-Induced Action Fields (HIAF), a method that couples the representation of spatial alternatives with the localization decision. Spatial hypotheses derived from observed objects jointly define an action field that estimates the probability of successful localization at different locations. Learning this field aligns hypothesis generation and scoring with the localization objective, allowing alternatives to inform the final prediction collectively. On Partial ScanNet, HIAF achieves the highest mean success rates and lowest mean successful localization errors among the evaluated methods across three evaluation protocols. At 1 m, query-level success reaches 30.06%, exceeding the strongest evaluated baseline by 4.86 percentage points. At the stricter 0.5 m tolerance, HIAF achieves 13.19% query-level success, compared with 9.54% for the strongest baseline at that tolerance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
object localization, scene understanding, spatial reasoning, graph neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_T8fMOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Hypothesis-Induced Action Fields (HIAF) for localizing an unobserved object in a partial scene. Instead of minimizing coordinate error, HIAF estimates the probability that a candidate location falls within the task's success radius of any hidden target. Anchor-conditioned Gaussian hypotheses define a continuous action field, and a full-plane binary cross-entropy objective aligns training with localization success. Experiments on Partial ScanNet show consistent gains over commonsense-graph baselines.

Critical strengths
  1. The estimation target is well motivated: predicting success probability within the evaluation radius directly addresses multimodal ambiguity and aligns the learning objective with the actual decision criterion more closely than squared-error coordinate regression.
  2. The mathematical development and protocol are careful, including the full-plane loss derivation, multiple evaluation protocols/tolerances, three-seed reporting, and detailed reproducibility material.
Critical weaknesses
  1. The paper's own ablations weaken the architectural claim. Changing the number of hypotheses per anchor from K=1 to K=2 or K=4 barely changes performance, and adding hypothesis interaction does not improve the primary 1 m metric and hurts at 0.5 m. The evidence suggests that most of the gain may come from the success-aligned objective rather than the hypothesis-generation machinery.
  2. There is no sufficiently simple control using the same success-aligned objective. A single probabilistic head, mixture-density head, or other multi-hypothesis baseline on the same D-SCG encoder would reveal whether the action-field construction is actually necessary. Without this, the paper does not isolate objective novelty from architectural novelty.
  3. The Bayes-action interpretation is stronger than the implemented inference rule. The theoretical optimum is defined over the continuous plane, but inference only selects among generated hypothesis centers. The paper itself notes that the finite Gaussian family need not contain the population optimum and that integrated risk does not guarantee correct ranking at the candidate centers.
  4. The practical computational cost is missing. Scoring requires H² disk integrals and the loss adds LH integrations, with adaptive quadrature up to order 256 in double precision. No latency or memory comparison against the baselines is reported, so the practicality of the method is unclear.
  5. The experimental setting remains highly idealized: only Partial ScanNet is used, inputs are oracle object classes/ground-plane positions rather than noisy perception outputs, and every query is guaranteed to have at least one hidden instance. These assumptions leave robustness to perception errors, absent targets, and dataset shift untested.
Questions for authors
  1. If K=1 performs almost the same as K=2, what does per-anchor hypothesis multiplicity actually contribute? Please compare against a much simpler probabilistic head trained with the same full-plane BCE objective.
  2. Why is the final point selected only from generated centers instead of maximizing the continuous action field? How often does this restriction change the selected location and LSR?
  3. Can the authors report inference latency and memory relative to D-SCG-OL/Graphormer, including the cost of double-precision adaptive quadrature?
  4. Can the method handle absent target classes and noisy observed object positions/classes, and can the authors provide an experiment beyond oracle Partial ScanNet inputs?
Reviewer_DwfYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces HIAF for localizing unobserved objects in partial scenes by combining Gaussian hypotheses into a continuous success-probability field. A tolerance-aware objective supervises the field using all hidden target instances, and the highest-scoring candidate is selected at inference.

Critical strengths

The paper is mathematically well defined, with a clear problem formulation. Its objective is well motivated by the mismatch between coordinate regression and tolerance-based evaluation. HIAF consistently improves over baselines on the Partial ScanNet benchmark.

Critical weaknesses

[Limited Practical Applicability and Unverified 3D Extension] Restricting localization to the ground plane discards vertical information: an object on a shelf and one on the floor may share identical projected coordinates—limiting the usefulness of predictions for height-sensitive tasks (I believe the majority of real-world tasks need to account for the z-axis). Further, while D-SCG-OL evaluates 3D localization and reports a performance decrease, HIAF is evaluated only in 2D. Replacing 2D Gaussians and disks with 3D Gaussians and balls appears possible, but the resulting accuracy and computational cost remain unverified. [Insufficient Discussion of Related 3D Approaches] Recent work such as [1] addresses analogous unobserved-object localization problems in 3D. The paper should more clearly distinguish HIAF’s position from these approaches and explain the practical advantages of its 2D formulation. [1] Believing is Seeing: Unobserved Object Detection using Generative Models (CVPR 2025). [Alternative Solution] Figure 1(c) illustrates a well-known limitation of direct averaging: combining spatially separated predictions can produce an unsupported intermediate location. I am curious whether a simpler aggregation solution could address this issue. A useful comparison would be to evaluate HIAF against variants of D-SCG-OL that replace mean pooling with alternatives such as selecting a representative prediction from the dominant spatial cluster or using the medoid. Unlike the mean, the medoid selects an existing prediction rather than introducing a new intermediate point, potentially offering a simpler solution to the problem identified in the paper. [Regarding Figure 3] The qualitative results in Figure 3 (a) and (c) should also show the D-SCG-OL candidates (purple).

Questions for authors

Please refer to the Weaknesses section. My main questions concern (1) the practical benefits (beyond simplicity) of the 2D ground-plane formulation relative to related 3D approaches; (2) whether HIAF can be extended effectively to 3D and how this would affect accuracy and computational cost; and (3) whether a simpler solution, such as medoid, could address the identified averaging failure and achieve comparable improvements.

1 comment
Reviewer_DwfYOpenReview ↗

Thank you for the rebuttal

Thank you for clarifying the scope of your work. However, my concern regarding W2 remains. Although D-SCG-OL also primarily focuses on 2D localization, its authors include a 3D analysis to examine its applicability beyond the ground plane. I understand that their 3D evaluation protocol and code may not be available; nevertheless, an independent 3D evaluation of HIAF would help assess its accuracy and computational cost, strengthening the completeness of the empirical evaluation and substantiating the proposed extension.

Without such an analysis, the empirical evaluation remains limited to 2D, and the feasibility of extending HIAF to 3D remains unverified despite the theoretical formulation provided by the authors. Therefore, I maintain my current score.

Reviewer_8it5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies object localization in partial scenes, which means inferring where an unobserved object is from the objects that are visible. Since regressing coordinates can place predictions between several plausible regions, the paper instead estimates the probability that a prediction succeeds within the evaluation tolerance. The proposed method, HIAF, generates Gaussian hypotheses anchored at observed objects and combines them into a continuous success-probability field, which is trained end to end and used to pick the final location. Experiments on Partial ScanNet show consistent improvements over prior methods.

Critical strengths
  • The motivation is clear and well grounded. Making the training target the success probability under the actual evaluation criterion is principled, it is the Bayes action for the task, and it naturally covers queries with several hidden instances.
  • The technical formulation is elegant. Using an exponential link lets the background term of the full-plane loss be computed analytically, the importance-weighted sampling over overlapping target disks is unbiased, and the numerical integration is carefully implemented with derivations provided.
  • The evaluation is careful. It uses three protocols, multiple tolerances, a single- versus multi-instance breakdown, and three seeds with standard deviations. The improvements over all baselines are consistent, and code is provided.
Critical weaknesses
  • The decision rule doesn't fully match the Bayes-action motivation. The paper motivates HIAF as maximizing success probability over the plane, but inference only picks among the generated hypothesis centers, not the field's actual maximum. The estimated probabilities are never checked for calibration, so it's unclear whether the field really approximates success probability or simply works as a ranking score.
  • The core claim about ambiguity isn't directly verified. The paper argues that its gains come from representing several separated plausible regions instead of averaging them, but the only evidence is three hand-picked qualitative examples. There's no quantitative analysis of whether the hypotheses actually capture distinct modes, or of how much of the gain comes from queries that really are ambiguous.
  • The evaluation setting is idealized. The model receives ground-truth object positions and class labels, and every query is assumed to have at least one hidden instance. All experiments use a single dataset.
  • Sensitivity to the tolerance and missing-cost analysis. Training and scoring use a fixed 1 m radius, and the paper doesn't study models trained with other radii, so it's unclear whether a new model is needed for each tolerance.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.