acceptodds
Under review as a conference paper at ICLR 2027

GLANCE: Free-Form 3D Object Identification through Active Geometric Inspection

Abstract

Identifying an arbitrary 3D object from geometry alone is challenging: a single viewpoint may be insufficient, and geometric identity need not admit a unique textual ground truth. We study this problem as free-form 3D object identification: inferring an object’s identity without color, texture, physical scale, canonical orientation, scene context, or a candidate vocabulary. We introduce GLANCE, an active recognition approach in which an off-the-shelf multimodal large language model (MLLM) iteratively inspects rendered geometric observations. Rather than committing to an initial guess, GLANCE maintains competing free-form identity hypotheses, predicts geometric evidence that would distinguish them, and selects subsequent viewpoints to test these predictions. No task-specific 3D training or candidate vocabulary is required. To evaluate free-form identification without imposing a predefined taxonomy, we construct a benchmark of 1,118 3D shapes, each independently identified by 4 human annotators together with their confidence score. We retain these annotations as multiple reference identities rather than collapsing them into a single ground truth, and evaluate predictions through semantic agreement with the references. In controlled experiments, GLANCE improves recognition over passive multi-view baselines while requiring fewer observations. At larger scale, GLANCE achieves 83.2% primary accuracy on 570 benchmark objects. On ShapeNet, GLANCE achieves 72.0% accuracy while keeping the category vocabulary hidden from the recognition model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
open-vocabulary 3D recognition, active perception, multimodal large language models, 3D shape understanding, zero-shot recognition, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JjEsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new benchmark and a VLM-based method for vocabulary-free texture less 3D object identification using active inspection. The model iteratively selects the next viewpoint based on previous observations and predicts discriminative geometric features that can support or reject competing identity hypotheses.

Critical strengths
  1. The benchmark targets a new task/setting: free-form identification of isolated 3D objects from multiple images from geometry alone without texture or environment cue.
  2. The method seem to give advantage compare to random sampling and single view point inspection.
Critical weaknesses

A major weakness is the limited set of baselines used to establish the benefit of the proposed active-inspection strategy. The paper compares against single-view recognition, random multi-view sampling, and an ablation in which the VLM selects views without explicit hypothesis testing. However, several obvious alternatives are missing. First, a deterministic geometric baseline using mutually orthogonal or approximately uniformly distributed viewpoints would provide a stronger comparison than random sampling. Second, the authors should evaluate a fixed multi-view baseline in which the VLM receives several images with viewpoints simultaneously . Third, a free form agentic baseline, in which the VLM is given a viewpoint-selection tool and independently decides where to look and when to stop with only basic prompt, would help establish whether GLANCE's explicit hypothesis-generation and verification procedure provides an advantage over generic VLM agent behavior. Given that active inspection is a central methodological contribution, these comparisons are essential for justification for the proposed strategy.

In addition, the recognition experiments are conducted with only a single proprietary VLM (GPT-6 Astra), while another proprietary model (GPT-5.6 Luna) is used for clustering and semantic evaluation. This makes it unclear whether the benefits of GLANCE generalize across different VLMs. Moreover, the experiments rely on hosted model aliases whose underlying snapshots may change or completely phased out by the time the paper published, which limits reproducibility. Evaluating GLANCE with several different VLMs, including open-weight models, would substantially strengthen both the generality and reproducibility of the results.

Questions for authors

See weakness section

2 comments

We thank the reviewer for the constructive comments and suggestions regarding stronger passive baselines, generalization across different VLMs, and reproducibility.

Passive multi-view baselines

To further investigate the contribution of active inspection beyond the sequential baselines presented in Sec. 5.2, we implemented passive multiview baselines in which the same VLM receives 1, 3, or 6 rendered viewpoints simultaneously in a single call.

Our preliminary experiments on the Fusion 360 Assembly dataset showed substantial sensitivity to viewpoint selection, consistent with the motivation in Sec. 3.1. Among the tested three-view combinations, accuracy ranged from 40% to 80.7%, depending on the selected viewpoints. These results highlight the importance of viewpoint selection and motivate GLANCE's hypothesis-guided inspection strategy (Sec. 3.3).

Robustness to object orientation

As discussed in Sec. 3.1, CAD models do not necessarily have a canonical orientation. We therefore evaluated GLANCE and passive multiview baselines under upright, upside-down, and randomly rotated configurations, keeping the passive camera configurations fixed.

Recognition accuracy (%):

MethodUprightUpside-downRandom
GLANCE71.1668.872
Passive-380.7057.750
Passive-683.572.965

Under upright orientation, Passive-3 and Passive-6 outperform GLANCE. However, under upside-down orientation, GLANCE achieves 68.8% accuracy, compared with 57.7% for Passive-3. Under random rotations, GLANCE achieves 72%, outperforming both Passive-3 (50%) and Passive-6 (65%). These results suggest that GLANCE is less sensitive to object orientation, particularly when the canonical orientation is unknown. Moreover, GLANCE adaptively selects observations and requires only 2.63 views on average (Sec. 5.2), whereas the passive baselines always process three or six views. This highlights the potential of active inspection to maintain recognition accuracy while reducing the number of observations.

Generalization across recognition VLMs

We conducted preliminary experiments using Qwen3.8-Max on a limited set of 20 objects. Although this sample size is too small to draw definitive conclusions, the results were generally consistent with those obtained using GPT. However, we did not extend the evaluation to more objects for the following reasons.

The GLANCE framework remained identical for both models, including the view selection procedure, geometric hypothesis verification, and overall decision-making pipeline. The main difference was in how the underlying models reasoned when prompted to perform these tasks. Specifically, Qwen generated approximately 15,000 tokens on average to reach a prediction, compared with around 1,121 tokens for GPT. This significantly increased inference time, making Qwen considerably slower, despite both models following the same GLANCE procedure. We also experimented with reducing Qwen's token budget, but this led to a degradation in performance. Given these computational costs and the lack of clear performance improvements in our preliminary experiments, we decided to use GPT for the main evaluation rather than extending the experiments to additional models.

Reproducibility

We will release the complete GLANCE implementation, including all prompts, viewpoint configurations, rendering parameters, hypothesis management, geometric verification, and stopping criteria. We will also document the model identifiers, API configurations, inference parameters, software dependencies.

I thank the authors but this don't really answer any of my concerns.
I maintain my rating.

*Mutually orthogonal meaning 90 angles between camera position and not dependent on object original orientation, same for letting LLM chose viewpoints.

*There plenty open and close models cheaper then Qwen3.8 Max which is both expensive and closed source.

Reviewer_VLj6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GLANCE asks an off-the-shelf MLLM (GPT-6 Astra) to name an untextured, arbitrarily oriented CAD shape without a candidate vocabulary. The shape is shown as a point cloud colored by surface normals from up to 5 of 14 fixed viewpoints, together with its bounding-box aspect ratios. At each step the MLLM gives four free-form hypotheses with confidences, a second model (GPT-5.6 Luna) clusters the strings, and the MLLM names a feature that should be visible if its leading hypothesis is right and picks the next view; on that view it reports the feature as present, absent or undetermined, which adds +1, -1 or 0 to the hypothesis. Recognition stops when the top cluster score is at least 1.5 times the runner-up.

Critical strengths

The benchmark and evaluation protocol are the strongest part. Keeping four independent identities with confidences instead of forcing a taxonomy, and showing that the vocabulary keeps growing (644 distinct strings, 45% of them used once), is a sound answer to the fact that geometry often has no unique name. The judge is validated against human labels with kappa reported, and model accuracy tracks human consensus (91.2 / 76.4 / 68.3 primary accuracy for high / medium / low consensus). The paper also states unflattering results plainly, for example, that random views match GLANCE on strict accuracy.

Critical weaknesses

The evidence that active hypothesis testing helps is too thin to support the method claim. The controlled comparison is 100 objects in one stochastic run: primary accuracy differs by three objects (76 vs 73), strict accuracy favors random views (69 vs 68), and in your own validation, the judge accepts 5.5 points more often than the human annotator. The view savings are not like-for-like either, because only GLANCE adds the self-graded verification term to the leading hypothesis, so its stopping rule fires sooner whenever those verdicts lean positive.

The comparisons that would place the method are missing: no passive baseline that shows the same MLLM all views in one call, no point-language model such as PointLLM on the same clouds, and no citation of or comparison with classical active recognition (e.g., next-best-view in 3D ShapeNets, Wu et al., CVPR 2015; Jayaraman and Grauman, ECCV 2016). With a single proprietary model and only 570 of the 1,118 shapes evaluated, it is unclear how far the result generalizes.

Questions for authors
  1. Strict, primary, and near-match accuracy after exactly 2, 3, 4, and 5 views for single, random, no-test, and GLANCE, over several runs, with a paired test.
  2. The distribution of present/absent/undetermined verdicts over all steps, and how often "present" occurs when the final answer is wrong.
  3. The same MLLM shown all 14 views, or a grid of the six face views, in one call.
  4. Why 570 of the 1,118 shapes? Who are the four annotators, how were they recruited and instructed, and are any of them authors?
  5. How often are GLANCE's and the random baseline's predictions accepted only as BROADER? A few examples of the broadest accepted answers would help judge whether the primary criterion rewards hedging.
1 comment

We thank the reviewer for the detailed and constructive comments.

Geometric verification and stopping behavior

We analyzed 1,473 observation steps across 568 objects. Since the initial observation has no prior geometric expectation to verify, we considered the remaining 905 verification steps. Among these, 420 (46.4%) were PRESENT, 204 (22.5%) ABSENT, and 281 (31.0%) UNDETERMINED.

VerdictFinal correctFinal incorrect
PRESENT34674
ABSENT13767
UNDETERMINED20972

PRESENT occurred in 50.0% of verification steps associated with correct final predictions, compared with 34.7% for incorrect predictions. This suggests that positive geometric verification is associated with successful identification, providing additional empirical support for the hypothesis-testing mechanism described in Sec. 3.3.

Evaluation criteria and BROADER predictions

BROADER-only predictions account for 10.8% accepted GLANCE predictions and 5.1% accepted random-view predictions. We further investigated this issue on 673 GLANCE predictions. Primary accuracy is 78.8% (530/673), compared with 71.9% (484/673) under strict SAME. The difference consists of 46 BROADER-only cases.

For example:

Reported labelHuman annotationModel's leading label
AircraftHelicopterHelicopter
Hand toolsScrewdriverScrewdriver
Musical instrumentsGuitarGuitar
FigurineElephantElephant figurine
FanCeiling fanCeiling fan
CarRace carToy race car

The model's leading label was specific in 45 of the 46 cases. These findings suggest that many BROADER final predictions may arise from semantic aggregation and representative selection rather than deliberate hedging by the recognition model. Nevertheless, we acknowledge that aggregation can reduce the specificity of the final reported identity.

Passive multi-view comparison

We conducted additional experiments where multiple rendered viewpoints are provided simultaneously to the same VLM in a single call. We evaluated the methods on under upright, upside down, and randomly rotated object orientations.

Recognition accuracy (%):

MethodUprightUpside-downRandom
GLANCE71.1668.872
Passive-380.7057.750
Passive-683.572.965

These results suggest that active view selection can help GLANCE compensate for unfavourable initial orientation, by identifying additional viewpoints that reveal discriminating geometric features, whereas a passive baseline is stuck with what it was given. We acknowledge that GLANCE uses adaptive stopping, so these comparisons do not independently isolate viewpoint selection under matched observation budgets.

Dataset and annotation procedure

As detailed in Sec. 4, the annotations were collected independently by four annotators (three authors and one volunteer), without predefined categories or access to one another's responses. We retained objects for which all four annotators provided identity labels, without requiring agreement. The 570-object evaluation subset is described in Sec. 5. Importantly, the annotations were collected before GLANCE was developed, ensuring that its predictions could not influence the annotation process.

Reviewer_p9ZgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates free-form 3D object identification from geometry alone and proposes GLANCE, a framework that uses a multimodal large language model (MLLM) to generate competing identity hypotheses, actively select viewpoints, and verify geometric evidence to identify objects. The authors also construct a benchmark containing 1,118 3D objects with multiple independent human-generated identity annotations. Experiments show that GLANCE achieves modest improvements in recognition accuracy over random multi-view observation while requiring fewer viewpoints.

Critical strengths
  1. The paper constructs a 3D object benchmark with multiple independent human identity annotations and introduces an evaluation protocol that accounts for semantic ambiguity, providing a valuable foundation for evaluating free-form 3D object identification.
Critical weaknesses
  1. Insufficient task motivation and unclear research gap.

The paper does not adequately establish the significance of the proposed task, its core challenges, or the research gap it aims to address. Although the authors repeatedly emphasize the difficulty of free-form 3D object identification from geometry alone and review numerous related methods, three fundamental questions remain insufficiently addressed:

Why this task? Why is it necessary to study 3D object identification while simultaneously excluding texture, color, physical scale, scene context, and a predefined candidate vocabulary? What practical applications motivate this particular setting, and what is the research or practical value of solving this problem?

Why is this problem not adequately solved? What are the fundamental limitations of existing 3D recognition, open-vocabulary recognition, and active viewpoint selection methods? The authors mainly describe their reliance on candidate vocabularies, specialized 3D encoders, or predefined viewpoints, without sufficiently explaining why these approaches fail to meet practical needs or what critical research gap remains unresolved.

Why GLANCE? Why does addressing this problem require competing hypothesis generation, geometric evidence prediction, and active verification? The paper does not derive the necessity of these components from specific failure modes of existing methods. Instead, it moves directly from stating that the task is challenging to proposing active observation with an MLLM, without sufficient methodological motivation.

Suggestion for improvement: I encourage the authors to substantially revise the motivation in the Introduction, establishing a clear logical progression from the value of the task and the limitations of existing research to the specific research problem and proposed method. This would significantly improve the paper's positioning, overall quality, and persuasiveness.

  1. Insufficient validation of the core method.

The core contribution of the paper is active viewpoint selection and geometric hypothesis verification, but the experiments do not sufficiently demonstrate their effectiveness. GLANCE improves primary accuracy only from 73% to 76% compared with random multi-view observation, while its strict SAME accuracy is slightly lower (68% vs. 69%). The existing ablations do not clearly isolate the individual contributions of active viewpoint selection and geometric hypothesis verification, and the larger-scale evaluation on 570 objects lacks corresponding baseline comparisons. Therefore, the current experimental evidence is insufficient to substantiate the claimed advantages of the core method.

Questions for authors

Under the same observation budget, what are the individual contributions of active viewpoint selection and explicit geometric hypothesis verification? Can the authors provide corresponding baseline comparisons on the 570-object evaluation?

2 comments

We thank the reviewer for the constructive comments. Our work is motivated by large CAD repositories in which detailed geometry is available but reliable object-level identities may not be. For example, the Fusion 360 Gallery dataset contains designer-created CAD models and information about modeling operations, but does not provide curated object-identity categories (Willis et al., 2022). Identifying objects directly from geometry would make such repositories easier to search, organize, retrieve, and reuse.

As discussed in Sec. 3.1, unlike real-world images or 3D scans, B-Reps may lack texture, meaningful color, physical scale, canonical orientation, or scene context.

Existing zero-shot 3D recognition methods, including PointCLIP V2 and OpenShape, typically evaluate recognition by selecting among candidate category names supplied at inference time (Sec. 2). In contrast, our task requires generating a plausible identity without knowing the possible object categories in advance. The long-tailed distribution of human-provided identities in Sec. 4 (Fig. 2) further illustrates the difficulty of defining a fixed vocabulary for this setting.

Finally, a single geometric viewpoint may conceal distinguishing features (Sec. 5.2). This motivates GLANCE's active inspection, which selects additional viewpoints to distinguish competing identity hypotheses.

Why GLANCE?

Since an initial image can support several interpretations, GLANCE maintains competing free-form identity hypotheses rather than committing to its first guess. Because those hypotheses imply different geometries, it predicts features that could distinguish them. Finally, because the relevant feature may not be visible in the current image, it selects a subsequent viewpoint to test the expectation.

Thus, GLANCE is not a way to supply more images; its observations are guided by what remains uncertain about the object's identity. Moreover, its stopping criterion (Sec. 3.4) adapts the number of observations according to the separation between competing identity scores. The accuracy/coverage analysis in Sec. 5.3 (Fig. 4) already demonstrates that greater hypothesis separation is associated with higher recognition accuracy, while Fig. 7 shows that objects requiring more observations are generally harder to recognize. Our additional analysis further quantifies the predictive value of hypothesis-score separation for recognition correctness under the primary criterion $C_p$ (ROC AUC = 0.816).

Controlled experimental evidence

As reported in Sec. 5.2 (Fig. 3), the controlled comparison on 100 objects evaluates multiview observations, active viewpoint selection, and explicit geometric hypothesis testing. GLANCE achieved 76.1% primary accuracy using 2.61 views on average, compared with 72.5% and 3.15 views for random-view sampling. The no-hypothesis-testing ablation, which retains model-directed viewpoint selection but removes explicit forward prediction and verification, achieved 67.9% primary accuracy and 3.08 mean views. These results support the contribution of explicit hypothesis testing to recognition accuracy and observation efficiency.

In a new orientation-robustness experiment (Sec. 3.1 motivates the non-canonical orientation setting), GLANCE achieved primary accuracies of 71.16%, 68.8%, and 72% for upright, upside-down, and randomly rotated objects, respectively, compared with 80.70%, 57.7%, and 50% for passive three-view recognition, where the three images were supplied simultaneously in one call. Although the passive baseline performs better on upright objects, GLANCE's accuracy varies less across the tested orientations, suggesting greater robustness when orientation is unknown.

Thank you for the rebuttal and clarifications. I have reviewed the authors' responses and will maintain my original rating.

Comments on the paper

1 comment

Dear Reviewers,

The authors have submitted detailed responses to your reviews.

Please review their feedback, engage with the authors if any points remain unclear, and update your evaluation and score if their response has addressed your concerns.

Thank you

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.