- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- The paper argues that 3D VLMs are evaluated mainly on category recognition and captioning, leaving fine-grained object understanding unmeasured. It introduces ObjectArena3D: 16,319 questions over 12,431 assets, covering 20 tasks in four families (semantic, camera, part structure, surface), plus random-orientation and random-color robustness splits.
- It proposes Objective3D, which fine-tunes Qwen3-VL-8B with multi-view augmentation and intra-view attention across aligned rendering modalities (RGB, normal, wireframe, component), trained on a two-stage multitask mixture.
- Benchmark results show that camera awareness is the weakest capability, part grounding relies heavily on appearance, and specialist 3D models trail general VLMs. Objective3D raises the average from 42.95 to 70.26, surpassing GPT-5.6 Sol, with gains concentrated on camera tasks.
Critical strengths
- The motivation is strong and sound. The benchmark idea is original, large-scale and diverse.
- The paper provides insightful analysis that I believe will benefit the community.
- The paper is well written, detailed, and easy to follow, with precise task definitions and deterministic scoring.
Critical weaknesses
[W1] Dataset curation unclear. The collection, filtering, and quality-control process is not clearly described, which limits reproducibility.
[W2] Limited novelty in Method development. The method mainly appears to add more modalities to existing approaches, without a clear technical contribution beyond that.
[W3] Annotation details missing. The paper does not report the number of annotators, annotation time, or any cross-checking or agreement measures used to ensure question quality.
[W4] Limited method effectiveness. In the Sec. 4.3 ablation, plain fine-tuning already reaches 58.24 while the proposed components add only +1.39, so most gains seem to come from training data that shares the benchmark’s pipeline and formats and may be biased toward it.
[W5] Limited Generalization. The improvement over Qwen3-VL-8B is only 0.0023, which doesn’t convincingly show the method’s benefit or generalization.
Minor Weakness:
- Abstract, L41: “an framework” should be “a framework.”
Questions for authors
[Q1] The paper describes only part of its data sources. Could you specify where the 12,431 assets come from, how many annotators were involved, and how label quality was cross-checked for the questions?
[Q2] Regarding the motivation of Objective3D: in the introduction, you show that specialized 3D object models have access to full geometric information, yet they underperform general VLMs. How does this fit your hypothesis that models need to reason explicitly over geometry?
[Q3] Why do you expect surface normals, wireframes, and components to help with question types such as view correspondence and orientation?
[Q4] How do other models perform when fine-tuned on the same training data used for Objective3D? For example, Qwen3-VL-8B trained on your data without architectural modifications, or a specialist model such as EVA01.
[Q5] Where does the training data come from, and how does it relate to the test set? If both are generated by the same pipeline, could this introduce bias that inflates the results?
4 comments
Q2, Q3, Q4, and minor problems
Q2. Why emphasize geometry when specialist 3D models underperform?
We clarify that Objective3D uses rendered images and text at both training and inference, without point-cloud, voxel, or mesh-token inputs (Line 273-274). By “geometry explicitly aligned with RGB,” we mean geometric cues—such as surface normals—rendered in the same image coordinates as the RGB observations.
Why specialist models’ results do not contradict this motivation. Direct access to a 3D representation does not guarantee effective geometric understanding. Sampling and token compression may discard fine geometric or appearance details, while a separate 3D input interface does not automatically inherit the visual representations and image–language alignment of pretrained VLMs. Differences in backbones and supervision also mean that the specialist models’ lower scores cannot be attributed solely to their representations.
Our approach exposes geometric evidence through the pretrained visual interface: multi-view RGB images capture complementary observations, normal maps expose surface orientation, and supplementary component and wireframe renderings highlight structural cues. Intra-view attention connects corresponding patches across modalities, while synchronized view augmentation varies the observations without breaking their alignment.
In the revised results, the complete framework improves Qwen3-VL-8B from 44.73% to 71.03% overall, with 2D–3D projection increasing from 19.64% to 79.96%. This demonstrates stronger spatial capabilities without native 3D inputs. The direction is consistent with VLM3 [1], which shows that standard image-based VLMs can learn diverse 3D tasks through appropriate supervision. We will clarify that our motivation concerns how geometric evidence is represented and used, rather than requiring native 3D inputs.
[1] Cai Z, Liu Z, Xiong Y, et al. VLM3: Vision Language Models Are Native 3D Learners. arXiv preprint arXiv:2605.30561, 2026.
Q3. Why might normals, wireframes, and component renders help correspondence or orientation?
These renderings expose complementary geometric cues through an image-based interface. Surface normals encode local surface orientation even when surfaces have similar colors. Supplementary wireframe and component renderings highlight mesh edges and part boundaries, providing additional structural cues during training.
For view correspondence, these cues may help distinguish regions with similar appearance through their local geometry and surrounding part configuration. For orientation, surface directions and the arrangement of visible parts provide evidence for relating an observation to the annotated object orientation.
Q4. How do plain Qwen and EVA01 perform when trained on the same data?
As shown in Global Response, we include a matched-data baseline using Qwen3.5-9B in our new single-stage ablations. With the same 210,283 training examples and training settings, plain fine-tuning achieves 68.86%, compared with 45.17% before fine-tuning and 69.43% with both proposed components. This demonstrates substantial gains from our training data without architectural modifications, with the components adding 0.57 percentage points.
For EVA01, its official repository lists the release of training code and recipes as future work. We therefore do not currently provide a matched-data fine-tuning comparison for this model.
Minor comment. Wording in the abstract
We will fix typo error in revised version.
W1/W3/Q1/Q5. Dataset construction, annotation effort and quality control, and training–test bias
Our Global Response details the asset sources and task-specific construction procedures. Here, we clarify the annotation workforce, acceptance protocol, and relationship between benchmark and training data.
Curation and human verification
Candidates undergo graphics-based and VLM-assisted screening, followed by category-aware sampling and task-specific validation. Every benchmark question underwent human inspection and correction where necessary. Each task involved 10 annotators with non-overlapping sample assignments. Annotation managers provided task-specific training and conducted sampled checks during a ramp-up phase; annotators proceeded to full-scale work only after their performance stabilized.
Acceptance review was an additional quality-control stage. In each round, two annotation managers and the task designer each independently sampled 10% of the submitted annotations. These samples could overlap. Submissions failing acceptance were returned for correction and rechecked until sampled accuracy exceeded 90%, with final acceptance determined by the task designer. We did not conduct independent double annotation and therefore do not report an inter-annotator agreement score. The 90% threshold is an acceptance criterion, rather than a measured accuracy for the entire benchmark. The annotation phase took approximately one week, with 10 annotators assigned to each task.
Training–test separation
Benchmark assets and their detected duplicates are excluded from training using file MD5, DINOv2 similarity between rendered images, and voxel-based geometry checks. Separation is defined at the underlying-asset level, with rendered and transformed variants inheriting their source asset’s split.
Shared generation procedures and potential bias. Programmatically constructed benchmark and training questions share a common generation framework. Within this framework, we introduce diversity in question formulations and rendering configurations, while distinguishing benchmark references from automated training supervision:
- Diverse question formulations. Part-understanding questions can identify a target through its part name, an affordance-based expression, or a caption describing its appearance or function. We augment question wording and captions and, where applicable, vary distractors and option order while updating the corresponding reference answers.
- Object and viewpoint augmentation. Camera questions incorporate object rotation, scaling, and translation, together with rendering-viewpoint augmentation. Reference answers are computed from the resulting transformations, geometry, and camera parameters, keeping supervision consistent with the rendered observations.
- Human-established surface references. Surface benchmark labels are annotated from scratch and verified by humans. Training labels are selected through agreement between a VLM and feature-based classifiers trained on human-labeled data, using descriptors such as contrast and curvature variation. Surface evaluation therefore uses human-established references rather than automatically accepting the predictions used to construct training supervision.
These measures are designed to reduce dependence on fixed wording and canonical rendering configurations. Together with asset-level separation, they support evaluation on unseen assets under varied observations. However, they do not establish disjoint template distributions or eliminate all potential
W2/W4. Methodological contribution and the roles of training data and architectural adaptation
Our contributions include ObjectArena3D, the accompanying object-centric training data, and the Objective3D adaptation framework. We agree that fine-tuning accounts for the largest share of the performance improvement. The task-specific data construction pipeline and training-data recipe that enable these gains are themselves key contributions of our work.
What Objective3D adds beyond additional modalities. Objective3D introduces an explicit mechanism for exchanging information between spatially corresponding observations. After the visual merger, intra-view attention operates across modalities at the same view and image-patch location, connecting RGB appearance with geometric cues from aligned renderings before language-model processing. A learnable gate controls this interaction, and the fusion module preserves the existing visual-token count and order. Synchronized view augmentation further varies the observations while maintaining alignment across modalities. These components specify how complementary evidence is selected and combined, beyond simply supplying additional images.
How we interpret the gains. We acknowledge that the reported difference between plain fine-tuning (58.24%) and the configuration with view augmentation and intra-view attention (59.63%) is modest. These results do not justify attributing the overall improvement to the added modules. The accompanying data construction is itself a substantive contribution: it converts rendering geometry, part annotations, and verified semantic and surface labels into supervision for fine-grained object understanding. We therefore present Objective3D as a combined data-and-adaptation framework and will distinguish its overall effectiveness from the incremental contribution of individual components.
Could benchmark familiarity explain the improvement? Shared task definitions and question-generation procedures may account for part of the gain, even when training and benchmark assets are disjoint. As described in our Global Response, we exclude benchmark assets and detected duplicates from training and diversify question formulations, referring expressions, object transformations, and viewpoints. These measures reduce reliance on fixed examples and formulations but do not eliminate all pipeline-related bias.
We therefore evaluate transfer beyond ObjectArena3D. In the additional Qwen3.5-9B-based comparison against its own backbone, Objective3D improves ERQA by 7.50 percentage points and CountBench by 1.98 points. These external gains provide evidence that the learned capabilities extend beyond familiarity with ObjectArena3D’s formats. They support the complete training approach, while leaving the causal contribution of each architectural component to be established separately.
W5. Generalization of Objective3D.
We respectfully disagree that the MMBench result alone supports the conclusion of limited generalization. MMBench assesses broad multimodal understanding and general VQA ability. Our fine-tuning data focus on fine-grained 3D object understanding and do not include a dedicated general-knowledge training component; improvement on MMBench was therefore not the objective of this experiment. Instead, we used it to assess whether object-centric fine-tuning preserves the backbone’s existing capabilities. MMBench accuracy changed from 88.58% to 88.35%, a decrease of only 0.23 percentage points, while ObjectArena3D performance increased from 42.95% to 70.26% (+27.31 points in Table 2). This suggests that Objective3D substantially improves the targeted capabilities while largely retaining general VQA performance.
Beyond this retention check, we further investigate positive transfer to external tasks by evaluating a Qwen3.5-9B-based Objective3D against Qwen3.5-9B on MMLU-Pro, SuperGPQA, MMBench, ERQA, and CountBench. These additional experiments examine both the preservation of general knowledge and VQA capabilities and transfer to related spatial-reasoning and counting tasks, as reported below.
Objective3D improves on four of the five benchmarks, with the largest gain on ERQA (+7.50 points) and an additional improvement on CountBench (+1.98 points). General VQA and knowledge performance are broadly preserved: MMBench and SuperGPQA improve, although MMLU-Pro decreases by 0.78 points. These results provide evidence that our object-centric training transfers beyond ObjectArena3D, particularly to spatial reasoning and counting, while largely maintaining general-purpose capabilities.
[1] Y. Wang et al., “MMLU-Pro: A more robust and challenging multi-task language understanding benchmark,” NeurIPS, 2024.
[2] X. Du et al., “SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines,” NeurIPS, 2025.
[3] Y. Liu et al., “MMBench: Is your multi-modal model an all-around player?” ECCV, 2024.
[4] Gemini Robotics Team et al., “Gemini Robotics: Bringing AI into the physical world,” arXiv:2503.20020, 2025.
[5] R. Paiss et al., “Teaching CLIP to count to ten,” ICCV, 2023