- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Lens3D for fine-grained object captioning in 3D scenes. LensUnd selects complementary target views for an external VLM, while LensDistill uses the generated descriptions to train 3D LLMs without external VLM calls at inference. Experiments on LensBench show improved captioning on two backbones while largely preserving their existing grounding and QA performance.
Critical strengths
-
The paper addresses a useful problem: existing 3D models often locate objects correctly but miss their detailed appearance and parts. Captioning improves substantially on both tested backbones, with human ratings and target-masking experiments providing evidence beyond text-overlap scores.
-
The evaluation includes controlled comparisons of view selection, target marking, and predicted versus ground-truth boxes. Training and evaluation scenes are explicitly separated, and tests on standard grounding and QA benchmarks show that detailed-caption training does not substantially weaken existing capabilities.
Critical weaknesses
-
The student gains are not clearly separated from the benefit of richer caption supervision. The proposed selector improves Summary F1 from 0.436 to 0.445 over ExCap3D’s selection strategy, but the paper does not compare students trained on equally detailed captions produced by the two methods. It is therefore unclear whether the new selector contributes meaningfully to the large student gains or whether similar gains could be achieved using an existing caption-generation pipeline.
-
The practical benefit of distillation is not fully evaluated. GPT and Gemini are used to generate supervision and references, but are not included as direct captioning baselines. Removing external VLM calls does not by itself establish lower deployment cost, particularly since the student processes 32 views while the teacher uses four selected views. Without a quality–cost comparison, it remains unclear when the distilled student is preferable to directly querying an external VLM.
Questions for authors
-
With the teacher, prompts, caption count, and student training budget fixed, how does student performance differ when training captions are generated using the proposed selector versus ExCap3D or quality-pool sampling?
-
How do the students compare with directly querying GPT/Gemini in caption quality, end-to-end latency, and computational or API cost? Please include view-selection overhead and assess quality without matching a teacher against its own reference captions.
-
Are teacher-generated details checked for visibility in the student’s 32 input views? If some details are absent, how is the risk of training the student to produce visually unsupported descriptions assessed?
3 comments
Dear Reviewer HzPy,
Thank you for taking the time to review our paper and for your valuable feedback. With one day left in the author–reviewer discussion period, we would be glad to address any remaining questions or concerns you may have.
Best regards, Authors of Submission 11626
Dear Reviewer HzPy,
Thank you for your review and for the constructive feedback. With less than 12 hours left in the author–reviewer discussion period, we would appreciate it if you could let us know whether our response has addressed your concerns. We are happy to clarify or address any remaining points.
Best regards, Authors of Submission 11626
We sincerely thank Reviewer HzPy for the time and efforts spent on our manuscript.
Q1: Controlled student ablation
A1: We fixed the teacher, prompts, objects, caption count, and student training budget, and varied only the view-selection strategy. The teacher is Qwen3.8 27B, which did not generate any LensBench reference. Each student was trained for two epochs on 10% of the training data.
Table Q1:
The student results follow the LensUnd results in Table 6 of the main paper. Table Q1 shows that bbox-voxel is best and Uniform sampling is lowest at both metrics. This shows that the quality of the data-construction scheme is positively correlated with the performance of the distilled student.
Q2: Distilled students versus external VLMs
A2: The relevant comparison is the complete localization-and-captioning pipeline. LensUnd uses 3DRS for localization, then selects and marks four views for an external VLM. LensDistill performs both tasks within one 3DRS model using its native 32-view input.
Local LensUnd with Qwen3-VL-8B requires 16.9B total parameters and approximately 36 GB peak GPU memory, versus 8.1B and approximately 19 GB for LensDistill student. Both pipelines can reuse the scene-level inputs across queries by caching the scene's visual tokens in the KV cache, so the scene is encoded only once. Given the cached scene, the student then needs only one model inference per query, whereas LensUnd still runs its localization inference, view selection and wireframe rendering, and a separate VLM inference on the additional target-dependent views.
Table Q2:
Table Q2 shows that LensDistill is 1.46–1.80× faster than local LensUnd and 3.19–4.80× faster than the GPT pipeline, while Table 6 in A.5 shows that LensUnd with the 8B VLM and the LensDistill student perform comparably, so the speedup comes at a comparable performance level.
Q3: Visibility of teacher supervision
A3: To check whether the teacher's supervision is grounded in what the student can see, we use Qwen3.8 27B to judge 4,260 attributes (parts, materials, and states) from the teacher's captions against the student's 32 views. Only 5.7% were visible in the teacher's views but not verifiable in the student's, so nearly all of the fine-grained details the student is trained on are supported by its own native input.
Coverage gaps and context prior. Although bbox-voxel covers more of the target surface, some regions are still not observed, and for these the model may rely on context priors. Indoor scenes are regular, and objects that belong together co-occur and tend to share a style, so the surrounding scene and the objects that typically accompany the target can supplement what the target views leave out. To check whether this reliance on context priors leads to hallucination, we evaluated hallucination directly through a user study. Raters scored 30 objects (114 responses, about 4 per object) on detail level and hallucination separately (higher hallucination-free scores mean fewer hallucinations). Table Q3 shows that our distillation does not come with more hallucination. More systematic hallucination suppression, such as uncertainty-aware supervision and hallucination-negative preference optimization, is left for future work.
Table Q3: