- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PAR3D extends 3D-LLaVA toward part-level scene understanding. The authors introduce ScenePart, a synthetic dataset with object-part annotations and language instructions, and combine part-aware contrastive learning, representation-preserving regularization, and object-part segmentation queries. The model supports question answering and referring segmentation, with improvements on synthetic part benchmarks, a real-world ScanNet part test set, and object-level segmentation tasks.
Critical strengths
- ScenePart provides 800 scenes, approximately 44,000 part masks, and 273,000 language-task annotations for training and evaluating scene-level part understanding.
- On real-world part segmentation, PAR3D achieves 39.4 mIoU versus 33.2 against the strongest baseline, providing initial evidence of transfer from synthetic part supervision.
Critical weaknesses
- Section 3.4 uses separate object and part tokens and also adds supervision for the host-object mask. It would be useful to keep both mask targets but use two identical [SEG] tokens. Without this comparison, it is hard to tell how much the separate token types matter.
- In Table 4, adding ScenePart-200K to 3D-LLaVA’s training raises ScenePart-Seg mIoU from 11.1 to 51.8. I would like to see this version evaluated on part-level QA and real-world scenes to understand how much the full model adds when the training data are the same.
Questions for authors
- How do two general [SEG] tokens compare with [OBJ] and [PART] under identical object and part mask supervision?
- Reason3D also supports language-conditioned point-level mask prediction. Could the authors clarify why 3D-LLaVA was chosen as the base framework and whether the proposed components could be applied to other 3D-MLLMs?
7 comments
Follow-up on Author Responses and Request for Feedback
Thank you again for your insightful and constructive feedback. We have provided detailed responses and additional experiments addressing your concerns. For ease of review, we briefly summarize the key clarifications and findings below.
-
Controlled two-token comparison (Q1). Under identical paired-mask supervision and training settings,
[OBJ][PART]improves overall mIoU over two[SEG]tokens (60.1 → 60.7), with gains on both coarse and fine parts. -
Gains beyond ScenePart supervision (W2). With the same training data and Utonia encoder, our modules provide consistent gains across segmentation and QA benchmarks. Real ScanNet part mIoU improves from 36.3 to 39.4, and all four QA metrics improve on both ScenePart-QA and ScanQA.
-
Applicability to other 3D-MLLMs (Q2). Part-aware representation learning targets point-level visual features, while hierarchical queries extend LLM tokens for multi-granularity segmentation. Both designs can be adapted to other 3D-MLLMs.
We would greatly appreciate your feedback on whether these comparisons address your concerns about the proposed designs. Please let us know if there are any remaining points we can clarify.
Thanks for the rebuttal. Some of my concerns are addressed. But the contribution of different [obj] and [part] tokens are limited, as it only brings a small miou improvement for both all miou and fine-part miou.
Response to Reviewer bwv7
Thank you for your careful review and constructive suggestions. We appreciate your recognition of ScenePart’s contribution to scene-level part understanding and PAR3D’s performance on real-world part segmentation. We address your questions and comments below.
Q1: Comparison with Two
[SEG]Tokens under Identical Mask Supervision.Thank you for this insightful question. We trained a baseline with two identical
[SEG]tokens, keeping object- and part-mask supervision and all other training settings unchanged. PAR3D achieves higher overall mIoU, with improvements on both coarse and fine parts. These results support using distinct object and part tokens to reduce granularity conflicts and improve part grounding.[SEG]Q2: Choice of 3D-LLaVA and Applicability to Other 3D-MLLMs.
Thank you for highlighting Reason3D as an alternative framework. We clarify our framework choice and the applicability of our designs below.
Choice of 3D-LLaVA. For each target task, Reason3D pretrains on data from other tasks before task-specific fine-tuning. In contrast, 3D-LLaVA unifies segmentation, QA, and captioning in one model. This matches our goal of extending unified multi-task understanding from objects to parts.
Generalizability. Part-aware representation learning focuses on learning point-level visual features, while hierarchical queries extend LLM tokens for multi-granularity segmentation. Both designs can be applied to other 3D-MLLMs.
W2: Extended Ablations on Part-Level QA and Real-World Segmentation.
We appreciate your suggestion and have extended the Table 4 ablation to QA and real-world part segmentation, including 3D-LLaVA trained with ScenePart and the Utonia-matched baseline. With the same training data and Utonia encoder, our modules consistently improve performance across multiple benchmarks:
Segmentation Ablation (mIoU).
Question Answering Ablation. C: CIDEr; B-4: BLEU-4; M: METEOR; R-L: ROUGE-L.