- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Inject3D is a 3D multimodal large language model (MLLM) designed to preserve fine-grained geometric details while maintaining a compact visual token sequence. The framework uses a pretrained Utonia point-cloud encoder to extract dense point-wise representations and introduces Anchor-Indexed Feature Re-Access (AFRA), which uses layer-specific learnable queries and cross-attention to inject localized geometric features into selected LLM layers. The authors complement AFRA with GRPO-based task-specific reinforcement post-training across 3D captioning, visual question answering (VQA), and part grounding. Experiments demonstrate competitive performance across several object-centric benchmarks.
Critical strengths
Solid Empirical Performance: Demonstrates strong performance across 3D captioning, VQA, and fine-grained part grounding benchmarks.
Structured Ablations: Provides systematic ablations evaluating geometric injection strategies, token budgets, and reinforcement post-training stages.
Critical weaknesses
1. Limited Methodological Novelty
- Overlap with Prior Work: AFRA conceptually aligns with existing local-global 3D feature aggregation frameworks, such as PerLA (CVPR 2025) and LSceneLLM (CVPR 2025).
- Static Feature Retrieval: In Eq. (3), the learnable queries and local neighborhoods are independent of evolving LLM hidden states. Because features are not dynamically conditioned on intermediate LLM representations, they could effectively be precomputed offline.
2. Marginal Incremental Improvements
-
AFRA vs. Input Injection: Table 3 shows that AFRA improves grounding mIoU by only +0.014 over input-level feature injection (0.603 vs. 0.589).
-
Reinforcement Post-Training: Table 2 indicates that GRPO post-training yields a modest +0.011 average grounding mIoU gain (0.638 vs. 0.649), requiring multi-run statistical validation to confirm robustness.
3. Influence of the Pretrained Encoder
-
Strong Encoder Prior: Inject3D relies on the powerful Utonia point-cloud encoder.
-
Confounding Gains: Table 9 shows that replacing Point-BERT with Utonia alone increases grounding mIoU from 0.482 to 0.520 (+0.038) under identical settings. Controlled comparisons with matched backbones are necessary to isolate AFRA's actual contribution.
4. Insufficient Efficiency and Memory Analysis
-
Inference Overhead: Table 3 reports an increase in time to first token (TTFT) from 136.86 ms to 154.76 ms upon introducing AFRA.
-
Missing Peak VRAM: Peak GPU memory consumption is not reported, leaving the true memory footprint of retaining dense point-wise features across multi-layer cross-attention uncharacterized.
5. Limited Evaluation on Large-Scale Point Clouds
-
Object-Centric Scope: Experiments are restricted to object-centric point clouds containing 8,192 points.
-
Unverified Scalability: The effectiveness and computational scalability of fixed local neighborhoods remain unproven for large-scale indoor scenes (e.g., ScanQA, ScanRefer) or outdoor LiDAR point clouds.
Questions for authors
- Methodological Novelty: How does AFRA technically and conceptually differ from local geometric aggregation in PerLA or adaptive preference retrieval in LSceneLLM?
- Dynamic Feature Access: Given that queries and neighborhoods are independent of current LLM hidden states, why is layer-wise cross-attention superior to precomputing layer-specific features offline?
- Statistical Significance: Are the gains observed for AFRA over input-level injection (+0.014 mIoU) and GRPO post-training (+0.011 mIoU) statistically significant across multiple random seeds?
- Controlled Baselines: Can you provide baseline comparisons using identical point encoders (Utonia), LLM backbones (Qwen3-VL-4B), and training data to isolate the true contribution of AFRA?
- Memory Overhead & Scalability: What is the peak GPU VRAM consumption during inference and training with and without AFRA? How do memory and latency scale when increasing point resolution beyond 8,192 points?
References
- Guofeng Mei, Wei Lin, Luigi Riz, Yujiao Wu, Fabio Poiesi, Yiming Wang. "PerLA: Perceptive 3D Language Assistant." IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
- Hongyan Zhi, Peihao Chen, Junyan Li, Shuailei Ma, Xinyu Sun, Tianhang Xiang, Yinjie Lei, Mingkui Tan, Chuang Gan. "LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences." IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.