- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Basically they say if you just dump 3D group tokens into a pretrained 2D ViT it doesnt work well, because FPS/KNN groups are messy and ViT wants a regular patch grid. So they add a pre-adapter (project to a few views, sample the ViT pos embeddings, then some masked attn at patch level) and a post-adapter with learnable queries. Train with InfoNCE on points/image/text, plus this “Gaussian MAE” thing that multiplies tokens by N(0,1), and they unfreeze the first two ViT layers. Eval is zero-shot cls on LVIS / ModelNet40 / ScanObjectNN, some retrieval viz, ScanNet200 seg after finetuning a head, and a few VQA examples with InstructBLIP
Critical strengths
- the mismatch they point at is real, Pipeline-2 stuff like ViT-Lens/Uni3D does have this issue. Table 1 and Table 4 at least show a plain MLP or cross-attn adapter is weaker than their stack
- cls setup is the usual OpenShape/ViT-Lens style on a big mixed pretrain set, and they do ablate the adapter bits / ft strategy / scale / mask type, which I appreciated
Critical weaknesses
- pretty incremental. adapters are just multi-view PE sampling + empty-patch mask + query cross attn glued together. full-ensemble Top-1 vs RECON++/Uni3D/ViT-Lens is only +0.3 / +1.2 / +0.5 on LVIS / MN40 / ScanObj (Table 2). hard to sell “universal 3D representation” off that, and they dont even match compute / same ViT / same cached I,T features
- a few things just dont line up. Table 1 says PointNet but sec 3.1 is FPS+KNN+PointBERT, 512 groups, and 54.0/88.5/65.9 show up again as Ada3D-G in Table 2?? so what did Table 1 actually run. Table 3 “OV seg” looks like they finetune tokenizer+head on ScanNet200, thats not really zero-shot. Fig 5 VQA is cherry picked and some panels look like normal 2D photos. also they wrote “fastest point sampling” lol
- Gaussian MAE with mean-0 multiply is a weird regularizer (can just zero the token). Table 6 is +0.9 on ModelNet vs zero-mask, not enough for the robustness talk
Questions for authors
- Table 1 vs 3.1 — PointNet or PointBERT? which ViT / data gives 54.0 / 88.5 / 65.9
- can you rerun Uni3D / ViT-Lens / RECON++ with the same OpenCLIP, same cached embeddings, same train set. how much gain is left
- is Table 3 actually zero-shot OV or just ft on ScanNet labels. if ft, say the split and compare to a ft Uni3D/OpenScene under the same proto
- any automatic VQA number at all? how many Qs, acc vs ViT-Lens
- Eq1 views — fixed orthographic at train/test? assume gravity aligned objects? what if you throw random SO(3) at it
Clarification Regarding AI Use in This Review
All substantive technical assessments, concerns about the paper, and the final score were independently formulated by me before using AI.
After reading the paper and drafting my own concerns, I used an AI assistant to refine the wording and organize those concerns into questions for the authors. The AI also suggested more specific experimental comparisons and reporting details. I reviewed these suggestions and independently decided which to include. I take full responsibility for the final review.
For reference, my original concerns, written before using AI, were as follows:
2.Table 4 shows better results when adding modules, yet parameters increase from 15.2M to 23.5M. A post-adapter-only baseline and capacity-matched comparisons are required. Comparisons across different positional encodings are also needed to identify the real source of performance gains.
3.Section 4.4 misses key experimental details, hurting reproducibility and fair comparison. Additional ablations are required to isolate the adapter’s effect on segmentation.
4.Figures 5 and 10 only show qualitative examples without quantitative metrics. They cannot prove consistent improvements on spatial reasoning or hallucination reduction. The image-only comparisons in Figure 5 cannot be attributed to the proposed 3D adapters.
I would be glad to provide further clarification if needed.