- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduce X-MV3D trying to benchmark the generalize ability of Multi-View 3D Object Detector across datasets. The paper analyzes the main gap on cross-dataset generalize are camera-dependent feature lifting and multi-view aggregation under LODO and target-included settings. Then the paper proposes CamGeoGate which adds camera-geometry encodings to image features and randomly drops views during training.
Critical strengths
- Cross-dataset transfer of multi-view 3D detectors has received little systematic study. The three-level setup (best single-source, LODO, target-included) clearly shows how much source diversity helps, and how much is still gained once target data is included. Using LODO as the main target-unseen setting is a sensible choice. Releasing unified metadata for four datasets also lowers the cost of follow-up work in this direction.
- The zero-initialized residual adapter CamGeoGate and training-time view dropout add little overhead and need no target annotations. The method improves [email protected] for most detector and datasets.
Critical weaknesses
- The failure analysis attributes the gap to acquisition shift, but the experiments can hardly support this, as there are only two in-domain perturbation tests on ScanNet. CamGeoGate does improve the performance, but it does not show the potential to close the gap, since the improvement is quite small compared with the cross-dataset gap.
- The dataset merging is quite simple, which only performs coordinate alignment and category merging. However, the different annotation conventions are also important in the cross-dataset setting. EmbodiedScan[1] analyzes the bbox sizes across different datasets, which shows that datasets differ in how objects are split and which category they are assigned to, but the authors neither analyze this nor apply any handling. Therefore, the Common-9 experiment in Table 4 is also of limited meaning, since unprocessed training data will already cause conflicts for the model, especially when a category needs to be learned in one dataset but is not annotated in another, and this pattern lacks analysis.
- All three baselines are RGB-only models, so why is MVSDet not compared? And why is the input restricted to RGB, instead of designing a point cloud input setting, which all the datasets support and is more popular?
- Table 3 looks strange in this position. The input settings of the methods in Table 3 all differ from the RGB-only setting of the paper, and it is unclear what this table tries to claim.
Questions for authors
- The failure analysis on the camera setting and annotation conventions.
- MVSDet and other popular methods comparison would be helpful. As ImVoxelNet[2022] is quite earlier.
- Explain why Table 3 is important.