- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
GEO-4D directly associates instance masks from a pretrained single-frame panoptic segmentation model without requiring an additional learned association network. It leverages centroid and covariance cues to match static instances, followed by iterative closest point (ICP) registration for the remaining candidate pairs. Its global geometry-aware soft matching module establishes correspondences through an entropy-regularized optimal transport solution. A short-term memory mechanism further retains unmatched instances. Experiments on SemanticKITTI and nuScenes demonstrate that this geometry-based association pipeline improves temporal consistency compared with learning-based and IoU-based alternatives.
Critical strengths
- A clear contribution is the use of point-cloud registration for instance association in training-free, single-scan 4D LiDAR panoptic segmentation. The method brings ICP into this setting and strengthens it with global geometric matching, rather than introducing another learned association network. This is a useful shift in design: it makes temporal association depend more directly on the geometric structure already present in the point clouds.
- The efficiency gains are substantial. On SemanticKITTI, GEO-4D uses 3.15 GB and 1.10 s per scan, compared with 10.4 GB and 3.51 s for two-scan Mask4Former and 15.5 GB and 3.67 s for two-scan 4D-StOP. It achieves a validation LSTQ of 72.7, comparable to Mask4Former's 72.0 and higher than 4D-StOP's 66.4.
Critical weaknesses
The main comparisons in Table 2 mix different numbers of input scans (single-scan vs. two-scan inputs) and differences in per-frame segmentation quality . Although the two-scan baseline methods inherently require overlapping scans to enable IoU-based tracking, and controlled comparisons using identical masks are provided in Table 3(b) and Appendix C.3, the presentation in the main text still makes it difficult to immediately distinguish the differences attributable to the backbone network from the gains introduced by the association algorithm. Furthermore, Appendix C.5 (Table A.6) shows that DBSCAN post-processing accounts for 0.76 seconds of the total runtime of 1.10 seconds (nearly 70%). However, the paper lacks a fine-grained runtime breakdown that would allow the individual computational costs of the 3D backbone, DBSCAN, static filtering, and the GGSM-ICP loop to be separately assessed.
Questions for authors
-
To better clarify the efficiency and accuracy metrics reported in Tables 2 and 3 (b), could the authors provide a detailed runtime breakdown for the 3D backbone, DBSCAN, static filtering, and GGSM-ICP? Additionally, could the authors report the performance of GEO-4D when directly applied to multi-scan predictions in the main benchmark (e.g., 2-scan Mask4Former) ?
-
Could the authors clarify the specific strategy and hyperparameters of the short-term memory bank , particularly the values of used for SemanticKITTI and nuScenes, as well as the Sinkhorn regularization parameter and the number of iterations? Furthermore, since caches historical world coordinates without performing motion prediction, how does GEO-4D handle fast-moving objects that reappear at locations far from their last observed positions after being occluded for multiple frames?
3 comments
Response to the authors
Thank you to the authors for the detailed responses. They have addressed my questions. I also appreciate the authors acknowledging that incorporating reliable motion prediction is a promising direction to investigate for the long-term occlusion issue I raised. I will keep my score at 3 and lean toward acceptance.
Response to Reviewer YBfp
We sincerely thank you for your positive feedback and are glad that our rebuttal has addressed your questions. We also appreciate your constructive suggestions regarding long-term occlusion and motion prediction. We are grateful that you are leaning toward acceptance while maintaining your score (3). Thank you again for your thoughtful review and consideration.
Response to Reviewer YBfp
Thank you for the insightful feedback. Below, we address your questions to each comment. We hope our responses have adequately addressed your questions.
We additionally provide a stage-wise runtime breakdown of Geo-4D on an NVIDIA RTX 3090, as the RTX A6000 used for the original evaluation was not available for these additional measurements. Despite using different GPU hardware, the total runtime is largely consistent, with a modest increase from 1.10 to 1.22 s/scan.
Regarding multi-scan predictions, this comparison is already included in Table 3(b). Specifically, we first obtain predictions from 2-scan Mask4Former and split them into single-scan instance masks. We then apply Geo-4D to associate these instances across frames, using the same underlying predictions as the IoU-based baseline. Geo-4D improves LSTQ from 72.0 to 72.8 and Sassoc from 76.3 to 77.8, while both methods achieve the same Scls of 68.0 due to their identical segmentation predictions. This demonstrates the benefit of our geometric association independently of segmentation quality.
In this section, we provide implementation details and hyperparameter analysis. We use a memory bank size of wmem=3, Sinkhorn regularization ϵ=0.2, and 30 Sinkhorn iterations, with the same settings applied to both SemanticKITTI and nuScenes.
Our ablation results show that a memory bank size of 3 achieves the best performance. While 30 Sinkhorn iterations provide the best trade-off between performance and runtime, with no further improvement from additional iterations. Performance also remains stable across ϵ=0.1 and 0.2, indicating limited sensitivity to the regularization parameter. We have also updated the implementation details in Section 4.1 of the revised manuscript to clarify these hyperparameter settings.
We acknowledge the reviewer’s concern regarding fast-moving objects reappearing after prolonged occlusion. Such cases remain challenging because Geo-4D relies on historical instance geometry without explicit motion prediction.
However, associating objects across prolonged occlusions is a general challenge in temporal instance association. For example, multi-scan IoU-based methods can also struggle when occlusions eliminate shared instance observations. Furthermore, while motion prediction could help, partial LiDAR observations and changes in visible surfaces can lead to unreliable velocity estimates. Under identical instance masks, Kalman-based motion prediction with 3D Box IoU and Hungarian matching achieves an Sassoc of 61.9, still substantially below Geo-4D's 78.3, as shown in Table 3(a).
While our overall pipeline and geometry-aware instance association demonstrate strong effectiveness, incorporating robust motion prediction to better handle prolonged occlusions remains a promising direction for future work, as discussed in Appendix D.1.