- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes Bridge Transformer, a many-to-many fusion framework that jointly refines hierarchical Gaussian representations across agents through alternating inter- and intra-agent attention before producing an ego-centric semantic occupancy prediction.
Critical strengths
The core idea of this paper is broadly applicable and potentially effective across many domains. The authors keenly identify this issue in the specific application field and develop a systematic method to address it, which is a notable strength of the work.
Critical weaknesses
- One limitation is that the evaluation is restricted to a single simulated benchmark, without validation on real-world data or under practical disturbances. This weakens the empirical evidence for the method’s generalizability and robustness.
- The reported results show substantial variation in performance gains across different classes, with the proposed method even performing considerably worse on some categories. I hope the authors could provide further analysis or explanation for these class-dependent differences.
Questions for authors
Please check the weaknesses and answer my questions.
4 comments
Response to Reviewer kypQ (2/2)
W2. Performance gains vary across classes, and the method is considerably worse on some categories.
Thank you for your careful look at the per-class results. Our gains do vary across classes, but this is not unique to our method, since the gains of one baseline over another vary just as much. In Table 1, the per-class gain of VQSOP over VOGS-CP ranges from −11.32 to +21.70 IoU, and that of VOGS-CP over CoHFF from −30.55 to +38.60.
Despite this variation, the improvement is broad, as our method outperforms VQSOP, VOGS-CP, and CoHFF on 9, 10, and 8 of the 12 classes, respectively. Where it trails a baseline, the gap is at most 2.34 IoU. The only exception is CoHFF on three classes, Building, Pole, and Wall, where our method trails by 16.67 to 19.15 IoU. Even so, on Pole and Wall, our gap to CoHFF is smaller than those of VOGS-CP and VQSOP.
Table R2. Per-class IoU (%, ↑) on Building, Pole, and Wall. Values in parentheses denote each class's share of occupied voxels, measured by us on the validation set. Single-agent results are quoted from the CoHFF, VOGS-CP, and VQSOP papers, and collaborative results repeat Table 1. GSFormer is the single-agent model reported in the VOGS-CP paper. Ours and VOGS-CP share this model and differ only in fusion.
| Setting | Method | Building (5.49%) | Pole (0.38%) | Wall (2.04%) |
|---|---|---|---|---|
| Single-agent | CoHFF | 21.04 | 31.66 | 33.30 |
| GSFormer | 3.84 | 5.94 | 30.79 | |
| VQSOP | 10.36 | 12.25 | 35.10 | |
| Collaborative | CoHFF | 25.72 | 42.74 | 58.27 |
| VOGS-CP | 9.61 | 12.19 | 36.45 | |
| VQSOP | 11.22 | 20.16 | 36.51 | |
| Ours | 9.05 | 23.59 | 40.23 |
Table R2 shows that on Building and Pole, GSFormer is much weaker than single-agent CoHFF, trailing it by 17.20 and 25.72 IoU. Although our fusion narrows both gaps, this weaker single-agent starting point still leads to large final gaps. On Wall, in contrast, the two single-agent models differ by only 2.51 IoU, and the gap opens with collaboration: CoHFF gains 24.97 IoU, whereas Ours and VOGS-CP gain only 9.44 and 5.66. A likely explanation for all three gaps is that these classes are tall vertical structures, whose voxels are determined mainly by depth along each camera ray. Unlike CoHFF, which predicts per-pixel depth with a pretrained depth network, our method has no depth supervision, so the depth of its Gaussians along each ray is ambiguous. This ambiguity may also make the agents' observations of the same structure harder to align, which would limit the gain from collaboration.
Thanks for the response. The newly added experiments are useful for addressing my concerns. I have also checked the authors’ responses to the other reviewers. I decided to raise my score to 3.
Dear Reviewer kypQ,
Thank you for your response and for reading our replies. We are glad that the new experiments addressed your concerns. We sincerely appreciate your thoughtful review and your decision to raise the score.
Best regards,
Authors of Submission 40381
Response to Reviewer kypQ (1/2)
Thank you for your comment on our evaluation. To our knowledge, no public real-world V2X dataset currently supports the evaluation of collaborative semantic occupancy prediction. Real-world V2X datasets, including DAIR-V2X [1], V2V4Real [2], and V2X-Real [3], are annotated only with 3D bounding boxes for vehicles, pedestrians, and other traffic participants. These boxes cannot be converted into dense semantic occupancy labels, because they do not cover static classes such as Road, Building, and Vegetation, and these datasets provide no point-wise semantic labels from which occupancy labels could be generated. This leaves Semantic-OPV2V as the standard benchmark for the task. CoHFF, VOGS-CP, and VQSOP all report results on this benchmark alone. Our method makes no simulation-specific assumptions, so it can be evaluated on real-world V2X data once such labels are available. We nevertheless agree with your point; our conclusion already acknowledges it and leaves validation on real data to future work.
Pose error is a key practical disturbance in collaborative perception because imperfect localization misaligns the features that agents share [4]. We therefore tested robustness to pose noise. Following the noise model of CoAlign [4], we added zero-mean Gaussian noise to each agent's position and heading, with σ ∈ {0.2, 0.4, 0.6} (m / °). The noise was added at inference only, and no model was retrained. CoHFF is not included because it does not spatially align collaborator features using the relative pose, so pose noise affects it through its learned pose embedding rather than through the feature misalignment that this protocol tests. Its authors report a GPS-noise study in their supplementary material.
Table R1. Robustness to pose noise at different noise levels σ, in mIoU (%, ↑). The Single-agent column reports results without collaboration, quoted from the VOGS-CP and VQSOP papers. Ours and Ours w/ mask are built on the VOGS-CP single-agent model and therefore share its value.
Ours stays the most accurate at every noise level. It leads the stronger baseline at each level by at least 3.18 mIoU. It also leads Ours w/ mask by 1.14 to 1.26 mIoU across all noise levels, showing that the M2M gain remains stable under pose error. Ours shows a large decline as σ grows. Since pose noise affects only collaborator information, the decline suggests that Ours makes more effective use of this information and thus has more to lose. Even at σ = 0.6, the highest noise level, Ours still keeps a collaboration gain of 8.86 mIoU, above the clean-pose gains of 8.24 for VOGS-CP and 7.93 for VQSOP.
References
[1] H. Yu, Y. Luo, M. Shu, Y. Huo, Z. Yang, Y. Shi, Z. Guo, H. Li, X. Hu, J. Yuan, and Z. Nie. DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection. In CVPR, 2022.
[2] R. Xu, X. Xia, J. Li, H. Li, S. Zhang, Z. Tu, Z. Meng, H. Xiang, X. Dong, R. Song, H. Yu, B. Zhou, and J. Ma. V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception. In CVPR, 2023.
[3] H. Xiang, Z. Zheng, X. Xia, R. Xu, L. Gao, Z. Zhou, X. Han, X. Ji, M. Li, Z. Meng, L. Jin, M. Lei, Z. Ma, Z. He, H. Ma, Y. Yuan, Y. Zhao, and J. Ma. V2X-Real: A Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception. In ECCV, 2024.
[4] Y. Lu, Q. Li, B. Liu, M. Dianati, C. Feng, S. Chen, and Y. Wang. Robust Collaborative 3D Object Detection in Presence of Pose Errors. In ICRA, 2023.