- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates domain generalization for multimodal 3D panoptic segmentation and proposes RAMP3D, a framework designed to improve robustness under unseen domain shifts. The proposed framework consists of two main components: Reliability-guided Multimodal Adaptation (ReMA) and Reference-based Calibration (ReCal). ReMA estimates modality-specific reliability through shared learnable tokens and cross-modal evidence, dynamically adjusting image- and LiDAR-conditioned feature updates. ReCal introduces road-relative representations to refine stuff-region predictions and recalibrate thing-instance confidence scores. This work evaluates RAMP3D on nuScenes, SemanticKITTI, and Waymo under illumination, weather, geographic, and sensor-related domain shifts. The reported results demonstrate improvements over several adapted domain generalization and multimodal segmentation baselines.
Critical strengths
-
The paper investigates multimodal 3D panoptic segmentation under domain generalization, which is a practically relevant.
-
The proposed framework considers robustness at both the feature-representation and panoptic-prediction levels, providing a coherent overall design.
-
The experiments cover multiple domain shifts and report substantial improvements over several baseline methods.
Critical weaknesses
- This work is not the first to investigate Domain Generalization for 3D panoptic segmentation. Frontier works include:
[1] PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Driving
-
Only PQ is reported as the evaluation metric in this work. It would be valuable to see comparisons of mIoU, , and .
-
The comparison in Table 1 is not fair. UniDSeg adopts a CLIP image encoder, while this work uses a DINOv2 image encoder.
-
This work does not report results for two settings: training and evaluation on the target domain, and training on combined source and target data. It remains unclear whether substantial gaps persist compared with these methods.
Questions for authors
Please refer to the above weaknesses and further answer questions.
Q1: Could results be supplied for two experimental setups: training and evaluation on the target domain, and training on combined source and target data? Do substantial performance gaps still exist between the proposed approach and competing methods under these settings?
Q2: Only PQ is reported as the evaluation metric in this work. Could comparisons of mIoU, , and be provided?
Q3: This work is not the first to investigate Domain Generalization for 3D panoptic segmentation. Could the discussion elaborate on the differences between this method and prior frontier work such as PanDA?
5 comments
Response to W3–W4 & Q1: Fair Comparisons and Target-Supervised Evaluation
W3. Fair Comparison with UniDSeg.
CLIP-B is an officially evaluated visual backbone in the original UniDSeg paper, which we adopt when adapting UniDSeg to our panoptic framework. Importantly, our original submission already reports RAMP3D results using the same CLIP-B backbone in Appendix Table 7.
We further extend this matched-backbone comparison to cross-dataset generalization, summarizing both intra-nuScenes and cross-dataset results in Table R9. RAMP3D consistently outperforms UniDSeg under the same CLIP-B backbone, achieving gains of 4.28/2.67/7.33/10.58 PQ on nighttime/rainy/SemanticKITTI/Waymo. Moreover, these gains exceed the performance variations caused by switching RAMP3D's 2D backbone from CLIP-B to DINOv2-B, demonstrating that the improvements primarily stem from our method design rather than the choice of 2D backbone.
Table R9. Fair comparison with UniDSeg across visual backbones. Italicized columns indicate source-domain results.
| Method | 2D Backbone | US S.D. PQ | Nighttime PQ | Rainy PQ | Singapore PQ | nuScenes PQ | sKITTI PQ | Waymo PQ |
|---|---|---|---|---|---|---|---|---|
| UniDSeg | CLIP-B | 79.01 | 44.14 | 73.29 | 63.09 | 80.43 | 40.75 | 32.27 |
| RAMP3D | CLIP-B | 80.29 | 48.42 | 75.96 | 67.28 | 82.27 | 48.08 | 42.85 |
| RAMP3D | DINO-v2-B | 80.12 | 52.06 | 77.65 | 69.02 | 82.18 | 46.13 | 43.80 |
W4 & Q1. Comparison with target-supervised oracles.
Table R10 compares models that share the same 2D and 3D backbones but are trained on source-only, target-only, or combined source and target data, each evaluated on the corresponding target domain. The target-supervised oracles serve as empirical upper-bound references for assessing the remaining gap between DG and target-supervised learning.
Table R10. Comparison with target-supervised oracle models.
| Method | Training Set | Nighttime PQ | Rainy PQ | Singapore PQ | sKITTI PQ | Waymo PQ |
|---|---|---|---|---|---|---|
| Baseline (IAL+DINOv2) | source | 42.08 | 71.99 | 64.69 | 35.90 | 26.39 |
| RAMP3D (DG) | source | 52.06 | 77.65 | 69.02 | 46.13 | 43.80 |
| Oracle (target only) | target | 62.00 | 76.24 | 79.62 | 67.41 | 63.05 |
| Oracle (source + target) | source+target | 64.61 | 79.95 | 80.40 | – | – |
Despite having no access to target-domain data, RAMP3D substantially narrows the gap to the target-only oracle and even surpasses it on rainy (77.65 vs. 76.24). The rainy result is likely explained by training-set size: the rainy training set (5,582 frames) is about half the size of RAMP3D's source set (Boston sunny-daytime, 10,635 frames), so the target-only oracle is data-limited. Once trained on both sets, the oracle reaches 79.95 and exceeds RAMP3D. The gap between RAMP3D and the oracle is larger on SemanticKITTI and Waymo, reflecting the greater challenge of cross-dataset sensor and distribution shifts.
Response to Authors
Thank you for conducting the additional experiments and providing the clarifications. Your responses have addressed my concerns.
Response to W1/Q3 & W2/Q2: DG–UDA Distinction and Additional Evaluation Metrics
W1 & Q3. Distinction from PanDA.
We respectfully disagree. PanDA studies Unsupervised Domain Adaptation (UDA), whereas RAMP3D addresses single-source Domain Generalization (DG). These are fundamentally different tasks: UDA requires unlabeled target-domain data for target-specific adaptation, while DG trains exclusively on source-domain data to generalize to multiple unseen target domains without retraining. We summarize these differences in Table R8 below. Notably, our manuscript already identifies PanDA as a UDA method in Sec. 2 (L116–117).
Table R8. Comparison of UDA and single-source DG settings.
PanDA and RAMP3D differ not only in setting but also in method design. PanDA employs a teacher–student framework, where an EMA-updated teacher generates and refines pseudo-labels to supervise learning on unlabeled target-domain data. This target-dependent training mechanism cannot be directly applied to the source-only DG setting.
In contrast, RAMP3D is trained exclusively on source-domain data and generalizes to multiple unseen targets without further adaptation. Specifically, it (i) estimates modality-specific reliability from cross-modal evidence learned solely on the source domain (Sec. 3.2), and (ii) calibrates panoptic predictions using within-frame road-relative features without requiring target-domain statistics (Sec. 3.3).
W2 & Q2. Additional Evaluation Metrics.
We thank the reviewer for the suggestion. We have added mIoU, PQ-th, and PQ-st to Tables 2 and 3, providing further insights into the effectiveness of ReMA and ReCal.
Effectiveness of ReMA. Global and local gating exhibit complementary benefits. Compared with no gating, global gating mainly improves PQ-st (+1.32/+2.69 on nighttime/Singapore), whereas local gating mainly benefits PQth (+4.88 on nighttime). Combining both improves PQ-th and PQ-st across all three targets, with gains of 5.85/1.39/1.44 PQ-th and 0.16/0.92/2.76 PQ-st on nighttime/rainy/Singapore. These results support the design of global and local gating.
Effectiveness of ReCal. Stuff calibration improves PQ-st by 2.52/0.65 on nighttime/Singapore, while thing re-scoring improves PQ-th by 1.35 on both nighttime and rainy. Combining both achieves the highest overall PQ across all three targets, with gains of 2.72/0.82/0.17, respectively, confirming their complementary benefits for panoptic prediction.