- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SPOR, a semantics-preserving ordinal refinement framework for open-vocabulary panoptic segmentation. It leverages ordinal depth information derived from monocular pseudo-depth estimates to guide feature aggregation, attention routing, and region partitioning. Experiments show improvements in segmentation performance while maintaining open-vocabulary generalization.
Critical strengths
- Well-motivated approach. Incorporating ordinal depth provides a reasonable way to improve spatial understanding in open-vocabulary panoptic segmentation.
- Clear presentation. The paper is well organized and clearly written, making the motivation, methodology, and experimental results easy to follow.
Critical weaknesses
- Limited performance improvements. Although the proposed method achieves improvements over the baseline, the gains are relatively marginal considering the additional complexity introduced by multiple modules.
- Insufficient qualitative analysis. More qualitative comparisons are needed, particularly for semantically similar instances at different depths and challenging occlusion scenarios, to better demonstrate the benefits of ordinal depth refinement.
Questions for authors
- Stability of performance improvements. The performance gains over baseline are relatively small. Results across multiple random seeds would help verify the stability of these improvements.
- Qualitative analysis. More visual comparisons in challenging scenarios, particularly involving semantically similar objects at different depths or complex occlusions, would better demonstrate the advantages of the proposed method.
- Robustness analysis. The paper lacks a systematic analysis of depth estimation errors. Evaluating the impact of noisy or inaccurate depth predictions on segmentation performance would help assess the robustness of the method.
- Effectiveness of ordinal depth. A comparison with simpler depth representations, such as normalized depth values or direct depth feature fusion, would help clarify the specific benefits of the proposed ordinal depth representation.
5 comments
To Q3.
Thank you for your detailed feedback.
To Q3:
SPOR uses predicted depth from Depth Anything V2 as auxiliary information. To quantify the effect of depth noise and inaccuracies, we conducted depth perturbation experiments with the trained SPOR model, keeping RGB inputs and model weights fixed. All evaluations use Depth Anything V2 predictions: the unperturbed reference uses the original predictions, while the perturbed settings apply Gaussian noise or Gaussian blur to assess segmentation stability under degraded depth inputs.
We apply perturbations after the standard inference resizing and before ordinal-depth processing: (1) pixel-wise, zero-mean Gaussian noise with standard deviation σₙ = 0.10 or 0.20 is added to depth maps normalized to [0,1], followed by clipping to [0,1]. Both noise levels share the same per-image noise pattern, differing only in magnitude. (2) A 5 × 5 Gaussian kernel with spatial standard deviation τ = 1 pixel simulates degradation of depth boundaries and local details, without additional random noise. We completed 18 full evaluations across six evaluation tasks.
Under the tested noise and blur conditions, SPOR performs similarly to the unperturbed reference. The maximum absolute PQ changes are only 0.074 and 0.045 percentage points on COCO and ADE20K, respectively; the maximum absolute mIoU change across the five semantic segmentation benchmarks is only 0.151 percentage points. These small changes provide quantitative evidence of SPOR’s robustness to the tested depth noise and local smoothing.
Table C: Effect of depth perturbations on SPOR’s segmentation performance on COCO and ADE20K.
| Setting | COCO PQ | COCO AP | COCO mIoU | ADE20K PQ | ADE20K AP | ADE20K mIoU |
|---|---|---|---|---|---|---|
| SPOR (σₙ = 0) | 55.534 | 46.224 | 65.330 | 22.872 | 14.405 | 29.471 |
| Gaussian Noise (σₙ = 0.10) | 55.503 | 46.283 | 65.366 | 22.834 | 14.394 | 29.612 |
| Gaussian Noise (σₙ = 0.20) | 55.460 | 46.222 | 65.319 | 22.827 | 14.436 | 29.568 |
| Gaussian Blur (5 × 5, τ = 1 px) | 55.517 | 46.226 | 65.323 | 22.855 | 14.407 | 29.482 |
Table D: Effect of depth perturbations on SPOR’s cross-dataset semantic segmentation performance.
| Setting | A-847 | PC-459 | A-150 | PC-59 | PAS-21 |
|---|---|---|---|---|---|
| SPOR (σₙ = 0) | 11.371 | 15.222 | 29.471 | 56.906 | 84.789 |
| Gaussian Noise (σₙ = 0.10) | 11.220 | 15.210 | 29.612 | 56.928 | 84.783 |
| Gaussian Noise (σₙ = 0.20) | 11.287 | 15.229 | 29.568 | 56.921 | 84.808 |
| Gaussian Blur (5 × 5, τ = 1 px) | 11.315 | 15.229 | 29.482 | 56.912 | 84.793 |
We welcome further discussion if any concerns remain.
To Q4.
Thank you for your detailed feedback.
To Q4:
We added a comparison with normalized depth to assess the effectiveness of SPOR’s ordinal-depth representation. Normalization standardizes the value range but retains numerical differences between predicted depth values. SPOR instead converts depth into normalized within-image ranks, making geometric modeling depend on relative ordering rather than specific depth magnitudes.
As shown in Tables E–F, using normalized depth reduces performance on nine of the ten metrics. Ordinal depth therefore yields better overall segmentation performance in these experiments, supporting SPOR’s use of relative ranks to organize geometric information and assist segmentation.
Table E: Segmentation performance comparison between ordinal and normalized depth on COCO and ADE20K.
| Setting | COCO PQ | COCO AP | COCO mIoU | ADE20K PQ | ADE20K AP | ADE20K mIoU |
|---|---|---|---|---|---|---|
| Ordinal depth | 55.534 | 46.224 | 65.330 | 22.873 | 14.405 | 29.471 |
| Normalized depth | 55.390 | 45.910 | 65.200 | 22.521 | 14.223 | 29.758 |
Table F: Performance comparison between ordinal and normalized depth on five semantic segmentation benchmarks (mIoU).
| Setting | A-847 | PC-459 | A-150 | PC-59 | PAS-21 |
|---|---|---|---|---|---|
| Ordinal depth | 11.371 | 15.222 | 29.471 | 56.906 | 84.789 |
| Normalized depth | 11.009 | 14.519 | 29.758 | 56.850 | 84.496 |
We welcome further discussion if any concerns remain.
To Q1&Q2.
Thank you for your detailed feedback.
To Q1:
We added two random seeds and evaluated all three seeds under the same predicted-depth inference setting. Tables A–B report the complete results. COCO PQ is 55.500 ± 0.048, with all three seeds outperforming ODISE, supporting the consistency of this modest gain across the evaluated runs. ADE20K PQ is 22.623 ± 0.216, indicating mean performance comparable to ODISE (22.6). These statistics are mean ± sample standard deviation. We will include the complete per-seed results and summary statistics in the revised manuscript.
While conducting the additional runs and checking the evaluation pipeline, we found that all original evaluations except COCO used RGB-only inference, bypassing ODFA and QOBR. This was inconsistent with the predicted-depth inference setting described in the manuscript. We will correct the main result tables and related analyses to consistently report predicted-depth inference results, and provide paired results from the same checkpoint under both inference settings in the appendix. The paired evaluation shows that predicted-depth inference improves mean PQ and AP on COCO and ADE20K while maintaining broadly comparable semantic segmentation performance. This supports ordinal geometry as a complementary cue for spatial grouping and instance prediction at test time. Accordingly, the revised main results will consistently use predicted-depth inference. The configuration without inference-time depth also retains comparable mean performance, offering flexibility in inference configuration. We thank the reviewer for prompting us to verify and correct this issue.
Table A: Multi-seed panoptic segmentation results of SPOR with predicted depth at inference on COCO and ADE20K, alongside the ODISE reference. Summary statistics are mean ± sample standard deviation across three seeds.
Table B: Multi-seed open-vocabulary semantic segmentation results of SPOR with predicted depth at inference, alongside the ODISE reference. Scores are mIoU (%); summary statistics are mean ± sample standard deviation across three seeds.
To Q2:
We added qualitative comparisons in the appendix (Figures 4–6), covering challenging scenes with same-class objects at different depths, partial occlusion, and overlapping contours. Compared with the baseline, SPOR delineates object boundaries more accurately (Figure 4) and separates same-class instances more clearly (Figure 5) in these examples. Figure 6 further presents representative results for instance assignment near occlusion boundaries and dense small-object segmentation.
We welcome further discussion if any concerns remain.