acceptodds
Under review as a conference paper at ICLR 2027

PointCRO: Class-Competition Relative Optimization for Point Cloud Segmentation

Abstract

Current regularization methods for point cloud segmentation typically rely on surrogate objectives defined over feature geometry or output distributions. Our diagnostics show that improvements in these objectives do not consistently translate into higher segmentation accuracy, motivating explicit modeling of point-wise class competition. We observe that many prediction errors arise from imbalanced competition among the ground-truth class, the most competitive incorrect class (TopWrong), and the remaining classes. To address this issue, we propose PointCRO, a prediction-state-aware fine-tuning framework that combines reward-based output regulation with representation-level regularization. Specifically, Class-Group Relative Optimization (CGRO) formulates each point prediction as a class competition and uses competition-aware relative rewards to rebalance the ground-truth, TopWrong, and remaining classes, directly correcting unfavorable prediction states rather than optimizing generic output statistics. Reward-guided Feature-level Kullback–Leibler (RFKL) regularization transfers these reward-induced preferences to pairwise feature relations, aligning representation structure with the desired competition dynamics and stabilizing the resulting prediction correction. Experiments across multiple benchmarks demonstrate consistent improvements in semantic segmentation performance. Endpoint prediction-transition analyses further show that PointCRO better balances error correction and the preservation of correct predictions, supporting prediction-state-aware class competition as a more direct optimization target and reducing the mismatch between surrogate regularization gains and segmentation improvements.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Point Cloud Segmentation, Class Competition, Reward-guided Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cfSdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PointCRO is a fine-tuning framework for point-cloud semantic segmentation. CGRO derives class-wise rewards from an epoch-frozen reference model using ground-truth confidence, the strongest incorrect competitor, and their margin, then optimizes normalized advantages through clipped probability ratios. RFKL uses the resulting advantage gaps to weight same-class feature references in a KL objective. Experiments cover multiple indoor and outdoor datasets and backbones, with matched continued-training baselines.

Critical strengths

The correction-versus-degradation analysis provides a useful view of why similar aggregate regularization objectives can have different segmentation outcomes.

Critical weaknesses
  1. The closest simple alternative to CGRO is not isolated by the existing controls. Table 7 compares scalar point-wise CE reweighting, while CGRO changes class-wise optimization directions through all-class probability ratios. Appendix A.6 discusses focal-style and margin objectives conceptually, but does not provide a matched direct class-wise objective using the same frozen-reference reward information without the ratio construction. Removing clipping or normalization changes individual ingredients rather than testing that alternative. Given the many reward-design choices and the 0.5 mIoU advantage over RWCE+RFKL in Table 7, the evidence establishes effectiveness more clearly than the necessity of this particular relative-optimization formulation.

  2. The representation analysis is conditioned on different success-selected point sets. Table 9 and Appendix A.10 evaluate each method on its own corrected points. The much larger same-class similarity change for PointCRO therefore mixes representation changes with differences in which points each method corrects. As Appendix A.10 explicitly acknowledges, this analysis cannot by itself establish that reward-guided feature support explains the extra corrections or stabilizes them during training. The uniform-KL ablation supports an aggregate contribution from weighting, but the stronger mechanistic interpretation requires a comparison on common point populations; endpoint transitions alone do not resolve this selection effect.

Questions for authors
  1. What evidence distinguishes CGRO from a direct class-wise optimization objective using the same frozen-reference rewards and matched tuning budget?

  2. Does the feature-support advantage persist on a common point set rather than each method's separately selected corrected points?

4 comments
AuthorsOpenReview ↗

Response to Reviewer cfSd

R3W1&R3Q1: What evidence distinguishes CGRO from a direct class-wise optimization objective using the same frozen-reference rewards and matched tuning budget?

R3A1: Thank you for this insightful suggestion. To isolate the contribution of the relative optimization formulation, we conducted a direct comparison against the closest alternative class-level objective (adv_logprob) under a matched training and tuning budget. This alternative retains the exact same frozen-reference reward construction, detached class-wise advantage computation, reference policy synchronization protocol, and expectation weighting as CGRO, while replacing the probability ratio and clipping surrogate with a direct objective based on signed advantage-weighted class log probabilities. All other configurations, including RFKL, initialization, random seeds, loss coefficients, batch size, and evaluation protocols, remain strictly identical.

Output objectivemIoU ↑
Direct class-level objective (adv_logprob) + RFKL78.2%
CGRO + RFKL78.6%

Under the evaluated setting, the simplified direct objective achieves 78.2% mIoU, leaving a substantial performance gap of 0.4 percentage points below the 78.6% mIoU attained by CGRO. Unlike direct log-probability optimization, which applies unbounded updates proportional to advantage magnitudes and risks over-penalizing or over-reinforcing ambiguous boundary points, the probability ratio and clipping operations in CGRO anchor policy updates to the frozen reference distribution. This bounded relative optimization prevents destructive gradient steps on noisy 3D point predictions, thereby translating identical reference-derived competition signals into superior segmentation accuracy. We will incorporate this controlled comparison, the formal objective definition, and experimental details into the revised manuscript and supplementary material.

R3W2&R3Q2: Does the feature-support advantage persist on a common point set rather than each method's separately selected corrected points?

R3A2: Thank you for raising this question regarding the impact of point-set selection. To ensure a fair evaluation, we recompute the feature metrics on three fixed populations corresponding to the points successfully corrected by CGRO, CGRO + UFKL, and PointCRO, respectively. Within each fixed population, all three methods are evaluated on the exact same point IDs, with the baseline-selected reference identities and TopWrong classes held constant.As detailed in the table below, PointCRO achieves the largest same-class support increase (ΔSGT\Delta S_{\text{GT}}) across all three fixed populations, including those corrected by the alternative methods. This consistency demonstrates that the performance advantage persists when evaluated on identical point sets and is not an artifact of selecting PointCRO's own successful corrections. Furthermore, PointCRO maintains positive relative margin changes (ΔMfeat\Delta M_{\text{feat}}) across all populations, confirming that its increase in ground-truth class support consistently outweighs the increase in TopWrong support. We will include this table and the corresponding evaluation protocol in Appendix.

Fixed point populationMethodΔSGT\Delta S_{\text{GT}}ΔSTW\Delta S_{\text{TW}}ΔMfeat\Delta M_{\text{feat}}
Corrected by CGRO (623,013 points)CGRO+0.058963−0.089326+0.148289
-CGRO + UFKL−0.001680−0.179472+0.177792
-PointCRO+0.333579+0.150770+0.182809
Corrected by CGRO + UFKL (647,763 points)CGRO+0.044097−0.077834+0.121931
-CGRO + UFKL+0.030939−0.209985+0.240924
-PointCRO+0.331329+0.153773+0.177556
Corrected by PointCRO (681,137 points)CGRO+0.040201−0.076494+0.116695
-CGRO + UFKL−0.007194−0.183610+0.176417
-PointCRO+0.366003+0.119659+0.246343

Dear Reviewer cfSd,

We sincerely appreciate your valuable suggestions. As the discussion period is nearing its end, with only a few days remaining, and to ensure that we have sufficient time to provide additional materials later, we kindly ask whether our responses have addressed your concerns. We are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Best regards,

Response to the Authors

Thank you for the detailed rebuttal and additional experiments. The new results substantially address my original concerns, although some mechanistic interpretations still require more cautious wording.

Regarding R3W1/R3Q1:

The newly introduced adv_logprob baseline provides a relevant and well-controlled comparison, directly addressing the missing class-wise alternative identified in my initial review.However, the 0.4-point improvement is relatively modest, and its robustness remains unclear without run-to-run variability. Moreover, the claim that probability ratios and clipping prevent destructive gradient updates is not directly established by this comparison. I suggest including the precise objective and experimental details while presenting the stability explanation more cautiously.

Regarding R3W2/R3Q2:

The fixed-population analysis effectively addresses my concern about selection bias. PointCRO consistently achieves larger ground-truth class feature-support increases across identical point sets, providing stronger evidence for the reported representation differences. Although, these results demonstrate consistent feature differences but do not establish that RFKL-induced support directly causes additional corrections or improved training stability.

Overall Assessment:

The rebuttal and additional experiments have strengthened the paper's empirical support to some extent. I maintain my weak accept recommendation (Rating: 3) and encourage the authors to incorporate the new evidence and appropriately qualify their conclusions.

Dear Reviewer cfSd,

We sincerely thank you for your thoughtful follow-up and insightful suggestions. Your emphasis on closely matched controls and common evaluation points helped us sharpen the evidence for the CGRO formulation and strengthen the feature-support analysis. We have incorporated these comparisons into the revised manuscript. Revised passages are highlighted in red. For subsections revised in full, the subsection headings are highlighted in red.

  • R3W1 & R3Q1. Appendix A.13, Table 26, compares CGRO with a direct class-wise objective using the same frozen-reference rewards and matched training and tuning budgets. CGRO achieves 78.6% mIoU, compared with 78.2% for the direct alternative.
  • R3W2 & R3Q2. Appendix A.10, Table 21, compares feature support on three common point sets. PointCRO achieves the largest same-class support increase across all three sets. We have also revised Table 9 and its accompanying discussion in the main text to evaluate all methods on the same set of points corrected by CGRO.

We greatly appreciate your engagement throughout the discussion. Your feedback has helped us make the controlled comparisons more informative and the conclusions more precise. We would welcome your assessment of these revisions and any remaining concerns.

Best regards,

The Authors

Reviewer_d4R9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper proposes PointCRO, a fine-tuning method for point cloud semantic segmentation that takes into account the relationship between the correct class of each point and the most competing incorrect class.
  • PointCRO consists of two parts. CGRO adjusts the output probabilities by assigning different signals to the correct class, the TopWrong class with the highest probability, and the remaining incorrect classes. Using the same information, RFKL normalizes the relationships among points of the same class in the feature space.
  • The authors evaluate PointCRO on indoor and outdoor point cloud segmentation datasets and various backbones, and present comparisons with existing regularization methods as well as component-wise ablation.
Critical strengths
  • The problem setup is practical. Confusion can occur between classes in point cloud segmentation. I think it is a reasonable approach to focus on the relationship between the correct-answer class and the most competitive incorrect-answer class.
  • Evaluated across multiple datasets and backbones. It handles both indoor and outdoor datasets, and the comparisons reported in the paper show performance improvements over the baseline across multiple backbones. In particular, in Table 2, the VOLT-S improves from 77.3 to 78.6 mIoU in ScanNetV2.
  • The inference structure is not changed. Since PointCRO is used during fine-tuning, the structure and number of parameters of the existing backbone can be maintained during inference, as described in the paper.
Critical weaknesses
  • It is difficult to determine the consistency of some performance improvements. Tables 2–4 present the standard deviation for the three-replication experiments of the PointCRO results, but the standard deviation is not shown for the baseline. Since the performance difference is small in some cases, it is recommended to compare the baseline and the results for each seed from PointCRO.
  • The comparison targets in feature analysis vary depending on the method. Table 9 measures feature changes for the points corrected by each method. As the authors explain in Appendix A.10, the composition of the set can also affect the difference in results, as the sets of corrections differ among methods. If there is a comparison on the same set of points, it will help in understanding the analysis.
Questions for authors
  • Can you provide seed-by-seed results for the baseline and PointCRO in Tables 2–4? If possible, please also report performance differences under the same seed conditions.
  • Can the analysis methods of Table 9 be compared even with the same set of points? For example, the results for the points corrected by the comparison methods seem helpful.
  • Could you specify the number of fine-tuning epochs and initial checkpoints used for each dataset and the backbone?
6 comments
AuthorsOpenReview ↗

Response to Reviewer d4R9-1

R2W1 & R2Q1. It is difficult to determine the consistency of some performance improvements. Can you provide seed-by-seed results for the baseline and PointCRO in Tables 2–4? If possible, please also report performance differences under the same seed conditions.

R2A1. Thank you for requesting a comparison under matched random seeds. We have completed the additional experiments and now report results for three seeds in each of 12 backbone–dataset settings from Tables 2–4. PointCRO improves mIoU in all 36 paired comparisons, with gains ranging from 0.23 to 2.16 percentage points.

For most settings, we initialized PointCRO from checkpoints released by the original authors. When the corresponding checkpoints were unavailable, we reproduced the baseline and initialized PointCRO from the resulting checkpoints. The reproduced baselines achieved mIoU close to the scores reported by the original authors. For the main comparisons in Tables 2–4, we retain the originally reported scores as the baseline reference. The tables below report the baseline and PointCRO scores for each matched seed so that readers can assess the consistency of the gains directly. Each summary row reports the mean ± standard deviation across three seeds, using n=3n=3 as the variance denominator.

VOLT on ScanNet200

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
2803440135.4936.46+0.97
3273378135.0736.61+1.54
4338945535.8536.32+0.47
Summary35.47 ± 0.3236.46 ± 0.12+0.99

VOLT on ScanNet

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
6024103077.5278.61+1.09
1149983877.4078.46+1.06
4828996177.4778.63+1.16
Summary77.46 ± 0.0578.57 ± 0.08+1.10

VOLT on ScanNet++

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
1162216449.2051.09+1.89
2356668549.4451.21+1.77
2653908249.2851.44+2.16
Summary49.31 ± 0.1051.25 ± 0.15+1.94

VOLT on nuScenes

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
5618700281.2181.68+0.47
840150181.3081.67+0.37
235130881.4281.90+0.48
Summary81.31 ± 0.0981.75 ± 0.11+0.44

VOLT on Semantic-KITTI

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
4966113669.4270.78+1.36
1489503169.2370.76+1.53
2961077769.0270.57+1.55
Summary69.22 ± 0.1670.70 ± 0.09+1.48

VOLT on Waymo

SeedVOLT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
5478496971.4371.87+0.44
759888171.4971.79+0.30
163230871.0271.66+0.64
Summary71.31 ± 0.2171.77 ± 0.09+0.46

LitePT on nuScenes

SeedLitePT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
9970406981.6182.46+0.85
333092281.6682.57+0.91
1447143881.6982.41+0.72
Summary81.65 ± 0.0382.48 ± 0.07+0.83

LitePT on Waymo

SeedLitePT baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
5103826972.5473.23+0.69
5382600172.4873.38+0.90
7976203171.8473.31+1.47
Summary72.29 ± 0.3273.31 ± 0.06+1.02

OA-CNN on ScanNet

SeedOA-CNN baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
1627268876.1476.45+0.31
5489468375.8576.38+0.53
7736947875.9976.42+0.43
Summary75.99 ± 0.1276.42 ± 0.03+0.42

OA-CNN on ScanNet++

SeedOA-CNN baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
8454765745.8846.25+0.37
2346455445.7246.06+0.34
3359459745.8346.12+0.29
Summary45.81 ± 0.0746.14 ± 0.08+0.33

CamPoint on S3DIS

SeedCamPoint baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
775682.8183.54+0.73
411883.2683.61+0.35
239883.4583.68+0.23
Summary83.17 ± 0.2783.61 ± 0.06+0.44

PointMC on S3DIS

SeedPointMC baseline mIoU (%)PointCRO mIoU (%)Δ mIoU (pp)
442183.6484.15+0.51
626883.6384.08+0.45
565083.5784.22+0.65
Summary83.61 ± 0.0384.15 ± 0.06+0.54
AuthorsOpenReview ↗

Response to Reviewer d4R9-2

R2W2&R2Q2: Can the analysis methods of Table 9 be compared even with the same set of points? For example, the results for the points corrected by the comparison methods seem helpful.

R2A2: Thank you for raising this question regarding the impact of point-set selection. To ensure a fair evaluation, we recompute the feature metrics on three fixed populations corresponding to the points successfully corrected by CGRO, CGRO + UFKL, and PointCRO, respectively. Within each fixed population, all three methods are evaluated on the exact same point IDs, with the baseline-selected reference identities and TopWrong classes held constant.As detailed in the table below, PointCRO achieves the largest same-class support increase (ΔSGT\Delta S_{\text{GT}}) across all three fixed populations, including those corrected by the alternative methods. This consistency demonstrates that the performance advantage persists when evaluated on identical point sets and is not an artifact of selecting PointCRO's own successful corrections. Furthermore, PointCRO maintains positive relative margin changes (ΔMfeat\Delta M_{\text{feat}}) across all populations, confirming that its increase in ground-truth class support consistently outweighs the increase in TopWrong support. We will include this table and the corresponding evaluation protocol in Appendix.

Fixed point populationMethodΔSGT\Delta S_{\text{GT}}ΔSTW\Delta S_{\text{TW}}ΔMfeat\Delta M_{\text{feat}}
Corrected by CGRO (623,013 points)CGRO+0.058963−0.089326+0.148289
-CGRO + UFKL−0.001680−0.179472+0.177792
-PointCRO+0.333579+0.150770+0.182809
Corrected by CGRO + UFKL (647,763 points)CGRO+0.044097−0.077834+0.121931
-CGRO + UFKL+0.030939−0.209985+0.240924
-PointCRO+0.331329+0.153773+0.177556
Corrected by PointCRO (681,137 points)CGRO+0.040201−0.076494+0.116695
-CGRO + UFKL−0.007194−0.183610+0.176417
-PointCRO+0.366003+0.119659+0.246343

R2Q3: Could you specify the number of fine-tuning epochs and initial checkpoints used for each dataset and the backbone?

R2A3: We primarily initialize PointCRO from checkpoints publicly released by the original authors for the corresponding backbone and dataset. When an official checkpoint is unavailable, we train the baseline backbone ourselves, use the resulting checkpoint to initialize PointCRO, and report the reproduced baseline performance alongside the result reported in the original paper. For PointMC on S3DIS, our reproduced checkpoint achieves 83.6% mIoU, matching the published baseline result at the reported precision. The table below lists the initialization checkpoint, the number of PointCRO fine-tuning epochs, and the number of epochs in the original backbone training schedule for each configuration.

BackboneDatasetInitialization checkpointPointCRO fine-tuning epochsOriginal backbone training epochs
VOLTScanNetOfficial checkpoint80800
VOLTScanNet++Official checkpoint50800
VOLTScanNet200Official checkpoint50800
VOLTnuScenesOfficial checkpoint1050
VOLTSemanticKITTIOfficial checkpoint1050
VOLTWaymoOfficial checkpoint2550
LitePTnuScenesOfficial checkpoint2050
LitePTWaymoOfficial checkpoint2050
OACNNScanNetReproduced OACNN checkpoint (76.1% mIoU)80800
OACNNScanNet++Reproduced OACNN checkpoint (45.8% mIoU)80900
CamPointS3DISOfficial checkpoint30150
PointMCS3DISReproduced PointMC checkpoint (83.6% mIoU)30150

Dear Reviewer d4R9,

We sincerely appreciate your valuable suggestions. As the discussion period is nearing its end, with only a few days remaining, and to ensure that we have sufficient time to provide additional materials later, we kindly ask whether our responses have addressed your concerns. We are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Best regards,

Reviewer_xv6yOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PointCRO, a fine-tuning framework for point cloud semantic segmentation. CGRO uses reference predictions to assign relative rewards to the ground-truth class, TopWrong, and remaining classes, and optimizes clipped probability ratios. RFKL uses the resulting advantages to weight same-class feature references. Experiments across indoor and outdoor benchmarks show consistent improvements over multiple backbones. The main contribution is a shared class-competition signal for output and feature regularization without changing the inference architecture.

Critical strengths
  1. Studying error correction together with preservation of correct predictions is a useful direction. The transition analysis separates corrections from newly introduced errors and shows that PointCRO reduces degradation. This provides information beyond aggregate segmentation metrics.

  2. The evaluation covers multiple indoor and outdoor datasets and several backbones. Comparisons use the same initialization and fine-tuning duration, with component ablations and conventional regularizers. These results provide useful evidence that the framework can improve existing segmentation models.

Critical weaknesses

1.The experiments establish the usefulness of the complete framework more clearly than the need for its particular formulation. CGRO adapts group-relative advantages and probability-ratio clipping, while RFKL is gradient-equivalent to weighted supervised contrastive learning. Scalar CE reweighting is evaluated, but it remains unclear whether the shared competition information would provide similar gains with a simpler output objective or feature-weighting rule.

2.The interpretation of the RFKL reference weights needs further evidence. The normalized advantage gap need not increase as confidence decreases or the margin between the ground-truth class and TopWrong worsens. Smaller scores therefore do not automatically identify better references. Endpoint comparisons do not establish persistent correction during training, and feature comparisons use each method's own corrected-point set. A common point set and training-time transition analysis would help assess the claimed mechanism.

3.The trade-off between training cost and accuracy gains also needs evaluation. Table 19 reports a 28.1% increase in training time per iteration for a 1.3-point mIoU gain on ScanNetV2. Most of the overhead comes from RFKL: adding it to CGRO increases iteration time from 374 ms to 460 ms, while mIoU rises from 77.9 to 78.6. The inference architecture is unchanged, but the benefit of the more expensive feature objective needs clearer justification.

Questions for authors
  1. With the same reference predictions and competition information, do the gains remain when a simpler output objective is used? How do confidence- or margin-based reference weights compare with the advantage-gap weights under the same feature-loss configuration?

  2. How well does the normalized advantage gap identify suitable same-class references? Can the authors compare feature changes on a common point set and report prediction transitions during training to assess whether corrected predictions remain correct?

  3. Can a lighter feature regularizer achieve accuracy comparable to RFKL with less training overhead? What additional evidence supports the trade-off between the increase from 374 ms to 460 ms per iteration and the mIoU gain from 77.9 to 78.6?

6 comments
AuthorsOpenReview ↗

Response to Reviewer xv6y-1

R1W1&R1Q1-V1: With the same reference predictions and competition information, do the gains remain when a simpler output objective is used?

R1A1: Thank you for raising this question regarding the necessity of the CGRO formulation. To isolate the effect of the objective function, we compare CGRO against a simplified signed advantage-weighted log-probability objective (adv_logprob). Both variants share identical reference predictions, detached class-wise advantages, reward constructions, expectation weighting, and RFKL configurations, except that the control directly optimizes advantage-weighted class log probabilities without the probability ratio and clipping mechanisms. Under the evaluated setting, the simplified objective achieves 78.2% mIoU, leaving a substantial performance gap of 0.4 percentage points below the 78.6% mIoU attained by CGRO. This discrepancy confirms that the probability ratio and clipping operations are indispensable for bounding policy updates and preventing over-optimization, thereby demonstrating the necessity of the proposed formulation. We will include the formal objective definition and experimental details in the supplementary material.

Output objectivemIoU (%)
adv_logprob78.2
CGRO78.6

R1W1&R1Q1-V2: How do confidence- or margin-based reference weights compare with the advantage-gap weights under the same feature-loss configuration?

R1A2: Thank you for suggesting a direct comparison with simpler reference weighting rules. We conducted two additional experiments with CGRO and the feature loss fixed, varying only the weights assigned to same-class references. The two alternatives use the predicted probability of the ground-truth class and its probability margin over the highest-probability incorrect class (TopWrong), respectively. All three variants share the same initialization, reference policy synchronization protocol, and training and evaluation settings. Advantage-gap weighting outperforms confidence and margin weighting by 0.4 and 0.6 percentage points, respectively. These results support the advantage gap as a more effective signal for weighting same-class references under the same feature loss.

Positive-reference weightingmIoU (%)
Ground-truth confidence78.2
Ground-truth–TopWrong probability margin78.0
RFKL78.6

R1W2&R1Q2-V1: The interpretation of the RFKL reference weights needs further evidence. Smaller scores do not automatically identify better references.

R1A3: Thank you for highlighting the need to validate the interpretation of the reference scores. We directly examined whether smaller scores are associated with higher prediction reliability. Using cached baseline predictions for 35,881,054 valid points from 312 ScanNetV2 validation scenes, we divided points into five within-group score quantiles. For each quantile, we computed class-balanced prediction accuracy, ground-truth class probability, and the probability margin between the ground-truth class.

Score quantilePrediction accuracy ↑GT probability ↑GT–TopWrong margin ↑
Q1: lowest100.00%0.97010.9680
Q2100.00%0.96760.9647
Q3>99.99%0.96240.9573
Q497.04%0.90160.8588
Q5: highest50.28%0.45160.0096
AuthorsOpenReview ↗

Response to Reviewer xv6y-2

R1W2&R1Q2-V2: Do corrected predictions remain correct across subsequent training checkpoints?

R1A3: Thank you for suggesting an analysis of prediction transitions during training. We initialized CGRO, CGRO + UFKL, and PointCRO from the same baseline checkpoint and tracked the same 2,999,421 validation points that the baseline misclassified before fine tuning. We evaluated all three methods every 10 epochs through epoch 80. For each observation window, we define continuous retention as the proportion of points already corrected at the start of the window that remain correctly classified at every subsequent observed checkpoint. We also measured retention over the 20 epochs following the first observed correction of each point, using points with a complete observation window. PointCRO shows lower retention in the two early windows. It achieves the highest retention in both later windows. These results indicate that PointCRO offers an advantage in sustaining corrections across successive checkpoints.

Observation windowCGRO (%)CGRO + UFKL (%)PointCRO (%)
Epoch 10 to 3051.8152.6246.60
Epoch 20 to 4055.7555.2148.48
Epoch 40 to 6060.1161.0761.83
Epoch 60 to 8071.7770.0176.09
20 epochs after the first observed correction47.1649.0450.03

R1W2&R1Q2-V3: How do feature changes compare across methods when evaluated on the same set of points?

R1A4: Thank you for raising this question regarding the impact of point-set selection. To ensure a fair evaluation, we recompute the feature metrics on three fixed populations corresponding to the points successfully corrected by CGRO, CGRO + UFKL, and PointCRO, respectively. Within each fixed population, all three methods are evaluated on the exact same point IDs, with the baseline-selected reference identities and TopWrong classes held constant.As detailed in the table below, PointCRO achieves the largest same-class support increase (ΔSGT\Delta S_{\text{GT}}) across all three fixed populations, including those corrected by the alternative methods. This consistency demonstrates that the performance advantage persists when evaluated on identical point sets and is not an artifact of selecting PointCRO's own successful corrections. Furthermore, PointCRO maintains positive relative margin changes (ΔMfeat\Delta M_{\text{feat}}) across all populations, confirming that its increase in ground-truth class support consistently outweighs the increase in TopWrong support. We will include this table and the corresponding evaluation protocol in Appendix.

Fixed point populationMethodΔSGT\Delta S_{\text{GT}}ΔSTW\Delta S_{\text{TW}}ΔMfeat\Delta M_{\text{feat}}
Corrected by CGRO (623,013 points)CGRO+0.058963−0.089326+0.148289
-CGRO + UFKL−0.001680−0.179472+0.177792
-PointCRO+0.333579+0.150770+0.182809
Corrected by CGRO + UFKL (647,763 points)CGRO+0.044097−0.077834+0.121931
-CGRO + UFKL+0.030939−0.209985+0.240924
-PointCRO+0.331329+0.153773+0.177556
Corrected by PointCRO (681,137 points)CGRO+0.040201−0.076494+0.116695
-CGRO + UFKL−0.007194−0.183610+0.176417
-PointCRO+0.366003+0.119659+0.246343

R1W3&R1Q3-V1: The trade-off between training cost and accuracy gains also needs evaluation. The benefit of the more expensive feature objective needs clearer justification. Can a lighter feature regularizer achieve accuracy comparable to RFKL with less training overhead?

R1A5: Thank you for raising this question regarding the trade-off between computational overhead and segmentation accuracy. RFKL is specifically designed to inject output-space class competition into feature-level regularization by constructing non-uniform reference weights from class-relative advantages, rather than enforcing generic feature compactness. In our evaluations, lighter feature-regularization alternatives such as CenterLoss and prototype-based contrastive regularization fail to match the segmentation accuracy of the full PointCRO framework, confirming that generic feature constraints cannot reproduce the observed gains. Moreover, replacing the proposed advantage-based reference weights with simpler confidence-based or margin-based heuristics reduces mIoU from 78.6% to 78.2% and 78.0%, respectively. Notably, while these heuristic rules simplify the weight definition, they still require pairwise feature computations and thus offer no meaningful reduction in training overhead. Together, these findings justify the design of RFKL and demonstrate that the additional computational cost translates into substantial accuracy improvements that lighter alternatives cannot achieve.

AuthorsOpenReview ↗

Response to Reviewer xv6y-3

R1W3&R1Q3-V2: What additional evidence supports the trade-off between the increase from 374 ms to 460 ms per iteration and the mIoU gain from 77.9 to 78.6?

R1A6: Thank you for highlighting the need for additional evidence regarding this trade-off. To investigate whether the accuracy gain can be preserved under different computational budgets, we conducted an ablation study on the RFKL sampling budget. Specifically, we varied the number of points sampled for RFKL computation while keeping the network architecture, reference-weighting rule, loss coefficients, and training protocols strictly identical. As summarized in the table below, all tested sampling budgets achieve higher mIoU than the baseline augmented with CGRO. Notably, reducing the sampling budget from 4096 to 1024 points maintains a consistent segmentation accuracy of 78.4% mIoU, while the iteration time remains largely invariant at around 454 to 460 ms. This behavior indicates that the additional training overhead is an inherent and necessary cost required to establish and optimize feature-level competition, rather than a bottleneck that scales linearly with the sample size. Furthermore, because RFKL is applied exclusively during training and introduces no additional parameters or computational overhead during inference, this necessary training cost is strictly confined. We believe these results demonstrate that the overhead is both bounded and well justified by the consistent accuracy improvements.

MethodRFKL sampling budgetmIoU (%) ↑Training time (ms/iter) ↓
Base+CGRO—77.9374
PointCRO409678.6460
PointCRO204878.4454
PointCRO102478.4455

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Our Responses

We thank all three reviewers for their thoughtful feedback. Reviewer xv6y encouraged us to test simpler objectives and reference weighting rules, examine how corrections persist during training, and better quantify the trade-off between accuracy and training cost. Reviewer d4R9 asked whether the gains were consistent across matched random seeds and requested clearer reporting of the initialization checkpoints and fine-tuning schedules. Reviewer cfSd asked for a direct comparison with a class-wise objective using the same frozen-reference rewards. All three reviewers also encouraged us to compare feature changes on common point sets. These suggestions helped us design more targeted controls and sharpen the interpretation of our results.

  • Objective and reference weighting. With the same frozen-reference rewards and matched training settings, CGRO with RFKL achieved 78.6% mIoU, compared with 78.2% for the direct class-wise objective with RFKL. Under the same feature loss, advantage-gap weighting also outperformed confidence-based and margin-based weighting. The score analysis further showed that lower score quantiles were associated with more reliable predictions on ScanNetV2.
  • Feature changes and correction retention. PointCRO produced the largest increase in ground-truth class feature support on all three fixed point sets, including those defined by corrections made by the comparison methods. Its correction retention was lower in the early observation windows and highest in both later windows. PointCRO also achieved the highest retention over the 20 epochs following the first observed correction.
  • Consistency and experimental protocol. PointCRO improved mIoU in all 36 matched-seed comparisons across 12 backbone and dataset combinations, with gains ranging from 0.23 to 2.16 percentage points. We also documented the checkpoint source and fine-tuning schedule for each setting.
  • Training cost. Reducing the RFKL sampling budget from 4,096 to 1,024 points retained 78.4% mIoU, compared with 78.6% at the full budget, while yielding only a modest reduction in iteration time. RFKL is used during training and adds no inference cost.

We are grateful for the time and care the reviewers devoted to this work. Their questions helped us substantiate the consistency of the gains and describe more precisely how PointCRO affects feature support and correction retention.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.