- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates an important question: whether RGB–geometry segmentation models can maintain reliable predictions when RGB textures become misleading. To address this issue, the authors introduce texture interventions during training to improve segmentation robustness. The experimental results can show improvement on dealing with texture-change in different settings.
Critical strengths
The analysis of RGB texture interventions is meaningful, as existing segmentation models may over-rely on texture cues associated with specific semantic categories.
The experiments on unseen appearance shifts, such as illumination changes and color shifts, also provide valuable evidence of the method's robustness and generalization ability.
Critical weaknesses
My main concern is whether the proposed replacement-based texture intervention truly encourages the model to learn better RGB–geometry fusion, or simply makes it rely more heavily on geometry while reducing its dependence on RGB. If the latter is the case, the improvements may primarily come from suppressing texture information rather than learning genuinely adaptive multimodal fusion.
Questions for authors
Is cross-category texture replacement the most appropriate intervention strategy? Replacing an object's texture with that of another semantic category creates relatively artificial conflicts that may not accurately reflect real-world distribution shifts. Would more realistic perturbations, such as illumination changes, color shifts, or sensor-induced appearance variations, provide more effective training signals for robust multimodal fusion?
Does the proposed method truly learn adaptive RGB–geometry fusion, rather than simply increasing its reliance on geometry? I suggest adding a controlled experiment where RGB information remains reliable while geometry is corrupted, for example, through coordinate noise, point displacement, or partial point dropout. Comparing the baseline and the intervention-trained model under these conditions would help determine whether the proposed training strategy preserves the model's ability to leverage reliable RGB cues when geometric information becomes unreliable.
Response to Reviewer 2W3g
Thank you for the constructive suggestions. We added analyses of RGB use and fusion weights, along with photometric-augmentation comparisons, and clarified the setting of the existing geometry-corruption experiments. Revisions are marked in green.
W1. Does the method learn adaptive fusion, or does it only rely more on geometry?
The model continues to use RGB when geometry degrades, and its fusion weights respond to which input is corrupted. The new analysis on ScanNetV2 with PointNeXt reports both weight responses and the effect of replacing RGB with the training-set mean color (Appendix F.3, Table 19; summarized in Sec. 4.4):
Drops are in full-scene mIoU percentage points, measured with each model's trained parameters fixed. Geometry corruption averages noise, jitter, and dropout at severity 3. A dash indicates an unreported measurement.
The geometry weight decreases under geometry corruption and increases under appearance corruption. The RGB-replacement result confirms that RGB remains useful when geometry degrades. The response is also local: under a partial texture swap, the mean geometry weight rises from 0.52 to 0.84 inside the affected region and stays at 0.50–0.52 outside it (Table 9, Fig. 3).
Geometry-only performs better under whole-scene appearance corruption, including external texture (68.04 vs. 55.40). Our model performs better on clean data (+1.25 mIoU), under geometry corruption (+16.14 mIoU), and on object-level texture conflicts (+7.38 TR; Tables 2 and 19). These diagnostics support input-dependent fusion weights and continued use of RGB, alongside reduced RGB sensitivity on clean data.
Q2. A controlled experiment with reliable RGB and corrupted geometry.
This setting is evaluated in Tables 3 and 17: Gaussian coordinate noise, bounded jitter, and random point dropout alter geometry while leaving RGB intact. None of these corruptions is used in our training. Sec. 4.3 now states this setting explicitly (first paragraph, L. 363–364).
These results demonstrate improved segmentation when geometry is corrupted and RGB remains intact. The RGB-replacement test in W1 confirms that RGB contributes to the predictions in this setting.
Q1. Is cross-category texture replacement the right intervention? Would realistic perturbations give a better training signal?
Photometric augmentation improves both models and provides additional gains when combined with our framework. Following your suggestion, Table 20 compares the baseline and our full framework, each with and without PhotoAug. This full-framework comparison tests complementarity with PhotoAug without isolating STS from the other components. PhotoAug applies random brightness, contrast, saturation, and hue shifts, plus random grayscale conversion, to the whole scene.
All entries are mIoU (%) on ScanNetV2 with PointNeXt. Clean, External texture, and RGB-C use full scenes; conflict-region mIoU uses affected points and evaluated target categories. External texture uses projection strength alpha = 1.0. RGB-C averages all five severities of fog, brightness, and hue shifts. †Brightness and hue families overlap with PhotoAug training; fog is excluded. External texture is held out for all four configurations.
With the same PhotoAug recipe, our framework improves external-texture mIoU by 13.47 points and conflict-region mIoU by 64.10 points over the augmented baseline, with a 0.64-point clean-mIoU cost. Without PhotoAug, our framework also exceeds the augmented baseline on external texture and semantic conflicts; Baseline + PhotoAug is slightly higher on RGB-C.
For practical training, the results support combining PhotoAug with our framework: it improves both clean accuracy and robustness to held-out external texture. The swap metrics provide controlled conflict diagnostics, and full-scene mIoU confirms gains under the evaluated simulated shifts.