- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents ReliaSplat, a training-free feature enhancement framework designed to tackle input feature unreliability in Language Gaussian Splatting. The authors observe that while existing works emphasize 3D representation, compression, and rendering aggregation, they largely overlook inherent errors in upstream 2D VLM features caused by occlusions, scale shifts, and region-partitioning noise. ReliaSplat addresses this at the input level via three integrated mechanisms: (1) Cross-View Reliability-Aware Aggregation, which models observation reliability via multi-view semantic consistency to penalize outlier features; (2) Consensus-Guided Feature Refinement, which utilizes the aggregated 3D consensus to correct noisy 2D features through a 3D -> 2D -> 3D projection cycle; and (3) Geometry-Aligned Semantic Propagation, which densifies semantic coverage using pseudo-camera viewpoints under geometric consistency constraints. Evaluated on LERF and ScanNet, the framework demonstrates plug-and-play generalizability by replacing input features across four downstream baselines (LangSplat, LangSplatV2, VALA, and THGS), yielding consistent mIoU gains.
Critical strengths
S1. Well-motivated perspective shifting from 3D representation engineering to input feature reliability: The vast majority of existing 3D language Gaussian splatting pipelines treat pre-extracted 2D VLM features as ground-truth semantic observations, focusing their innovations almost entirely on downstream distillation, codebook quantization, or feature field rendering. ReliaSplat directly targets the upstream bottleneck ("garbage in, garbage out"). Crucially, the authors distinguish between geometric/rendering significance and semantic consistency, correctly arguing that high rendering contribution does not imply semantic correctness. Formulating observation reliability via multi-view cross-view agreement provides an intuitive, theoretically grounded, and training-free inductive bias for 3D language fields.
S2. Genuine plug-and-play applicability supported by extensive cross-framework validation: Rather than coupling the denoising mechanism to a custom 3D architecture, the framework is designed to output cleaned, standardized 2D language features that can be fed directly into existing pipelines. The authors substantiate this generality through rigorous experiments across four major downstream architectures (LangSplat, LangSplatV2, VALA, and THGS) under their respective official training scripts and benchmark protocols on LERF and ScanNet. Achieving consistent mIoU improvements across all four frameworks confirms that the feature enhancements provide practical, architecture-agnostic utility to the community.
Critical weaknesses
W1. Confounding data augmentation via pseudo-camera pose generation undermines fair baseline comparison: In Section 4.4 and Appendix A.3, the Geometry-Aligned Semantic Propagation module synthesizes pseudo-camera poses to propagate features and supplement valid semantic observations. However, standard downstream baselines (LangSplat, THGS, VALA) operate strictly on the original training viewpoints. Synthesizing novel pseudo-view observations effectively performs multi-view semantic data augmentation. Consequently, the performance gains reported in Table 1 and Section 5.4 are confounded: it is currently impossible to isolate whether the downstream improvements stem from the core reliability modeling (reliability-aware aggregation and consensus refinement) or simply from training on a denser, augmented set of viewpoints. The paper lacks a strictly controlled baseline where downstream methods either receive the same pseudo-camera dense views, or where ReliaSplat is evaluated strictly without pseudo-camera interpolation.
W2. Metric divergence between mIoU gains and stagnant mAcc raises questions regarding the nature of semantic correction: In the cross-framework evaluation, the paper notes that ReliaSplat consistently boosts mean mIoU while keeping mean mAcc "stable overall." If the framework genuinely corrects noisy, misclassified VLM observations back to their true semantic categories, categorical accuracy (mAcc) should experience a concurrent increase. The stagnation of mAcc suggests that the 3D -> 2D -> 3D filtering and propagation cycles may primarily act as a spatial low-pass filter that smooths object boundary masks and suppresses isolated false-positive outliers (which reduces the union in IoU and inflates mIoU), rather than actively repairing true semantic misclassifications on challenging or long-tail classes.
W3. Omission of wall-clock runtime and memory profiling for a framework claiming to be lightweight and training-free: While ReliaSplat is highlighted as "training-free," training-free does not automatically imply computationally inexpensive. The pipeline requires dense multi-view feature unprojection, pairwise cross-view cosine similarity calculations across dense SAM masks, iterative 3D -> 2D rasterization rendering, and novel-view feature propagation across multiple pseudo-cameras. Depending on scene resolution and viewpoint count, these offline pre-processing steps could incur massive CPU/GPU memory footprints and substantial wall-clock overhead. Without quantitative reporting of pre-processing latency versus downstream distillation training time, the paper's claims regarding "lightweight" and practical efficiency remain unsubstantiated.
Questions for authors
Q1. Disentangling Pseudo-Camera Data Augmentation: Could the authors provide a decoupled ablation study on the downstream frameworks (e.g., on LangSplat and THGS) evaluating: (a) ReliaSplat operating strictly on original training camera poses without pseudo-camera generation, and (b) baseline methods trained with naive pseudo-camera interpolated feature maps? How much of the observed mIoU gain is uniquely attributable to reliability-aware filtering versus viewpoint densification?
Q2. Detailed Breakdown of mIoU vs. mAcc Divergence: What is the underlying cause for the improvement in mean mIoU while mean mAcc remains flat across all four downstream methods? Could you provide per-class confusion matrices or accuracy delta breakdowns on ScanNet / LERF to clarify whether the performance delta is driven by boundary sharpening/denoising or genuine semantic label corrections?
Q3. Quantitative Profiling of Computational Overhead: What is the exact wall-clock pre-processing runtime and peak GPU VRAM consumption for each stage of ReliaSplat (Aggregation, Refinement, Propagation) on standard benchmark scenes (e.g., LERF figurines and ScanNet scenes) compared to the standard feature extraction and downstream training time of LangSplat?
Q4. Robustness to Systemic Multi-View Hallucinations: How does the cross-view consistency metric behave when an underlying VLM produces a shared systematic error across a majority of viewpoints (e.g., 4 out of 5 viewpoints incorrectly classifying an ambiguous object as class A, and only 1 close-up view correctly identifying it as class B)? In such cases, does the consensus mechanism suppress the single accurate observation as an outlier?
2 comments
Dear Reviewer,
As the rebuttal discussion period is approaching its deadline, we would appreciate it if you could review our response and share any further comments or concerns.
Thank you for your time and consideration.
We thank the reviewer for the careful review and constructive suggestions. These comments have helped us strengthen our experimental analysis and improve the clarity of our method description and presentation of results. We address your questions point by point below.
Revisions and additions are highlighted in blue in the updated PDF for ease of review.
W1: Data augmentation through pseudo-camera poses confounds the results and compromises fair baseline comparisons
Semantic propagation only constructs ReliaSplat's 3D semantic field. Table 6 reports its modest cost.
Table 1 evaluates the performance of the 3D semantic field constructed by ReliaSplat itself and provides a fair comparison with other methods.
Tables 4(C) and 13 isolate propagation's effect, showing mean mIoU gains when added to R2 on LERF and ScanNet.
Table 5 shows downstream gains from replacing only real-view inputs with R1 features, without view densification.
Table 9 shows sparse outputs (31.00% valid pixels at R2 versus 90.94% in source maps). Given the training costs and benefits for downstream methods, we aggregate them online into our own field without saving dense pseudo-view maps.
Q1: Disentangling Pseudo-Camera Data Augmentation
For (a), Table 5 shows that refining real-view features alone improves downstream performance without view densification.
Tables 4(A) and 3 show that consistency weighting and two-round feature refinement yield mean mIoU gains of 1.15 and 1.28 percentage points, respectively, without adding views, demonstrating the effectiveness of both components.
Q2: Detailed Breakdown of mIoU vs mAcc Divergence
Section 5.1 of the revised manuscript explains that LERF-OVS mAcc is mean localization accuracy, which checks whether the pixel with the highest CLIP response falls inside the ground-truth bounding box; mIoU measures overlap between predicted and ground-truth masks. Thus, mIoU can improve while mAcc remains unchanged when masks improve without changing the localization decision.
Figure 2 illustrates improved localization, with the localization points after refinement falling inside the ground-truth bounding boxes in all examples shown. Figure 3 illustrates improved masks, consistent with the mIoU gains reported in Table 5.
LERF-OVS uses open-vocabulary text queries without a unified space of mutually exclusive class labels, so conventional multiclass confusion matrices are not directly applicable. We add this analysis on ScanNet, which has fixed class labels.
Table 11 shows gains of 2.05, 1.58, and 1.72 percentage points in ScanNet 10-class mIoU, mAcc, and OA, respectively, with more corrected errors (23,377) than new errors (9,103), indicating that feature refinement improves semantic class predictions.
Table 12 and Figures 7 and 8 further report per-class accuracy, confusion matrices, and their differences, including degraded classes. The existing results support the presence of semantic class correction.
Q3: Quantitative Profiling of Computational Overhead
Table 6 reports detailed runtime and peak memory for each stage across the four LERF scenes.
For figurines, the runtime and peak GPU memory for the four stages are:
(1) Aggregation (including qi precomputation): 355.85 s, 12.67 GiB.
(2) R1 (cumulative from the Original feature maps): 396.85 s, 15.98 GiB.
(3) R2 (cumulative from the Original feature maps): 589.32 s, 15.98 GiB.
(4) Propagation: 187.04 s, 13.15 GiB.
Table 7 reports the costs and peak GPU memory for feature extraction, feature refinement, and feature-field construction by downstream methods.
Table 7 shows that ReliaSplat takes 776.36 s to construct its feature field, longer than VALA and THGS but approximately 81.8% and 94.6% less time than LangSplat and LangSplatV2, respectively.
Q4: Robustness to Systemic Multi-View Hallucinations
ReliaSplat targets open-vocabulary segmentation by directly aggregating and refining continuous CLIP features, without relying on predefined class labels or assigning discrete class labels to observations from individual views.
Cross-view CLIP agreement need not imply correctness; valid minority observations may be downweighted.
The refinement teacher is obtained by alpha-compositing the consensus features of multiple Gaussians and pooling them within SAM masks, allowing it to integrate regional evidence. Thus, an incorrect consensus at a single Gaussian does not independently determine refinement for the entire region; correct evidence from other Gaussians in the region may still contribute. However, if the erroneous consensus is widespread throughout the region, the mechanism cannot guarantee recovery of the correct semantics.
Figure 6 and Table 12 show gains in most scenes and most valid classes.
Thank you for considering our responses. We would be happy to clarify any points that remain unclear.