acceptodds
Under review as a conference paper at ICLR 2027

VINO: Revisiting 3D Representation Learning with Volume Transformers

14 views

Abstract

Self-supervised representation learning has proven highly effective in 2D, but comparable success has yet to be achieved for 3D scenes. Motivated by this, we identify key limitations in current approaches to 3D representation learning and revisit their design choices. We first replace the commonly used hierarchical PTv3 backbone with the Volume Transformer, a ViT-like architecture for 3D scenes. We then redesign two core objectives of modern approaches built around teacher–student networks: matching representations across views and predicting masked content from visible context. For the former, we separate the roles of teacher and student: the teacher provides clean, consistent targets while the student learns invariance to stronger perturbations. For the latter, we prevent the student from accessing the masked geometry and instead require it to infer missing representations from visible context only. Finally, we redesign 2D-to-3D knowledge distillation by replacing direct regression of view-dependent image features with more view-consistent supervision. Together, these changes form VINO, a representation learning framework that learns semantic representations of 3D scenes without labels. VINO substantially improves dense linear probing across indoor and outdoor benchmarks and transfers strongly to out-of-domain settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
self supervised learning, 3D scene understanding, representation learning, point clouds

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9WZ1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces VINO, a self-supervised representation learning framework for 3D scenes based on teacher–student self-distillation. The authors systematically revisit the current 3D SSL recipe (e.g., Sonata) and propose four key design improvements: (1) adopting the isotropic Volume Transformer (Volt) to replace hierarchical backbones; (2) using asymmetric augmentations (clean global teacher vs. corrupted local student) with an independently learned prototype target head; (3) eliminating geometric shortcuts in masked modeling by removing masked tokens from the encoder and reconstructing them via a cross-attention decoder; and (4) performing view-consistent 2D-to-3D distillation by clustering in the frozen 2D VFM feature space. Evaluated on six indoor and outdoor benchmarks, VINO sets new state-of-the-art results on frozen linear probing and fine-tuning, while demonstrating strong out-of-domain transfer on zero-shot registration and single-view RGB-D probing.

Critical strengths

S1. The systematic redesign of 3D self-distillation is well-motivated and effective. Rather than proposing a single isolated heuristic, the paper carefully identifies key limitations in existing 3D self-distillation frameworks (e.g., Sonata) and addresses each with a principled solution. The step-by-step ablations cleanly isolate the orthogonal contribution of each design choice on ScanNet200. This cumulative improvement convincingly demonstrates that these design choices are complementary and form a cohesive, well-engineered framework.

S2. Strong out-of-domain transfer validates the general-purpose representation quality. Beyond standard in-domain semantic segmentation, VINO's frozen features transfer remarkably well to unseen tasks and domains: achieving competitive performance on cross-modal probing. These results substantiate that the learned representations capture intrinsic geometric correspondence and view-invariant semantics rather than merely overfitting to segmentation-specific priors.

Critical weaknesses

W1. The architectural contribution of the backbone is partially confounded with the algorithmic redesign. In Table 5, VINO (Volt-B, 85.7M) is compared against competing methods predominantly based on the hierarchical PTv3-B backbone . While Table 1 shows that replacing PTv3 with Volt under the Sonata baseline already yields a +1.3 mIoU gain, the full benchmark comparisons do not isolate how much of the final mIoU improvement over Sonata stems from the stronger Volt backbone itself versus the proposed self-distillation objectives. To cleanly disentangle these factors, the paper should ideally provide either: (a) competing methods (e.g., Sonata, Concerto) ported to the Volt backbone, or (b) VINO implemented on a PTv3 backbone. Without these controlled comparisons, it remains difficult to assess whether VINO's improvements over prior SOTA are driven primarily by the algorithmic innovations or the shift in backbone architecture.

W2. Outdoor performance gains are notably modest and lack in-depth analysis. While VINO achieves substantial improvements on indoor benchmarks, its gains on outdoor LiDAR datasets are significantly less pronounced. Under 3D-only pretraining (Table 5), VINO's 65.3 mIoU on Waymo falls 1.7 mIoU short of DOS (67.0), and its 68.5 mIoU on SemanticKITTI is only marginally higher than DOS (68.3). Since separate models are trained for indoor and outdoor settings, this performance disparity is striking. The paper lacks a dedicated discussion analyzing whether this plateau is due to the inherent sparsity of LiDAR point clouds, the absence of RGB features outdoors, or lower camera-LiDAR overlap in driving datasets. Providing this analysis is crucial for understanding the boundaries and failure modes of the framework.

Questions for authors

Q1. Disentangling Backbone vs. Algorithmic Gains: In Table 5, VINO (Volt-B) achieves impressive state-of-the-art results over prior methods (e.g., Concerto, Utonia, DOS), yet all competing approaches rely on the hierarchical PTv3 architecture. Given that Table 1 already shows a +1.3 mIoU gain on Sonata purely from switching the backbone (28.3 → 29.6), it is unclear how much of the final benchmark advantage comes from Volt's intrinsic inductive bias versus the proposed distillation objectives.

  • Could the authors provide a controlled comparison to disentangle these factors, such as: (a) porting a baseline method (e.g., Sonata or Concerto) to the Volt backbone, or (b) training VINO with a PTv3 backbone?
  • If running a full pretraining suite is computationally prohibitive during the rebuttal, a controlled ablation on a single representative benchmark (e.g., ScanNet200 linear probing) under the same compute budget would be sufficient to clarify this attribution.

Q2. Diagnosis of Modest Outdoor Performance and Scaling Limits:

  • Since indoor and outdoor models are pretrained independently, what explains this noticeable performance disparity?
2 comments
AuthorsOpenReview ↗

Rebuttal Response to 9WZ1

We thank the reviewer for the detailed review and constructive feedback.

W1 / Q1: Comparison under the same backbone. The paper already contains two controlled comparisons on Volt-B. First, Tabs. 1 and 11 report Sonata trained with both PTv3 and Volt-B. Swapping the backbone alone improves Sonata by +1.3 mIoU on ScanNet200 and +1.1 on ScanNet++. On the same Volt-B backbone, our 3D-only objectives reach 35.9 and 42.4, which is +6.3 and +3.3 mIoU over Sonata (Volt). This shows that most of the improvement comes from our redesigned objectives rather than from the backbone. Second, Concerto combines Sonata’s 3D self-distillation with a 2D–3D distillation loss. Their 2D–3D distillation relies on two main design choices: (i) aligning with 2D features from the start of training, and (ii) cosine alignment to raw DINOv2 features. In our 2D–3D ablations (Tabs. 4 and 11), we use these two choices as the baseline, applied to Volt-B on top of our 3D objectives. We then revisit both: we distill in a second stage after 3D-only pretraining, and we use multi-view-consistent prototype assignments as targets instead of raw DINOv2 features. Each change improves performance, and together they outperform the baseline by +2.6 mIoU on ScanNet200 and +3.8 on ScanNet++. This shows the benefit of our approach over Concerto’s key design choices under fair conditions with the Volt-B backbone.

For a complete same-backbone comparison, we additionally trained Concerto on Volt-B with the same pretraining data and the same GPU-hour budget as VINO.

MethodBackboneScanNet200ScanNetScanNet++
ConcertoPTv3-B37.477.345.6
Concerto (reproduced)Volt-B37.177.043.6
VINO, with 2DVolt-B39.877.948.6

Under the same backbone, pretraining data, and GPU-hour budget, VINO outperforms our Concerto reproduction by +2.7 mIoU on ScanNet200, +0.9 on ScanNet, and +5.0 on ScanNet++. VINO also exceeds Concerto's published PTv3-B results on all three benchmarks (+2.4, +0.6, and +3.0 mIoU, respectively).

Our Concerto (Volt-B) reproduction closely matches the published Concerto (PTv3-B) results on ScanNet200 and ScanNet (-0.3 mIoU each), but performs 2.0 mIoU lower on ScanNet++. This gap may arise from differences in the backbone and pretraining data. First, Concerto includes synthetic Structured3D scenes, which are excluded from VINO's training data. Second, Concerto was designed around PTv3's multi-resolution hierarchy, applying its 3D self-distillation and 2D–3D alignment losses at different feature levels. Concerto reports that applying both losses at the same level reduces performance (Tab. 6f). Since Volt lacks this hierarchy, both losses operate on the same output representations in our adaptation. Thus, Volt may be less suitable for Concerto's original training formulation, although we cannot isolate the contributions of these differences.

W2 / Q2: Outdoor performance. Please see the general response for updated outdoor results. With the updated settings, VINO achieves the best 3D-only linear-probing results on all three outdoor benchmarks, including Waymo. Under fine-tuning, VINO is best on nuScenes and SemanticKITTI but remains 1.0 mIoU below DOS on Waymo. We do not have a conclusive explanation for this gap. We added new results to the paper.

Thank you for the detailed rebuttal. I will maintain my current positive rating.

Reviewer_DGMdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VINO introduces a framework for self-supervised 3D point-cloud representation learning that revisits several design choices in previous works. First, VINO replaces PTv3 with the Volume Transformer (Volt) to simplify dense feature supervision. Second, it uses lightly augmented global teacher views and strongly corrupted local student views, together with an independently learned prototype head that produces balanced targets through cross-view clustering. Third, it removes masked tokens before student encoding and predicts their prototype assignments using a cross-attention-only decoder. Finally, VINO introduces a second-stage 2D-to-3D distillation procedure in which multi-view correspondences are used to learn view-consistent prototype targets from a frozen vision foundation model. Experiments demonstrate improved linear probing and fine-tuning performance on indoor and outdoor segmentation benchmarks, as well as strong transfer to point-cloud registration and RGB-D segmentation.

Critical strengths
  1. Clear motivation and coherent design: The paper identifies concrete limitations of existing 3D self-distillation methods and provides well-motivated design choices to address them, particularly regarding target quality, geometric leakage, and view consistency.

  2. Systematic ablations and insightful analyses: The authors evaluate successive design changes, demonstrating improvements beyond the backbone replacement. The analyses of masked-query attention and voxel-occupancy-related representation artifacts further support their design decisions.

  3. Strong and comprehensive empirical results: VINO achieves strong performance across indoor and outdoor semantic segmentation benchmarks under both linear probing and fine-tuning. Additional evaluations on point-cloud registration and RGB-D semantic segmentation demonstrate the transferability of the learned representations.

Critical weaknesses

While the overall framework is effective, some contributions could be more carefully positioned relative to prior work, particularly in the Introduction.
First, replacing PTv3 with Volt appears to be a reasonable adoption of a recent backbone rather than a distinct methodological contribution. Second, removing masked tokens before encoding has already been explored in Point-MAE, while cross-attention-only masked prediction is closely related to CrossMAE and AsymDSD. Therefore, it is questionable whether these design choices can be regarded as VINO's original contributions.
I acknowledge the value of appropriately adapting and combining existing techniques, including those from 2D vision, for 3D representation learning. However, I think the current Introduction presents these techniques as more original to VINO than they actually are, even if that was not the authors' intention. I encourage the authors to reconsider the positioning of their contributions on page 2 and provide appropriate citations to the relevant prior work, as already done in the Method section.

Questions for authors
  1. Related to the backbone concern mentioned above, comparing VINO with recent methods such as DOS and Utonia under the same backbone would strengthen the evaluation by isolating the contribution of VINO's pretraining design.

  2. Table 4 and 11 shows that prototype-based distillation with multi-view swapped clustering outperforms raw embedding distillation. However, it would be better to include an ablation comparing the downstream performance of prototype-based distillation with and without multi-view consistency? This would help isolate the benefit of the proposed cross-view swapped clustering.

  3. Figure 5 provides qualitative evidence of view consistency, but how consistent are the learned 2D prototype assignments across different views quantitatively?

  4. Since Stage 2 changes the training data, augmentations, and training duration in addition to introducing 2D–3D distillation, is there a baseline trained under the same Stage 2 settings but without 2D supervision to isolate its contribution?

4 comments
AuthorsOpenReview ↗

Rebuttal Response 1 to DGMd

We thank the reviewer for their detailed and constructive feedback.

W1: Giving credit to related prior work in the introduction. We thank the reviewer and agree that the Introduction should credit prior work as done in later sections. We revised it to cite Point-MAE, CrossMAE, and AsymDSD there, and presented Volt as a building block of VINO rather than as a contribution. Our key contributions are in identifying key limitations of current 3D self-supervised learning approaches and revisiting their design choices for sparse 3D scenes.

W1 / Q1: Comparison under the same backbone. As requested, we trained DOS on Volt-B. Since DOS has no public code, we reimplemented it from the description in the paper and trained it with the same pretraining data, schedule, and linear-probing protocol as VINO.

MethodBackboneScanNet200ScanNetScanNet++nuScenesSem.KITTIWaymo
DOSPTv330.773.9–74.868.367.0
DOS (reproduced)Volt-B32.475.836.070.159.462.7
VINO, 3D onlyVolt-B35.976.142.476.770.768.6

On indoor datasets, our DOS reproduction on Volt-B improves over the published PTv3 results by +1.7 mIoU on ScanNet200 and +1.9 on ScanNet, suggesting that the backbone contributes to the performance gains. However, under the same backbone and training protocol, VINO still outperforms DOS by +3.5 mIoU on ScanNet200 and +6.4 on ScanNet++, demonstrating that the improvements cannot be attributed to the backbone alone.

On outdoor datasets, our DOS reproduction falls below the published PTv3 results by 4.7, 8.9, and 4.3 mIoU on nuScenes, SemanticKITTI, and Waymo, respectively. We cannot conclusively explain these gaps. Since DOS has no public implementation, differences in implementation details may contribute to the discrepancy.

Regarding Utonia, its self-supervised design builds directly on Concerto and mainly extends it to joint pretraining across heterogeneous point-cloud domains. Since VINO trains separate models for indoor and outdoor scenes, Concerto is more directly applicable for a same-backbone comparison. For a complete same-backbone comparison, we additionally trained Concerto on Volt-B with the same pretraining data and the same GPU-hour budget as VINO.

MethodBackboneScanNet200ScanNetScanNet++
ConcertoPTv3-B37.477.345.6
Concerto (reproduced)Volt-B37.177.043.6
VINO, with 2DVolt-B39.877.948.6

Under the same backbone, pretraining data, and GPU-hour budget, VINO outperforms our Concerto reproduction by +2.7 mIoU on ScanNet200, +0.9 on ScanNet, and +5.0 on ScanNet++. VINO also exceeds Concerto's published PTv3-B results on all three benchmarks (+2.4, +0.6, and +3.0 mIoU, respectively).

Our Concerto (Volt-B) reproduction closely matches the published Concerto (PTv3-B) results on ScanNet200 and ScanNet (-0.3 mIoU each), but performs 2.0 mIoU lower on ScanNet++. This gap may arise from differences in the backbone and pretraining data. First, Concerto includes synthetic Structured3D scenes, which are excluded from VINO's training data. Second, Concerto was designed around PTv3's multi-resolution hierarchy, applying its 3D self-distillation and 2D–3D alignment losses at different feature levels. Concerto reports that applying both losses at the same level reduces performance (Tab. 6f). Since Volt lacks this hierarchy, both losses operate on the same output representations in our adaptation. Thus, Volt may be less suitable for Concerto's original training formulation, although we cannot isolate the contributions of these differences.

AuthorsOpenReview ↗

Rebuttal Response 2 to DGMd

Q2: Separating the 2D-3D loss from cross-view consistency. We thank the reviewer for this suggestion and have run the requested ablation. We keep the prototype-based 2D–3D distillation loss and all other settings fixed (same image data, feature queue, temperature, and number of prototypes) and change only how the 2D prototypes are learned. Instead of swapped clustering between corresponding tokens across views, we use each token’s Sinkhorn–Knopp assignment as the target for its own predicted distribution (self-clustering). This removes all cross-view supervision from the target space.

MethodScanNet200ScanNet++
Cosine38.646.1
Prototypes (self-clustering)39.548.3
Prototypes (swapped clustering, VINO)39.848.6

Prototype-based distillation improves over direct cosine alignment by +0.9 mIoU on ScanNet200 and +2.2 mIoU on ScanNet++, even without cross-view consistency. Adding multi-view swapped clustering provides a further +0.3 mIoU on both datasets. These results show that the main improvement comes from distilling prototype assignments rather than raw DINOv2 embeddings, while explicitly enforcing cross-view consistency provides an additional benefit. This changes how our contributions should be presented. We added this row to Tabs. 4 and 11 and revised the introduction and Sec. 3.4. It now attributes the improvement primarily to prototype-based distillation targets, with explicit cross-view consistency as an additional refinement, rather than presenting view consistency as the main source of the gain.

Q3: Quantitative view consistency of the 2D targets. To quantify the view consistency of the 2D targets, we take all pairs of overlapping images from the 50 ScanNet++ validation scenes and encode them with the frozen DINOv2-g/14. For each image patch whose centre is visible in the other image (from the rendered depth), we rank all patches of the other image by similarity and report the mean percentile rank of the true match. 0 means it is always ranked first and 50 is chance. A low rank is desired as it requires targets that are both consistent across views and distinct from other patches. For each target type, we use the similarity corresponding to the loss used to train against it: cosine similarity for raw embeddings and negative symmetric cross-entropy between softmax assignments for prototypes. Since the percentile rank depends only on the ordering of candidates, the resulting numbers are directly comparable.

Target spaceMean rank of GT match ↓
Raw DINOv2 features20.33
VINO prototype assignments (ours)14.52

The prototype assignments rank the true match 5.8 percentile points lower than the raw features (14.5 vs. 20.3), so they are measurably more consistent. Neither target space is close to 0, which is expected, since indoor scenes contain many near-identical patches (walls, floors) that cannot be told apart from appearance alone. We added this analysis to the paper.

Q4: Second stage without 2D supervision. To separate the effect of the 2D–3D distillation loss from the Stage-2 training configuration, we rerun Stage 2 starting from the same Stage-1 checkpoint with identical data, training steps, augmentations, and optimization settings, removing only the 2D-3D distillation.

SettingScanNet200ScanNet++
Stage-1 checkpoint35.942.4
Stage 2 with 3D losses only35.542.0
Stage 2 with 2D–3D distillation (VINO)39.848.6

Without the 2D-3D loss, performance drops slightly by 0.4 mIoU on both datasets. This is likely due to the stronger Stage-2 augmentations (color dropout and unrestricted rotations), which enforce invariances that these colored, gravity-aligned benchmarks may not reward. We keep them so that the model remains usable on in-the-wild point clouds without color or gravity alignment. The improvement in Stage 2 therefore comes entirely from 2D-3D distillation, not from additional training or data.

Thank you for the comprehensive rebuttal, which has addressed most of my concerns. I do, however, have just a few minor follow-up questions.

Q1. Given the different indoor and outdoor trends observed when adapting DOS to Volt-B, do VINO's key design choices show similar benefits on outdoor LiDAR datasets as demonstrated in the indoor ablations (Tab. 2-4, 11)?

Q4. Stage 2 training without 2D–3D distillation performs worse than the Stage 1 baseline. Could the authors clarify whether this degradation is mainly due to changes in the data distribution or augmentations? For example, training on the Stage 2 dataset with Stage 1 augmentation and crop settings, without 2D–3D distillation, could help isolate their effects.

We are glad the rebuttal addressed most of your concerns.

Q1: Do VINO's design choices transfer to outdoor LiDAR? Yes. We repeated the cumulative ablation on nuScenes (linear probing, mIoU):

SettingnuScenesΔ
Stage 1 (3D only)
Local prediction baseline60.9
+ Asymmetric augmentations69.5+8.6
+ Decoupled teacher head71.7+2.2
+ Masked prediction (with tokenizer artifacts)76.4+4.7
+ Artifact mitigation (Stage-1 checkpoint)76.7+0.3
Stage 2 (2D–3D distillation)
Distillation from scratch79.0
Second stage, raw embeddings79.5+0.5
Second stage, prototypes (self-clustering)80.2+0.7
Second stage, prototypes (swapped clustering, VINO)80.6+0.4

Every design choice improves outdoor performance, consistent with the indoor ablations (Tabs. 2–4, 11). This shows that VINO's design choices generalize beyond indoor scenes and are not domain-specific.

Q4: Data distribution vs. augmentations. Starting from the Stage-1 checkpoint, we continue training with the 3D losses only and introduce the Stage-2 changes one at a time:

SettingDataAugsScanNet200ScanNet++
Stage-1 checkpoint––35.942.4
Stage 2, 3D losses onlyStage 1Stage 136.042.5
Stage 2, 3D losses onlyStage 1Stage 236.242.4
Stage 2, 3D losses onlyStage 2Stage 235.542.0
Stage 2 with 2D–3D distillation (VINO)Stage 2Stage 239.848.6

Continuing training with the Stage-1 data and augmentations has minimal impact (+0.1 mIoU on both datasets). Adding the stronger Stage-2 augmentations gives mixed results (+0.2 on ScanNet200, −0.1 on ScanNet++). Adding the reconstructed RealEstate10K scenes lowers performance (−0.7 / −0.4 mIoU). The small degradation therefore comes mainly from the data. Possible reasons are that the benchmark datasets are sampled less often in the larger training set, and that reconstructed geometry differs from the sensor scans in these benchmarks. We nevertheless keep this data so that VINO also performs well on reconstructed geometry, which is increasingly common in practice as point clouds are often obtained from images and videos rather than from depth sensors. Without 2D–3D distillation, all variants stay within 0.4 mIoU of the Stage-1 checkpoint, so the Stage-2 gains (+3.9 / +6.2 mIoU) come from the distillation itself.

Reviewer_gdTdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VINO revisits self-supervised representation learning for 3D scenes using Volume Transformer. It introduces asymmetric teacher-student augmentations, independently learned teacher prototypes, and a masked prediction mechanism that removes masked tokens from the encoder and excludes interactions between decoder queries. Later on, a 2D-3D distillation stage supervises the 3D encoder with prototype assignments learned through cross-view consistency. Evaluation shows good performance, covering semantic segmentation through linear probing and fine-tuning, frozen-feature point-cloud registration, and cross-modal semantic probing.

Critical strengths
  1. The paper is focusing on an important problem and the method is well motivated.
  2. The method addresses concrete sources of information leakage and inconsistent supervision in 3D learning.
  3. The method achieves strong results across multiple evaluation settings, and the paper also provides a systematic examination of various design choices.
Critical weaknesses
  1. The ablations establish useful configurations more clearly than they establish the proposed explanations. For example, the decoupled teacher head changes both the head formulation and its learning objective, so the improvement does not isolate the claim that EMA coupling is problematic specifically because teacher and student see different input distributions. Similarly, the 2D distillation comparison changes the target representation, loss, and cross-view training simultaneously. The improvement from “linear + cosine” to “prototype + swapped” therefore does not establish how much comes from removing view-dependent information versus changing the supervision space or objective. Because much of the contribution consists of explaining why existing designs are suboptimal, more direct evidence for these explanations would strengthen the paper substantially.

  2. The contribution of the second training stage is not fully separated from additional data and training changes. Indoors, this stage expands pretraining from approximately 18k to 90k scenes, adds 30k optimization steps, and introduces additional robustness augmentations alongside 2D supervision. Thus, the improvement over the first-stage checkpoint cannot be attributed entirely to 2D distillation. Tables 4 and 11 provide useful comparisons among distillation variants, but the paper does not clearly establish a continuation baseline with the same data, steps, and augmentations and no 2D loss. This limits interpretation of the headline gains attributed to adding 2D supervision.

Questions for authors
  1. Are the second-stage comparisons in Tables 4 and 11 matched in training data, optimization steps, and augmentations? Is there a matched second-stage continuation using only the 3D objectives, and how does it perform?
  2. Does decoupling also help under symmetric augmentations, or is its benefit specifically associated with asymmetric views?
  3. Is there quantitative evidence that the learned 2D prototype targets are more consistent across corresponding views?
3 comments
AuthorsOpenReview ↗

Rebuttal Response 1 to gdTd

We thank the reviewer for their detailed and constructive feedback.

W1 / Q2: Symmetric augmentations. We thank the reviewer for this suggestion and ran the missing experiment, training the decoupled teacher head with symmetric augmentations:

AugmentationsEMA-coupled headDecoupled head
Symmetric28.931.6
Asymmetric31.332.8

Decoupling improves performance under both augmentation schemes. Its benefit is therefore more general than we stated: it does not depend on the asymmetric augmentations. We updated the paper to present the decoupled head as an independent design choice instead of as a consequence of the asymmetry.

Similarly, asymmetric augmentations improve performance with either head, and combining both gives the best result. We view the two changes as complementary ways of separating teacher and student. Asymmetric augmentation separates their inputs: the teacher provides targets from clean views, while the student learns invariance to strong perturbations. Decoupling separates their heads: the targets are constructed from teacher representations alone and no longer track the student's prediction head. The new experiment strengthens the case for each contribution individually. We revised the reasoning in the Introduction and Sec. 3.2 accordingly.

W1: Head formulation vs. learning objective. We agree that decoupling changes two aspects of the teacher head: (i) its architecture, replacing the bottleneck MLP + prototypes with a linear prototype layer applied directly to the encoder features, and (ii) its learning objective, replacing EMA coupling with optimization via the swapped clustering loss.

To disentangle these changes, we train a variant that retains the MLP + prototypes architecture but optimizes the teacher head with the swapped clustering loss. We chose this comparison because an EMA-coupled linear head would also require changing the student head to match its architecture, introducing another confounding factor. All other settings follow Tab. 2 (asymmetric augmentations, masked branch removed).

Teacher headUpdate RuleScanNet200
MLP + prototypesEMA of student head31.3
MLP + prototypesSwapped clustering loss30.5
Linear prototypes (VINO)Swapped clustering loss32.8

Changing only the update rule reduces performance from 31.3 to 30.5 mIoU, whereas replacing the MLP with linear prototypes under the same clustering objective improves it to 32.8 mIoU. We hypothesize that this difference stems from the flexibility of the head under the clustering objective. With linear prototypes and cosine similarity, the head can only adjust prototype directions in the teacher's feature space, so the targets remain a clustering of the encoder representations themselves. In contrast, an MLP can learn a nonlinear transformation that makes cross-view assignments consistent while potentially discarding information present in the encoder features, leading to less informative targets.

W1: Separating the 2D-3D loss from cross-view consistency. We thank the reviewer for this suggestion and have run the requested ablation. We keep the prototype-based 2D–3D distillation loss and all other settings fixed (same image data, feature queue, temperature, and number of prototypes) and change only how the 2D prototypes are learned. Instead of swapped clustering between corresponding tokens across views, we use each token’s Sinkhorn–Knopp assignment as the target for its own predicted distribution (self-clustering). This removes all cross-view supervision from the target space.

MethodScanNet200ScanNet++
Cosine38.646.1
Prototypes (self-clustering)39.548.3
Prototypes (swapped clustering, VINO)39.848.6

Prototype-based distillation improves over direct cosine alignment by +0.9 mIoU on ScanNet200 and +2.2 mIoU on ScanNet++, even without cross-view consistency. Adding multi-view swapped clustering provides a further +0.3 mIoU on both datasets. These results show that the main improvement comes from distilling prototype assignments rather than raw DINOv2 embeddings, while explicitly enforcing cross-view consistency provides an additional benefit. This changes how our contributions should be presented. We added this row to Tabs. 4 and 11 and revised the introduction and Sec. 3.4. It now attributes the improvement primarily to prototype-based distillation targets, with explicit cross-view consistency as an additional refinement, rather than presenting view consistency as the main source of the gain.

AuthorsOpenReview ↗

Rebuttal Response 2 to gdTd

W1 / Q3: Quantitative view consistency of the 2D targets. To quantify the view consistency of the 2D targets, we take all pairs of overlapping images from the 50 ScanNet++ validation scenes and encode them with the frozen DINOv2-g/14. For each image patch whose centre is visible in the other image (from the rendered depth), we rank all patches of the other image by similarity and report the mean percentile rank of the true match. 0 means it is always ranked first and 50 is chance. A low rank is desired as it requires targets that are both consistent across views and distinct from other patches. For each target type, we use the similarity corresponding to the loss used to train against it: cosine similarity for raw embeddings and negative symmetric cross-entropy between softmax assignments for prototypes. Since the percentile rank depends only on the ordering of candidates, the resulting numbers are directly comparable.

Target spaceMean rank of GT match ↓
Raw DINOv2 features20.33
VINO prototype assignments (ours)14.52

The prototype assignments rank the true match 5.8 percentile points lower than the raw features (14.5 vs. 20.3), so they are measurably more consistent. Neither target space is close to 0, which is expected, since indoor scenes contain many near-identical patches (walls, floors) that cannot be told apart from appearance alone. We added this analysis to the paper.

W2 / Q1: Comparability in Tables 4 and 11. In both Tables, the second-stage distillation variants use the same training data, optimization steps, and augmentations, and differ only in the distillation targets (raw DINOv2 features vs. prototype assignments). Thus, the results are directly comparable.

W2 / Q1: Second stage without 2D supervision. To separate the effect of the 2D–3D distillation loss from the Stage-2 training configuration, we rerun Stage 2 starting from the same Stage-1 checkpoint with identical data, training steps, augmentations, and optimization settings, removing only the 2D-3D distillation.

SettingScanNet200ScanNet++
Stage-1 checkpoint35.942.4
Stage 2 with 3D losses only35.542.0
Stage 2 with 2D–3D distillation (VINO)39.848.6

Without the 2D-3D loss, performance drops slightly by 0.4 mIoU on both datasets. This is likely due to the stronger Stage-2 augmentations (color dropout and unrestricted rotations), which enforce invariances that these colored, gravity-aligned benchmarks may not reward. We keep them so that the model remains usable on in-the-wild point clouds without color or gravity alignment. The improvement in Stage 2 therefore comes entirely from 2D-3D distillation, not from additional training or data.

Area_Chair_MrRHOpenReview ↗

Please help to join the discussion

Dear Reviewer gdTd,

The discussion deadline is approaching. Please help to join the discussion and offer the reviewer's opportunity to further respond to your concerns.

Thanks!

AC

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all three reviewers for their detailed and constructive feedback. We appreciate their positive assessment of the paper's motivation, systematic ablations, and experimental results. In response to their comments, we have conducted several additional experiments to provide further insight. We updated the paper with the information provided in the rebuttal, and marked those changes in red.

New Outdoor Results. We thank reviewer 9WZ1 for highlighting this. After submission, we found that the outdoor training hyperparameters were undertuned. We made three changes to the Stage-1 setup, leaving the method itself unchanged: (i) we used a linear instead of a cosine warmup, (ii) adjusted sampling ratios so that training sees more SemanticKITTI and Waymo scenes, and (iii) tuned existing outdoor augmentations (point dropout rate, centering, and jitter).

SettingModelnuScenesSem.KITTIWaymo
Linear probing, 3D onlyBest prior (DOS)74.868.367.0
VINO (submission)76.268.565.3
VINO (updated)76.770.768.6
Linear probing, with 2DBest prior (Utonia)75.567.763.8
VINO (submission)79.269.868.5
VINO (updated)80.671.170.6
Fine-tuningBest prior82.2 (Utonia)73.5 (DOS)73.9 (DOS)
VINO (submission)82.574.172.4
VINO (updated)83.275.272.9

With 3D-only pretraining, VINO now achieves the best linear-probing performance on all benchmarks, including Waymo (+1.6 mIoU over DOS, previously -1.7). With 2D–3D distillation, the margin over Utonia grows to +5.1 on nuScenes and +6.8 on Waymo. Under fine-tuning, VINO is best on nuScenes and SemanticKITTI and remains 1.0 mIoU below DOS on Waymo.

Area_Chair_MrRHOpenReview ↗

Reviewers, please help to join the discussion.

Dear reviewers,

The authors have uploaded the rebuttal. Please help to join the discussion and the further process.

Thanks!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.