acceptodds
Under review as a conference paper at ICLR 2027

D-ViCe: A Dual Variance-Enhanced Contrastive Framework for Graph Representation Learning

Abstract

Contrastive learning (CL) has become one of the most powerful paradigms in visual self-supervised learning (SSL), but its extension to non-visual modalities such as graphs and CAD data remains underexplored. Although SSL in graphs is closely related to vision, the transfer is non-trivial in practice because graph and CAD representations lack the regular structure and mature augmentation priors of images. For SSL on graph-based CAD representations, we argue that the design of the contrastive targets and learning dynamics can reduce the reliance on carefully tuned, task-specific augmentations. In this work, we propose a novel Dual Variance-Enhanced Contrastive Learning (D-ViCe) framework that employs a dual contrastive mechanism between two target encoders and a shared source encoder. The first target encoder, kept frozen at its random initialization, provides a stationary target for global instance discrimination, while the second, variance-adaptive momentum encoder promotes the refinement of locally invariant representations. Temperature and momentum are central to many CL objectives, yet they are typically fixed or follow predetermined schedules. In contrast, D-ViCe uses training statistics as explicit control signals: the positive-pair probability modulates the contrastive temperature in one branch, and the relative variance of source-teacher similarities governs the target momentum update in the other, thereby balancing global discrimination and local consistency. Experiments on CAD retrieval and graph classification benchmarks show that D-ViCe consistently outperforms state-of-the-art SSL baselines, improving Recall@1 on SolidLetters from to , and yields systematic gains when added to existing graph contrastive methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Aided Design, Graph Representation Learning, Self-supervised Learning, Contrastive Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3qcjOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces D-ViCe, a method for CAD model retrieval and graph representation learning. It employs two target networks to promote consistency between representations of different views of the same sample. Experiments demonstrate substantial performance improvements on the SolidLetters retrieval task, as well as gains when the method is integrated into existing graph contrastive learning methods.

Critical strengths
  1. The method achieves substantial performance improvements on the main retrieval tasks.
  2. Combining a fixed target network with a target network that is updated during training yields additional performance gains, which is an interesting finding.
Critical weaknesses

None.

Questions for authors
  1. Regarding adaptive temperature adjustment during training, the temperature reaches its lower bound within the first 45 training steps and remains essentially constant thereafter. Could the temperature simply be fixed at this lower bound from the outset?
  2. Tables 3–5 in the main text report R@5, whereas Tables 8–10 in the appendix report R@10. However, the value 0.9496 repeatedly appears as the default result in both. Is this merely a numerical coincidence?
1 comment

We thank the reviewer for the positive assessment and for the two precise questions, which helped us clarify the paper. We answer them below.

Note on the metric. While preparing the rebuttal, we found that the metric reported as R@5 counts a hit within the ten nearest neighbors. We have relabeled it as R@10 throughout; this changes only the label (see Q2).


Q1. Could the temperature simply be fixed at its lower bound from the outset?

We tested this directly. Training the full model with the controller disabled and τ\tau fixed at τmin⁡=0.05\tau_{\min}=0.05 from the first step reaches R@10 = 0.9269±0.0010.9269 \pm 0.001 over three seeds, compared with 0.9497±0.00050.9497 \pm 0.0005 for D-ViCe, a drop of 2.28 points. Although the temperature is constant after about 45 steps, this early phase matters: starting directly at a sharp temperature from random initialization hurts training, while the controller reaches it gradually. The controller's path is practically a short schedule from 0.1 to 0.05, which it obtains without tuning the schedule length. We add this comparison to Table 3 and state this interpretation in Section 2.2.

Q2. Is the repeated value 0.9496 a numerical coincidence?

No, and we thank the reviewer for spotting this. All these tables report the same metric for the same default configuration. While checking the evaluation code, we found that the metric labeled R@5 in Tables 3–5 counts a hit within the ten nearest neighbors, i.e. it is R@10, as correctly labeled in Tables 8–10. We have relabeled the metric as R@10 throughout the paper; all values are unchanged. The default configuration is now reported as 0.9497±0.00050.9497 \pm 0.0005, the mean over three seeds.


We are grateful for the reviewer's positive evaluation and careful reading, which helped us correct the metric label and strengthen the analysis of the temperature controller. If any further clarification or experiment would help, we would be glad to provide it. We hope these answers reinforce the reviewer's assessment, and we would sincerely appreciate it if the reviewer would consider raising the score.

Reviewer_7qU2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes D-ViCe, combining a trainable source encoder, a frozen random teacher, and a VA-EMA teacher to merge global NT-Xent contrastive loss with local alignment loss. It regulates temperature via running positive-pair probability and modulates teacher momentum via relative similarity variance. The primary empirical contributions are performance gains in CAD retrieval on SolidLetters and systematic accuracy improvements on TU graph classification benchmarks.

Critical strengths
  1. Investigating self-supervised targets suited for the unique geometry and topology of B-rep CAD representations is of practical significance.

  2. Shows impressive improvement in Top-1 Recall and reveals an interesting empirical phenomenon where a frozen teacher outperforms updated ones.

  3. Candidly discloses mechanism limitations, including temperature saturation, modest individual gain of VA-EMA, and single-run ablation setups.

Critical weaknesses
  1. The temperature controller rapidly saturates to τmin=0.05\tau_{min}=0.05 within ~45 steps due to unattainable target confidence p∗p^*, reducing the global branch to fixed low-temperature contrastive learning. Gains from VA-EMA fall within run-to-run variance, failing to demonstrate essential necessity for either controller.

  2. Quantitative CAD retrieval relies heavily on SolidLetters without direct comparison to specialized B-rep baselines like CADGCL. On TU datasets, average gain is marginal using published baseline values, lacking paired significance tests under unified implementations.

  3. Theoretical stability analysis assumes fixed representations and monotonic response, which holds for only 5.7% of anchors in practice and does not explain representation quality or joint training convergence.

Questions for authors
  1. After properly tuning a fixed low temperature, what are the individual marginal gains of the frozen teacher and the two controllers across multiple random seeds?

  2. Does the 0.42% margin on TU hold under identical splits, re-run baseline implementations, and paired statistical significance tests?

  3. Can the framework’s CAD performance be validated on an additional CAD dataset or through direct empirical comparison against CADGCL?

Reviewer_5MmQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

D-ViCe studies self-supervised representation learning for B-rep CAD graphs, focusing on contrastive targets and training dynamics. A UV-Net encoder learns through contrast against a frozen random global teacher and alignment to an EMA local teacher, with feedback-controlled temperature and variance-modulated momentum. On SolidLetters, the method reports retrieval R@1 of 0.798 versus at most 0.521 for the compared SSL baselines, alongside improvements when added to seven graph-learning methods on TU benchmarks.

Critical strengths
  1. Under a shared encoder and schedule, the retrieval gain is large.
  2. Only the source encoder is deployed, so inference cost is unchanged, and the algorithms are fully specified.
Critical weaknesses
  1. Fixed temperature nearly matches the default, while the adaptive-EMA gain lies within the reported run spread. Matched-seed comparisons with fixed temperature, a short schedule, and ordinary EMA are needed; current results establish neither superiority nor equivalence.
  2. The main ablation changes teacher structure and temperature together. Matched-temperature and random-feature controls should isolate the frozen-target benefit.
  3. TU gains compare new runs with published values; CAD baseline modifications are disclosed, but DINO's temperature rule and tuning are unspecified. Matched reruns should also resolve differing results from equivalent search backends.
  4. Quantitative CAD evidence is limited to SolidLetters; core ablations and sensitivity checks use single runs. Repeated results and a clear checkpoint/hyperparameter-selection protocol are needed.
  5. CADGCL [1] is cited but absent from experiments despite addressing unsupervised B-rep retrieval. Compare it and a MACL-style [2] adaptive-temperature control under matched conditions.

[1] Qin et al. CADGCL: Unsupervised Retrieval of CAD Models via Boundary Representations. The Visual Computer 41:6493–6505, 2025.

[2] Huang et al. Model-Aware Contrastive Learning: Towards Escaping the Dilemmas. ICML 2023.

Questions for authors

See weaknesses.

Comments on the paper

1 comment

Dear Reviewers,

Thank you once again for your constructive reviews.

With less than 24 hours remaining in the discussion period, we were wondering whether our responses have adequately addressed your concerns or whether any points require further clarification. We would be happy to provide additional explanations if needed.

We appreciate your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.