- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces FBDM, a non-adversarial self-supervised learning framework that aligns learned representations with a geometric reference through spherical conditional velocity regression. A structured reference design and an alignment loss are further proposed to enhance representation learning. Experimental results demonstrate that FBDM attains performance comparable to existing methods while substantially reducing training time, and offers theoretical guarantees for downstream classification.
Critical strengths
(1) This work presents a non-adversarial, flow-based framework for distribution matching that circumvents the expensive optimization associated with adversarial training.
(2) FBDM attains performance on par with competitors while delivering a 1.48–1.83× acceleration in training relative to DM.
(3) Theoretical assurances are established in the paper, linking the pretraining objective with performance on downstream classification tasks.
Critical weaknesses
(1) The ImageNet evaluation is confined to 100 epochs, and only a modest set of competing methods is included for comparison.
(2) The theoretical assurances rest on rather stringent assumptions, notably that of transport identifiability.
(3) The empirical study centers primarily on image classification, so how well the approach transfers to other downstream tasks remains uncertain.
Questions for authors
-
Would it be possible for the authors to include broader ImageNet comparisons against stronger SSL baselines?
-
To what extent does the transport identifiability assumption hold under realistic conditions?
-
How effectively does FBDM transfer to downstream tasks beyond classification, e.g., object detection or segmentation?
4 comments
Official Comment by Authors 2
Weakness 3: The empirical study centers primarily on image classification, so how well the approach transfers to other downstream tasks remains uncertain. Question 3: How effectively does FBDM transfer to downstream tasks beyond classification, e.g., object detection or segmentation?
Response. Thank you for suggesting detection and segmentation as transfer tests. With a frozen ResNet-50 pretrained on ImageNet for 100 epochs with , FBDM reaches 63.37% VOC 2007 [email protected] and 95.54% Oxford-IIIT Pet mIoU (Table 16).
Object detection. We use Faster R-CNN with FPN on the complete VOC 2007 trainval/test splits. The backbone is frozen, and the detection components are trained for 15 epochs. We report VOC 2007 11-point mAP at IoU 0.5 from one run per configuration.
| Backbone initialization | Detection [email protected] (%) |
|---|---|
| Random initialization | 7.23 |
| FBDM | 63.37 |
Foreground segmentation. We freeze the backbone and train a linear segmentation head on inputs. We report binary mIoU and foreground IoU, excluding the unclassified trimap region, from one run per configuration.
| Initialization | mIoU (%) | Foreground IoU (%) |
|---|---|---|
| Random initialization | 60.22 | 44.81 |
| FBDM | 95.54 | 94.20 |
The revised manuscript reports the transfer protocols and results in Appendix E.6 (Table 16).
If you have any further questions, please feel free to let us know. We appreciate your recognition of our work and value your constructive advice.
Thank you for your detailed response and the considerable effort you have put into providing additional clarifications and addressing my some concerns. I sincerely appreciate the authors' dedication throughout the rebuttal process.
However, I still have substantial concerns regarding the paper's contributions, presentation quality, and experimental evidence. In my view, the current manuscript does not yet meet the acceptance standards of ICLR. Therefore, I have decided to maintain my original score.
I have also carefully reviewed the updated version of the manuscript and would like to offer a few constructive suggestions:
-
Clarity and Presentation: I strongly encourage the authors to improve the clarity and organization of the manuscript. More informative tables and figures would help readers understand the key ideas and findings more effectively, rather than relying heavily on lengthy mathematical formulations.
-
Contributions and Experimental Evidence: The authors should more clearly articulate and highlight the main contributions and advantages of the proposed method. For instance, based on the performance comparisons in Table 1, I still find it difficult to identify the clear advantages of FBDM over existing approaches.
-
Computational Efficiency: The reported speedup appears relatively modest, and the efficiency results also seem inconsistent with those presented in the original submission. I encourage the authors to clarify these discrepancies and provide more convincing evidence of the claimed efficiency improvements.
-
Overall Research Quality: A strong research paper requires a combination of clear writing, rigorous experiments, meaningful methodological contributions, and careful presentation. These aspects are equally important and should be developed together. I sincerely encourage the authors to study strong papers from previous ICLR conferences and continue refining their research, step by step, across these dimensions.
I hope these comments will be helpful in further improving the work. I appreciate the authors' efforts and wish them all the best in their future research.
Responds for Reviewer ZSna
Thank you for reading the revision. In response to your original questions, we added 100-epoch ImageNet comparisons with VICReg and BYOL (Table 2), evaluated transfer to detection and segmentation (Table 16), and clarified the geometric condition for transport identifiability (Appendix C.4).
Table 1 shows that FBDM achieves competitive linear-evaluation accuracy. Its practical advantage is that it lets us train representations against a chosen reference distribution without an adversarial critic, at a lower per-epoch cost than critic-based DM.
The original implementation repeatedly computed the same fixed reference centers across training iterations. We now compute them once and reuse them. We also matched the parameter counts of DM’s critic and FBDM’s velocity network and remeasured both methods. The measured per-epoch speedup over DM increased from 1.48–1.83× to 2.17–2.36×. The full timings are in revised Table 3 and in our response to Reviewer rZf5, Question 2.
And thank you for your suggestions. We would appreciate it if you could keep your feedback focused on the specific claims, experiments, and presentation of this manuscript. We also encourage reviewers to respond promptly to rebuttals and explain which concerns still weigh against acceptance in light of the new evidence. This gives authors time to address those concerns before the discussion closes.
Official Comment by Authors
We sincerely thank you for the time and technical care you brought to this review and greatly appreciate your recognition of the framework, its efficiency, and its connection to downstream classification. Your suggestions to broaden the ImageNet comparison and test transfer beyond classification, and your question about when transport identifiability holds, helped us strengthen the evaluation and clarify the scope of the theory.
Summary of our response.
Weakness 1: The ImageNet evaluation is confined to 100 epochs, and only a modest set of competing methods is included for comparison. Question 1: Would it be possible for the authors to include broader ImageNet comparisons against stronger SSL baselines?
Response. Thank you for suggesting stronger ImageNet baselines. At 100 pretraining epochs and global batch size 512, FBDM reaches 61.55% linear top-1, compared with 61.29% for VICReg and 59.94% for BYOL, which we reproduced using the same frozen-backbone evaluation (Table 2). The SimCLR entry uses the same epoch budget and batch size and remains an external reference.
The revised manuscript reports these results in Section 4.1 (Table 2), with the evaluation settings and baseline protocol differences in Appendices D.2 and E.7.
Weakness 2: The theoretical assurances rest on rather stringent assumptions, notably that of transport identifiability. Question 2: To what extent does the transport identifiability assumption hold under realistic conditions?
Response. Thank you for asking when this assumption is plausible in practice. One situation in which the condition can hold is when a positive fraction of the mismatched mass still lies near the center matched to its own class. If the reference components are well separated, pairing that mass with a wrong component incurs a nontrivial distance cost, even if some other mismatches are cheap.
For one component per class, let Γ⋆ be a Wasserstein-1 optimal coupling between the learned terminal distribution p1 and the reference distribution νε. Write qkj⋆ for the mass of class k paired with component j, and qkj⋆,<Δsep for the part paired at a distance below Δsep. Let τ⋆ be the one-to-one class–component matching that retains the most mass. The sufficient condition in Proposition 9 is
\le \alpha_{\mathrm{sep}} \sum_k\sum_{j\ne\tau^\star(k)}q^\star_{kj}, \qquad 0\le\alpha_{\mathrm{sep}}<1,\quad \Delta_{\mathrm{sep}}>0.$$ The right-hand sum is the total mismatched mass; the left counts only the low-cost part. Thus at least a fraction $1-\alpha_{\mathrm{sep}}$ of the mismatched mass incurs a distance of at least $\Delta_{\mathrm{sep}}$. This cost is included in $W_1$, giving $$\operatorname{Leak}(\Gamma^\star) \le \frac{2W_1(\widehat p_1,\nu_\varepsilon)} {(1-\alpha_{\mathrm{sep}})\Delta_{\mathrm{sep}}},$$ where leakage is the largest sum of outgoing and incoming mismatched mass for any matched class–component pair (Appendix C.4). The class labels are used only in the analysis. This is a data-dependent condition on the class-wise distribution of the learned terminal representations. For example, if two equally weighted classes are mixed equally within each of two reference components, the terminal distribution can match the reference exactly even though half of the mass is mismatched.