- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In this paper, the authors propose BRIDGE, a label-free iterative framework that augments self-supervised learning (SSL) with synthetic data. In each cycle, mean squared -NN distances are computed using normalized embeddings from the current encoder. These distances are used to select anchors from relatively sparse regions of the embedding space. Then, a frozen Stable Diffusion (SD) 3 model generates variants of these anchors, which are added to the training set for continued SSL pre-training.
Critical strengths
[S1] Compatibility with Different SSL Objectives
BRIDGE augments the training data while leaving the underlying SSL loss unchanged. The paper applies the same framework across different SSL objectives and reports improvements in transfer performance for several of them.
Critical weaknesses
[W1] Limited Theoretical Justification and Unclear Novelty
As I understand it, the propositions in Section 3 are largely disconnected from the proposed method. Their assumptions are not established for BRIDGE, key quantities in the theoretical model are neither instantiated nor estimated, and the implemented selection rule is not derived from the analysis.
Proposition 1: It holds only in a fixed embedding space. Furthermore, the proposition uses , while BRIDGE uses the mean squared -NN distance. Although fixed-space monotonicity also holds for this score, this property provides no guarantee after encoder updates.
Proposition 2: As stated in the paper (lines 161–163), the regions and parameters in Eq. (1) are neither instantiated nor estimated when using BRIDGE. Moreover, the paper does not establish that the image-level -NN score reflects the region-level effective support defined in Eq. (1).
Proposition 3: The conclusion follows directly from the assumptions. Since has the sign of , the conditions " strictly decreasing" and "" already impose unimodality on with an interior maximum. Neither nor is measured, and the lack of fidelity measurements is explicitly acknowledged in line 472. The optimum , obtained for in Proposition 6, is never estimated. Moreover, the proposition yields a single point , and it gives no reason why the fixed q75–q99 band, or a candidate pool centered at the histogram mode of the score distribution, should identify it. The upper cutoff q99 is never ablated and appears non-binding for the reported candidate pools, since the mode always lies at the lower edge of the band and the candidates are effectively the 4$B$ samples just above q75.
Proposition 4: As stated in the paper (lines 192–193), the guarantee does not apply to the changing representation in BRIDGE, since the image encoder is updated in each cycle. Even within its fixed-objective setting, the proposition does not establish an advantage over an optimal allocation computed in advance. If is fixed and known, the optimal fixed allocation can be computed once and attains the same objective value as the greedy allocation. The counterexample in Appendix C compares greedy against a naive allocation, yielding 5, rather than against the optimal fixed allocation, , which matches the greedy gain of 7.
Unlike the greedy marginal-gain allocation analyzed in Proposition 4, BRIDGE selects anchors using -NN scoring, a percentile band, a mode-based candidate pool, and FPS. The combined selection rule is not shown to select the region with the largest marginal gain, and 2,500 images are added per cycle without re-estimation between individual repairs. -NN neighborhoods also overlap, so repairing one location can change the support of nearby points. Therefore, the assumption that is separable across regions is not established for this process.
Proposition 5: This proposition bounds the difference in between any two equal-budget allocations by . In particular, within the same fixed-representation setting, any absolute advantage of greedy over another equal-budget allocation is bounded by the same quantity. Therefore, this advantage vanishes in the stated regime. The subsequent claim that the effect accumulates over multiple cycles is not established by this bound (lines 203–205). Proposition 4 does not establish this accumulation either.
The related-work discussion also leaves the novelty insufficiently established. MAK [1] already combines tailness, outlier avoidance and diversity (-center) for label-free acquisition in imbalanced SSL. These acquisition principles overlap substantially with the sparsity, outlier avoidance, and diversity criteria used in BRIDGE. D$^2$ Pruning [2] jointly considers difficulty and diversity, while Vo et al. [3] address balanced SSL data curation through hierarchical clustering and sampling. LoRA-based synthesis methods, such as DataDream [4], LoFT [5] and ChimeraLoRA [6], address faithful generation from scarce real examples.
These methods need not be directly equivalent to BRIDGE, but the paper should clarify what BRIDGE contributes beyond existing acquisition and synthesis techniques. Tables 5 and 6 include some alternative selection rules (e.g., TADA with the SSL loss, AIDE inverse occupancy, uniform, top-tail) and alternative generators (FLUX, caption conditioning), but closely related prior methods are missing. The paper should compare alternative anchor-selection rules, such as MAK- or D$^2$ Pruning-style selection, under the same frozen generator and image budget. For the synthesis component, BRIDGE keeps the generator frozen, whereas the LoRA-based methods discussed above adapt it to the real examples. Given that repair fidelity is not measured (line 472), the paper should provide evidence that the frozen generator produces faithful variants of the selected anchors.
[W2] Limited Scope and Inconsistent Reporting
The practical relevance of the experimental setting is insufficiently justified. (i) Substantial external knowledge is already available through the pre-trained SD 3 generator, and a pre-trained CLIP image encoder is also used for the AIDE baseline (line 374). However, the paper does not evaluate the more direct alternative of adapting a pre-trained image encoder, either with or without BRIDGE. (ii) The experiments are restricted to small source datasets, including standard long-tailed CIFAR configurations, a constructed ImageNet-100-LT source, and 10,000-image subsets of PASS and DiffusionDB. These experiments provide limited evidence for applicability to web-scale SSL pre-training. Larger, uncurated image collections, such as DataComp [7], would provide a more convincing setting.
Moreover, there are unexplained discrepancies across tables, some of which affect the highlighted result. (i) The imbalanced source in Table 1 is built with “the power-law profile of Section 5” (lines 313–315), which also defines ImageNet-100-LT (lines 268–269 and 298–299 in Section 5; Source Data paragraph in Appendix D). The paper does not describe a difference in source construction or training configuration that explains why the SimCLR linear-probe average is 31.2 in Table 1 but 39.8 in Table 2. This baseline discrepancy is consequential because the paper highlights a gain of 11.1 points over SimCLR in Section 1 (lines 078–079) and the Table 1 caption, whereas Table 2 reports a gain of only 2.7 points. (ii) The BRIDGE row of Table 1 (42.3) is identical to the default rows in the Selection, Cycle and Architecture groups of Table 6, whereas Table 2 reports 42.5 with different per-task values (e.g., Cars 15.2 vs. 16.4). (iii) In Table 6, the default BRIDGE result is 41.5 in the Generator group but 42.3 in the Selection, Cycle and Architecture groups.
The paper does not explain the discrepancies listed above or clearly identify which results should be directly comparable. The authors should clarify the configurations behind each baseline and BRIDGE row, and report the main gain against a consistent baseline. These reporting ambiguities reduce confidence in the reported gains.
References:
[1] Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling (NeurIPS 2021; Jiang et al.)
[2] D$^2$ Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning (ICLR 2024; Maharana et al.)
[3] Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach (TMLR 2024; Vo et al.)
[4] DataDream: Few-shot Guided Dataset Generation (ECCV 2024; Kim et al.)
[5] LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidance (BMVC 2025; Kim et al.)
[6] ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets (CVPR 2026; Kim et al.)
[7] DataComp: In search of the next generation of multimodal datasets (NeurIPS 2023; Gadre et al.)
Questions for authors
I have several questions, and I list them below in descending order of importance to my assessment.
[Q1] Consistency Across Tables
As mentioned in [W2], what differences in source data, initialization, training budget, or evaluation explain the SimCLR averages of 31.2 and 39.8 in Tables 1 and 2, the BRIDGE row in Table 1 that matches the default Selection, Cycle and Architecture rows in Table 6 but only partially matches Table 2, and the default BRIDGE averages of 41.5 and 42.3 across the ablation groups in Table 6? Please identify which rows should be directly comparable and correct any reporting errors, since these discrepancies materially affect the reported gain.
[Q2] Connection Between Theory and Method
As mentioned in [W1], which conclusions in Section 3 justify the implemented selection rule? In particular, what establishes that the -NN score reflects effective support or marginal repair utility, and that mode-window selection with FPS approximates the allocation studied in Proposition 4?
[Q3] Evaluation on Pre-trained Encoders
As mentioned in [W2], could the authors evaluate BRIDGE starting from a pre-trained image encoder and compare it with adaptation of the same encoder without repair?
[Q4] Anchor Selection and Generation Fidelity
As mentioned in [W1], could the authors compare alternative anchor-selection rules, such as MAK- or D$^2$ Pruning-style selection, under the same frozen generator and image budget? Could the authors also provide evidence that the frozen generator produces faithful variants of the selected anchors?
[Q5] Alternative Explanation via Conventional Augmentation
Table 6 reports similar performance for generated variants and the real-image oracle (41.5 vs. 40.7), while caption conditioning does not improve performance. These comparisons do not establish whether the benefit requires a pre-trained diffusion generator or could also be obtained through conventional augmentation of the selected anchors. Could the authors compare BRIDGE with a baseline that adds the same number of strongly augmented copies of the same anchors, for example using RandAugment [8], while matching sampling exposure and optimizer updates?
[Q6] Generated Images Selected as Anchors
Generated images account for 34–45% of anchors at cycle 4 (line 474). Could the authors provide qualitative examples that trace anchors and their generated descendants across cycles? Please explain whether later selections revisit already repaired regions, cover newly represented regions, or follow generator artifacts, and how the selection mechanism responds as a region receives additional images.
[Q7] Training Budget Control
BRIDGE uses approximately 1.5 times as many optimizer updates as the corresponding baselines in Tables 2–3 (lines 261–263) because the training set grows while the epoch budget remains fixed. Although the paired ViT-S experiments match optimizer updates, could the authors provide update-matched results for the main comparisons in Tables 2–3?
[Q8] Evaluation on DataComp
As mentioned in [W2], could the authors evaluate BRIDGE on a larger subset of DataComp images, using an update-matched SSL baseline, to assess whether the gains extend beyond the small source datasets used in the paper?
References:
[8] RandAugment: Practical automated data augmentation with a reduced search space (NeurIPS 2020; Cubuk et al.)