- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes FluxLoRA, a low-rank adaptation framework designed to mitigate the adverse effects of noisy pseudo-label gradients in semi-supervised vision foundation model adaptation. The paper provides theoretical analyses of diffusion stability, conditional core-gradient denoising, and residual contraction, and evaluates the proposed method on VTAB, CIFAR-10, and CIFAR-100 using CLIP and DINOv3 backbones. Nevertheless, several concerns remain regarding the reliability of pseudo-label-conditioned graph guidance, the attribution of the main performance improvements, and the empirical validation of the proposed geometric mechanisms.
Critical strengths
- Well-motivated research problem. The paper identifies an important limitation of conventional pseudo-label filtering and weighting methods: accepted pseudo-label gradients may still introduce harmful updates into low-rank adaptation.
- Interesting geometric perspective. The distinction between gradient contamination within a low-rank subspace and residual gradient components that may induce shifts in the subspace itself provides a logically coherent explanation for how noisy pseudo-labels affect low-rank adaptation.
- Structured method design. The combination of labeled-anchor-guided diffusion, residual gating, tangent-space projection, and low-rank retraction is technically motivated and reasonably well organized.
- Promising empirical performance. The proposed method consistently improves average accuracy across multiple SSL frameworks and two vision foundation models. The gains over standard PEFT baselines on VTAB are substantial.
Critical weaknesses
- The paper motivates FluxLoRA by identifying two distinct effects of noisy pseudo-label gradients: within-subspace core contamination and noise-induced subspace drift. Although the mathematical decomposition establishes how these components can affect low-rank adaptation, it does not demonstrate their effects during semi-supervised training. In particular, Figure 1 provides a conceptual illustration rather than direct empirical evidence that incorrect pseudo-labels substantially corrupt core gradients or cause harmful subspace deviations.
- The labeled-anchor retrieval mechanism in Eq. (5) selects an anchor according to the teacher-predicted class of each unlabeled sample. When this prediction is incorrect, the retrieved labeled anchor necessarily belongs to the predicted class rather than the sample's true class. Consequently, a correctly labeled anchor may provide semantically mismatched guidance to the unlabeled sample. Although the proposed reliability estimation and flux limiter may attenuate such effects, the paper does not directly examine whether incorrect anchor assignments improve or worsen the resulting gradient directions.
- While the ablation studies and matched one-step interventions support the effectiveness of the overall optimization procedure, they do not directly establish that core-gradient correction reduces the discrepancy between pseudo-label core gradients and their clean-label counterparts Similarly, the paper provides a theoretical analysis of subspace drift using principal angles and explicitly introduces principal-angle drift as an evaluation metric in Appendix D.4, but the experimental results do not report quantitative comparisons of subspace geometry across methods.
- Limited attribution of the main VTAB performance gains under fully matched settings. The main VTAB results demonstrate substantial improvements over standard LoRA. The existing component ablations and controlled comparisons are valuable and partially address these factors. Nevertheless, the most tightly controlled end-to-end comparisons in Table 3(b) are limited to CIFAR-10 and CIFAR-100 with DINOv3, where fixed-rank FluxLoRA improves over warm-up-matched LoRA by 0.4 and 0.1 percentage points, respectively. It remains unclear how much of the performance gain on the VTAB benchmark can be attributed to the proposed gradient construction method.
- Insufficient discussion of recent closely related work. The paper does not adequately position FluxLoRA against recent advances in geometric LoRA optimization and noise-robust adaptation. In particular, LoRA meets Riemannion (ICLR 2026) and Riemannian Optimization for LoRA on the Stiefel Manifold (EMNLP 2025) are highly relevant to its manifold-based optimization framework. Recent gradient-aware adaptation methods, including RaLoRA and IGU-LoRA (ICLR 2026), as well as reliability-aware semi-supervised adaptation methods such as TriNoL (2026 preprint), are also not discussed. These omissions make it difficult to assess the methodological novelty of FluxLoRA relative to recent developments.
Questions for authors
- An analysis of standard LoRA under noisy pseudo-labels, comparing core-gradient errors and subspace evolution between correctly and incorrectly pseudo-labeled samples, would help establish the practical significance of the proposed core contamination and subspace drift mechanisms.
- An analysis separating correctly and incorrectly pseudo-labeled samples, together with comparisons of alternative anchor retrieval or graph filtering strategies, would help establish when the proposed graph guidance is reliable.
- A direct comparison of core-gradient alignment or estimation error before and after diffusion, particularly for incorrectly pseudo-labeled samples, would help establish the effectiveness of Core Correction. Similarly, comparing principal-angle trajectories or projection-matrix distances between FluxLoRA and relevant baselines would provide stronger evidence for the effectiveness of Residual Gating in mitigating noise-induced subspace drift.
- The results in Table 3(b) do not invalidate the larger VTAB gains, since the experiments involve different datasets and settings. However, comparable initialization-, rank-, and optimizer-matched evaluations on the VTAB benchmarks would clarify how much of the headline improvement can be attributed specifically to the proposed gradient construction. Statistical uncertainty should also be reported for the smaller controlled gains.
- The authors should discuss the closely related studies mentioned in the Weakness (5) to verify the novelty of FluxLoRA relative to recent developments.