- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper builds on RankUp and proposes SymRankor, which treats self-generated pairwise orders as noisy pseudo-labels and replaces the cross-entropy loss on the unlabeled ranking term with a symmetric loss to improve robustness against noisy pseudo-labels. The method is evaluated on vision, audio, and text benchmarks, with an additional chemistry case study that incorporates coarse ordinal information derived from domain knowledge.
Critical strengths
- Replacing the cross-entropy loss on the unlabeled ranking term with a symmetric loss is a lightweight change to RankUp that could be easily adopted.
Critical weaknesses
- The effectiveness of the symmetric loss is not convincingly demonstrated. The primary SymRankor results in Table 1 rely on hyperparameter optimization (HPO), while all baselines, including RankUp, use their default configurations. The tuned hyperparameters include those shared with RankUp (e.g., the ARC loss weights and unlabeled ratio). Without HPO, SymRankor does not outperform RankUp on vision (7.060 vs. 7.026) or audio (0.454 vs. 0.458), and even with HPO it remains behind RankUp + RDA and SSR-RCUS on vision. Since the tuned values vary widely across tasks and no sensitivity analysis is provided, the reported gains may depend more on tuning than on the symmetric loss itself.
- Limited novelty and weak theoretical support. The method largely transfers the symmetric pairwise loss of SymPO to RankUp and is evaluated only within RankUp. Its theoretical analysis restates a known result (Charoenphakdee et al., 2019) under a constant flip-rate assumption, which may not reflect the instance-dependent errors of self-generated pseudo-orders. Recent work that explicitly models such instance-dependent pseudo-label noise in semi-supervised regression (Sun et al., NeurIPS 2025) is neither discussed nor compared. Sun, Xueqing, et al. "Semi-Supervised Regression with Heteroscedastic Pseudo-Labels." Advances in Neural Information Processing Systems (2025).
Questions for authors
- Could the authors tune RankUp with the same HPO protocol? If SymRankor still outperforms it, this would substantially strengthen the main claim.
- How sensitive is SymRankor to its hyperparameters (e.g., β and the ARC loss weights)? A sensitivity analysis would help assess its practicality, given the large variation of tuned values across tasks.
- Can the symmetric loss be applied to other semi-supervised regression methods beyond RankUp? Demonstrating this would support its generality.
- Could the authors discuss or compare with Sun et al. (NeurIPS 2025), which models instance-dependent pseudo-label noise in semi-supervised regression?
- Why is the sigmoid loss chosen as the default? Under the default vision setting with = 0, several symmetric losses in Table 13 (e.g., Gaussian, smooth ramp) achieve lower MAE than sigmoid in Table 1.