acceptodds
Withdrawn from ICLR 2027

Relaxed Distribution-wise Consistency: Learning Affinity Order for Semi-supervised Medical Image Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Semi-supervised medical image segmentation learns from pseudo-labels produced by an imperfect teacher, and its supervision faces a tension between reliability and richness: pseudo-label filtering pursues reliability but discards the boundary pixels where supervision is most needed, whereas inter-pixel supervision pursues richness and reaches these pixels but aligns the teacher's noisy affinity values in a value-alignment manner. Since noisy values may still preserve the correct order, we propose Relaxed Distribution-wise Consistency (RDC), which relaxes inter-pixel alignment from values to order and combines it with filtered pixel-level pseudo-labels. RDC is a simple and lightweight addition to the standard teacher-student framework: each pixel is represented by affinities to well-separated reference pixels (agents), and the affinity order is aligned between teacher and student through a differentiable Kendall's . Our analysis shows that, under symmetric teacher noise, value-alignment targets drift and can even reverse the order of two agents, whereas order alignment preserves the direction of the expected update for each agent pair and only weakens it; we further bound the expected fraction of inverted agent pairs. On ACDC, LA, Pancreas-NIH, and Synapse, RDC achieves state-of-the-art performance with the largest gains on the most difficult structures, and consistently improves GapMatch, GraphCL, VQ-Seg, and the SAM-based ESDE as a plug-in. With the agents fixed, every tested order-based objective outperforms every value-based one.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Semi-supervised Medical Image Segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pcMzOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Relaxed Distribution-wise Consistency (RDC) for semi-supervised medical image segmentation. It replaces strict affinity value alignment with ranking consistency between teacher and student networks, using reference pixels called agents and a differentiable Kendall's tau loss. The authors provide theoretical analysis of robustness to teacher noise and report improved segmentation performance across four medical image datasets.

Critical strengths
  1. The paper addresses a relevant problem in semi-supervised learning, where noisy pseudo-labels can reduce segmentation quality, especially near organ boundaries.

  2. The proposed ranking loss can be integrated into existing teacher-student frameworks without additional inference cost. The reported ablations and noise experiments provide some evidence of its effectiveness.

Critical weaknesses
  1. All quantitative experimental tables are placed in the appendix, while the main paper contains no experimental tables. This makes it difficult to assess the empirical contributions and verify the main claims from the paper itself. For a paper that relies heavily on empirical improvements, the current presentation does not meet the expected standard for publication.

  2. The method mainly combines existing pixel affinity representations with established ranking objectives, including differentiable Kendall's tau. The so-called agents are essentially reference or anchor pixels, rather than autonomous agents, and their selection is adopted from RankMatch. Previous works have already explored ranking-based segmentation and pixel-level relational learning, so the paper needs a clearer distinction from prior methods. To list a few works: He, H., Yuan, Y., Yue, X., and Hu, H. (2022). RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation. European Conference on Computer Vision (ECCV). Ke, T.-W., Hwang, J.-J., and Yu, S. X. (2021). Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning. International Conference on Learning Representations (ICLR). Mai, H., Sun, R., Zhang, T., and Wu, F. (2024). RankMatch: Exploring the Better Consistency Regularization for Semi-Supervised Semantic Segmentation. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3391–340

Questions for authors
  1. Why are all quantitative experimental results placed in the appendix? Which core experiments do the authors consider essential for supporting the main claims?

  2. What is the main conceptual difference between RDC and existing ranking-based or pixel affinity methods? Beyond changing the alignment loss, what is the new technical contribution?

  3. How do the authors distinguish improvements due to ranking information from those due to the robustness and gradient properties of the chosen loss function?

Reviewer_ayobOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Relaxed Distribution-wise Consistency (RDC) for semi-supervised medical image segmentation. Instead of directly aligning teacher–student affinity values, RDC aligns their relative orders using a differentiable Kendall's τ, providing more robust supervision under noisy pseudo-labels. The authors present theoretical analyses of order alignment under teacher noise and evaluate the method on four medical imaging benchmarks. The results demonstrate competitive performance and consistent improvements when RDC is integrated into existing semi-supervised segmentation frameworks.

Critical strengths
  1. The paper identifies an important limitation of value-based consistency under noisy teacher predictions. The proposed order-based alignment is intuitive and supported by theoretical analyses explaining its robustness to teacher noise.
  2. Experiments across four datasets demonstrate strong segmentation performance. The controlled comparisons between order-based and value-based alignment, together with ablation studies and consistent improvements across multiple frameworks, provide convincing evidence of the method's effectiveness.
Critical weaknesses
  1. The theoretical analysis relies on assumptions about teacher noise and the reliability of reference affinity orders. These assumptions may not fully hold in practice, particularly when teacher predictions contain systematic errors. The robustness of order alignment under incorrect teacher rankings remains insufficiently explored.
  2. The method requires pairwise affinity-order comparisons, potentially increasing computational costs as the number of agents grows. The authors should clarify this issue.
Questions for authors
  1. How does RDC perform when the teacher consistently produces incorrect affinity rankings? Can the authors provide additional experiments with systematic or asymmetric teacher noise to validate the theoretical conclusions?
  2. The analysis assumes fixed agent indices and specific perturbation properties. How well do these assumptions hold during actual teacher–student training, where teacher representations and selected agents may evolve?
  3. How does the computational and memory overhead scale with the number of agents and image resolution? Could the authors provide additional efficiency comparisons and sensitivity analyses for different agent numbers?
Reviewer_rrGUOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1(−1)
Presentation
1(−2)
Soundness
1(−1)
Read review
Paper summary

This paper proposes Relaxed Distribution-wise Consistency (RDC) for semi-supervised medical image segmentation. The proposed method adopts orthogonal agent selection from RankMatch and employs a differentiable Kendall's tau objective to align the affinity rankings between teacher and student. The paper also provides a theoretical analysis of the effects of teacher noise on value-based and order-based alignment. Experiments are conducted on four medical image segmentation benchmarks (ACDC, LA, Pancreas-NIH, and Synapse), including comparisons with existing methods, ablation studies, noise robustness evaluations, and integration into several semi-supervised segmentation frameworks.

Critical strengths
  1. A simple and practically useful relational consistency objective.

The proposed method addresses a meaningful limitation of conventional teacher–student consistency learning: uncertain pixels may still contain useful relational information even when their pseudo-labels are unreliable. Replacing affinity-value matching with ranking consistency is intuitive and computationally practical. The method can be incorporated into existing segmentation frameworks without introducing additional inference overhead, which increases its potential applicability.

  1. Extensive ablation studies supporting the usefulness of order-based supervision.

The paper goes beyond standard benchmark comparisons by investigating multiple value-based and order-based objectives under shared agent representations. The noise injection experiments, low-confidence pixel analysis, repeated runs, and plug-in experiments provide useful evidence that order-based consistency can improve segmentation performance. These analyses are among the strongest aspects of the submission.

Critical weaknesses

1. Incomplete comparisons with recent state-of-the-art methods significantly weaken the empirical claims.

My primary concern is that the experimental comparisons are not sufficiently up to date for an ICLR 2027 submission. Although the authors evaluate numerous established baselines, Tables 1–4 largely rely on methods published before 2025, while several highly relevant and competitive approaches from recent top-tier conferences are omitted.

In particular, DyCON (CVPR 2025) [1] is a critical missing baseline. DyCON explicitly addresses the limitations of confidence-based filtering by retaining uncertain voxels through uncertainty-aware consistency and contrastive learning. This motivation closely overlaps with RDC's central argument that uncertain regions should receive additional supervision instead of being discarded. More importantly, DyCON reports substantially stronger results on several overlapping experimental settings:

  • LA (5% labeled): DyCON achieves 91.18% Dice using V-Net, compared with 90.01% reported by RDC.
  • LA (10% labeled): DyCON achieves 91.58% Dice using V-Net, compared with 90.98% reported by RDC.
  • Pancreas-CT (20% labeled): DyCON achieves 84.81% Dice, compared with 83.37% reported by RDC.

Another relevant method is BCSI (AAAI 2026) [2], which reports 91.07% Dice on LA with 10% labeled data, compared with RDC's 90.98%. Furthermore, ALHVR (ICCV 2025) [3] explicitly investigates how to learn from unreliable and high-value regions, making it directly relevant to the motivation and discussion of RDC.

These published results raise substantial concerns about the paper's repeated claims of achieving state-of-the-art performance. The omission of DyCON is particularly consequential because it addresses a similar problem and reports higher accuracy on multiple overlapping benchmarks.

2. Limited methodological novelty and insufficient validation of the claimed noise-robustness mechanism.

The proposed method is conceptually intuitive, but its methodological novelty appears relatively limited. The pixel-agent affinity representation and orthogonal agent selection are directly adopted from RankMatch (CVPR 2024) [4], while the differentiable Kendall's tau objective follows DiffKendall (NeurIPS 2023) [5]. Moreover, Guan et al. (ICML 2025) [6] have already investigated Kendall's tau ranking loss for teacher–student knowledge distillation, including its advantages over conventional KL-based alignment.

Consequently, the main algorithmic contribution of RDC appears to be the integration of existing agent-based relational representations and differentiable ranking objectives into a semi-supervised medical image segmentation framework. While this combination may be practically useful, its conceptual distinction from previous ranking-based distillation and consistency learning methods needs to be articulated more clearly.

More importantly, the experimental evidence does not fully isolate the proposed noise-robustness mechanism:

  1. the theoretical noise assumptions are not adequately connected to the empirical measurements. Theorem 1 analyzes teacher-affinity perturbations around a reference affinity vector, with symmetric perturbations providing the strongest directional preservation guarantee. However, Appendix C mainly measures differences between student and teacher affinities. Such differences are not necessarily representative of the fluctuations of the teacher itself across weak augmentations or training states. Therefore, the empirical relevance of the assumed perturbation model remains insufficiently established.

  2. the discussion of orthogonal agent selection in Section 4 appears to overinterpret the geometric bound. A lower bound on the distances between agent embeddings does not directly imply a lower bound on their affinity margins for arbitrary pixel embeddings, since the affinity gap also depends on the projection of the pixel embedding onto the difference between agents.

Overall, these concerns limit my confidence in both the claimed methodological contribution and the strength of the theoretical and empirical conclusions.

References

[1] Assefa, M., Naseer, M., Ganapathi, I. I., Ali, S. S., Seghier, M. L., and Werghi, N. "DyCON: Dynamic Uncertainty-aware Consistency and Contrastive Learning for Semi-supervised Medical Image Segmentation." CVPR, pp. 30850–30860, 2025.

[2] Huang, K., Zhang, Y., Zhou, Y., Xu, T., and Zhou, T. "Bidirectional Channel-selective Semantic Interaction for Semi-Supervised Medical Segmentation." AAAI, 40(7):5040–5048, 2026.

[3] Lei, T., Yang, Z., Wang, X., Wang, Y., Wang, X., Sun, F., and Nandi, A. K. "Adaptive Learning of High-Value Regions for Semi-Supervised Medical Image Segmentation." ICCV, pp. 21450–21459, 2025.

[4] Mai, H., Sun, R., Zhang, T., and Wu, F. "RankMatch: Exploring the Better Consistency Regularization for Semi-supervised Semantic Segmentation." CVPR, pp. 3391–3401, 2024.

[5] Zheng, K., Zhang, H., and Huang, W. "DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation." NeurIPS, vol. 36, 2023.

[6] Guan, Y., Cheng, R., Liu, K., and Yuan, C. "Enhancing Logits Distillation with Plug&Play Kendall's τ Ranking Loss." ICML, PMLR 267:20537–20552, 2025.

Questions for authors
  1. Recent SOTA comparisons.

Could the authors clarify why DyCON (CVPR 2025), ALHVR (ICCV 2025), and BCSI (AAAI 2026) are absent from the experimental comparisons and related discussion? DyCON is especially relevant because it explicitly leverages uncertain voxels and reports stronger results on several overlapping settings.

  1. Connection between the theoretical assumptions and empirical teacher noise.

Theorem 1 considers perturbations of teacher affinities around a reference vector, whereas Appendix C analyzes student–teacher affinity differences. Could the authors clarify why the latter is an appropriate proxy for the former? Is there empirical evidence that the actual teacher perturbations across weak augmentations or training states satisfy, at least approximately, the assumed symmetry condition?

1 comment
Reviewer_rrGUOpenReview ↗

Serious research-integrity concern: undisclosed overlap with a previously published RDC paper and possible plagiarism or redundant publication.

I identified substantial overlap between this submission and the previously published RDC paper [1], which appeared on January 5, 2026. The earlier paper already presents Relaxed Distribution-wise Consistency for semi-supervised medical image segmentation, including orthogonal agent selection, pixel–agent affinity modeling, and the replacement of strict affinity-value alignment with ranking alignment to improve robustness to teacher noise. These elements closely correspond to the central methodological framework and motivation of the current submission.

The overlap also extends to specific experimental results. Table 5 of [1] reports ACDC Dice scores of 87.21, 88.25, and 88.98 for the baseline, contrastive alignment, and distribution consistency, respectively. Table 7 of [1] reports 88.68, 88.79, and 88.98 for the L2, CE, and KL objectives. These values appear identically in Table 2 of the current submission. This correspondence requires an explicit explanation of the relationship between the two methods and the provenance of the reported results.

I acknowledge that the current submission replaces the earlier soft-rank objective with a differentiable Kendall objective and adds theoretical analysis and experiments. However, I could not find a citation to [1] or an explanation of this relationship in the revised manuscript or the rebuttal provided for review. Consequently, the manuscript does not adequately distinguish its new contributions from this directly relevant, already published work.

This unexplained overlap raises serious concerns about possible plagiarism or undisclosed redundant publication and warrants investigation by the Area Chair. The authors should clarify the relationship to [1], identify any reused methods or experimental results, and explicitly establish the incremental contribution. The observed overlap alone does not establish misconduct, but it is sufficiently substantial to require formal clarification and assessment before an acceptance decision. Until this concern is adequately resolved, I cannot support acceptance.

Reference

[1] SUN R, MAI H, LI Z, et al. Relaxed distribution-wise consistency for semi-supervised medical image segmentation[J]. CAAI Transactions on Intelligent Systems, 2026, 21(1): 132–145. DOI: 10.11992/tis.202507034.

Comments on the paper

2 comments
Reviewer_rrGUOpenReview ↗

Serious research-integrity concern: undisclosed overlap with a previously published RDC paper and possible plagiarism or redundant publication.

I identified substantial overlap between this submission and the previously published RDC paper [1], which appeared on January 5, 2026. The earlier paper already presents Relaxed Distribution-wise Consistency for semi-supervised medical image segmentation, including orthogonal agent selection, pixel–agent affinity modeling, and the replacement of strict affinity-value alignment with ranking alignment to improve robustness to teacher noise. These elements closely correspond to the central methodological framework and motivation of the current submission.

The overlap also extends to specific experimental results. Table 5 of [1] reports ACDC Dice scores of 87.21, 88.25, and 88.98 for the baseline, contrastive alignment, and distribution consistency, respectively. Table 7 of [1] reports 88.68, 88.79, and 88.98 for the L2, CE, and KL objectives. These values appear identically in Table 2 of the current submission. This correspondence requires an explicit explanation of the relationship between the two methods and the provenance of the reported results.

I acknowledge that the current submission replaces the earlier soft-rank objective with a differentiable Kendall objective and adds theoretical analysis and experiments. However, I could not find a citation to [1] or an explanation of this relationship in the revised manuscript or the rebuttal provided for review. Consequently, the manuscript does not adequately distinguish its new contributions from this directly relevant, already published work.

This unexplained overlap raises serious concerns about possible plagiarism or undisclosed redundant publication and warrants investigation by the Area Chair. The authors should clarify the relationship to [1], identify any reused methods or experimental results, and explicitly establish the incremental contribution. The observed overlap alone does not establish misconduct, but it is sufficiently substantial to require formal clarification and assessment before an acceptance decision. Until this concern is adequately resolved, I cannot support acceptance.

Reference

[1] SUN R, MAI H, LI Z, et al. Relaxed distribution-wise consistency for semi-supervised medical image segmentation[J]. CAAI Transactions on Intelligent Systems, 2026, 21(1): 132–145. DOI: 10.11992/tis.202507034.

The paper [1] raised in this comment (Rui Sun, Huayu Mai, Zhi Li, Yu Liu, and You He, Relaxed distribution-wise consistency for semi-supervised medical image segmentation, CAAI Transactions on Intelligent Systems, 21(1): 132–145, 2026, in Chinese, DOI: 10.11992/tis.202507034) is our own prior work.

In fact, this submission goes well beyond [1]. [1] proposed an empirical rank alignment, which approximates the rank of each agent with sigmoid functions and matches the teacher and student ranks through a rank-overlap ratio, without any analysis of why or when such a relaxation helps. This submission starts from this question, and Section 4 characterizes what the student learns from an imperfect teacher. Lemma 1 shows that matching the Plackett–Luce distributions of two affinity vectors contains matching the affinity distributions themselves, so the affinity consistency of RankMatch remains a value alignment. Theorem 1 decomposes the expected update under teacher noise: for value alignment, the update consists of the update toward the teacher's reference affinities plus a drift term, which can push the student against the correct direction even when the teacher's order never changes; for order alignment, teacher noise only scales each pairwise update by a factor κ, and the direction is kept if and only if κ > 0. Proposition 1 bounds the expected fraction of inverted relations without any independence assumption across agents or relations, and the accompanying remarks show that an individual relation inverts with a probability that decays exponentially in its squared margin, that the aggregate inversion rate follows a power law in the noise scale, and that orthogonal agent selection lifts the cap on the margins. Following this analysis, the objective is changed from the rank-overlap ratio of [1] to pairwise order alignment through a differentiable Kendall's τ, which directly counts the disagreeing relations bounded in Proposition 1 and decomposes exactly into relational pseudo-labels trained with a margin-weighted symmetric loss (Equation 9). On the same baseline and agents, this raises the Dice on ACDC-5% from 90.15 to 90.45.

The experiments are likewise rebuilt, and all results of RDC are new runs with the Kendall objective. The controlled comparison of Table 2 fixes the agents and the baseline and changes only the alignment across twelve objectives, including the Plackett–Luce KL, SoftSort, the noise-robust GCE and SCE, an auxiliary Kendall loss added to KL, RankNet, the pairwise hinge, and Spearman, and every order alignment (89.75–90.45) outperforms every value alignment (88.68–89.30). Section 5.4 and Appendix C test the analysis on trained models: the observed inversion rate stays below the bound of Proposition 1 in every margin stratum and lies within 16% of it in the largest-margin stratum; the teacher's own fluctuation satisfies κ̂ > 0 for 97.82% of the relations and is approximately symmetric at every stage of training; on labeled pixels whose class decision fails, 58.25% of the relations still point in the right direction, above the chance level of 50%; removing near-tied relations leaves the Dice unchanged (90.49); and noise injected into the teacher embeddings reproduces the exponential decay of individual relations (R² = 0.868) and the power law of the aggregate inversion rate (slope 0.89, R² = 0.993). Consistent with Theorem 1, noise injected into the teacher affinities widens the gap between KL and RDC from 1.47 to 3.31 Dice under symmetric noise and to 3.96 and 4.16 under asymmetric and systematic noise. The pixel-subset ablation of Table 3 shows that 1.30 of the 3.24 Dice gain comes from the low-confidence pixels that filtering leaves behind. The evaluation further adds Synapse, where RDC improves the best baseline by 4.0 and 2.4 Dice with one and two labeled cases; plug-in results on six frameworks, namely GapMatch, GraphCL, VQ-Seg, DyCON, BCSI, and the SAM-based ESDE, with gains of 0.66–2.73 Dice; comparisons with DyCON, BCSI, and ALHVR under the same protocol; five repeated runs with paired t-tests (p < 0.01); and analyses of the training cost, the agent selection, and the regimes in which order alignment helps less.

The scope of the work and its central claim therefore differ from [1]. [1] is an empirical method, whereas the central claim of this submission, which our responses in this discussion also focus on, is the analysis of what the student learns from an imperfect teacher, and the objective and the experiments are designed around this analysis. This submission is therefore not substantially similar to [1] and does not violate the ICLR dual-submission policy.

Stating that [1] is our own work, however, breaks the anonymity of the double-blind review. We therefore withdraw the submission and will revise its presentation, including a proper citation of and comparison with [1].

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.