acceptodds
Withdrawn from ICLR 2027

Self-improving Semi-supervised Medical Image Segmentation via Consensus-enhanced Prototype-prompted Segment Anything Model

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The scarcity of expert-annotated data has driven the emergence of Semi-Supervised Medical Image Segmentation (SSMIS) as a promising approach. However, due to the absence of ground-truth supervision, current SSMIS methods cannot verify the correctness of self-consistency signals on unlabeled data, resulting in Blind Update. To address this, we propose Self-Improving Consensus-enhanced Prototype-prompted Segment Anything Model (SI-CP2SAM), the first self-improving SSMIS framework, which achieves reliable and accurate segmentation under limited annotation through self-improving mechanism. Specifically, it contains three key novel designs: Attention-guided Kolmogorov-Arnold Prototype Learning (AKaPL), which promotes reliable and debiased representations via Attention-guided Prototype Retrieval (APR) and Kolmogorov-Arnold enhanced Consensus Aggregation (KaCA); Consensus-guided Prototype Decoding (CPD), which yields complementary verifiable signals via prototype-conditioned prompting of SAM; and Anchor-Verified Gated Refinement (AVGR), which maintains a global unlabeled prototype memory and certifies its verifiable evolution via an accept-reject gating mechanism against trustworthy anchors. Extensive experiments on datasets demonstrate the effectiveness of SI-CP2SAM. Code is available at https://github.com/***.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
semi-supervised learning, medical image segmentation, segment anything model, self-improving learning, prototype learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nMkZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SI-CP2SAM, a semi-supervised medical image segmentation framework that aims to address unreliable consistency supervision on unlabeled data. It combines attention-based prototype retrieval and KAN fusion (AKaPL), prototype-conditioned prompting of a pretrained segmentation model (CPD), and a prototype-memory accept/reject mechanism (AVGR). Evaluations on NIH-PAN, MSD-PAN, and BraTS2019 report improved overlap metrics over several conventional semi-supervised baselines. The authors attribute these gains to their proposed verification-gated self-improvement mechanism.

Critical strengths
  1. Encouraging performance in low-label settings. On NIH-PAN with three labeled volumes, the reported Dice is 79.54%, versus 74.90% for the strongest comparator in Table 1 (+4.64 percentage points). The paper also evaluates three datasets and several segmentation metrics. [p. 7, Table 1]
  2. A potentially useful design direction. The combination of labeled-anchor prototypes, an additional pretrained segmentation branch, and an explicit memory-update gate offers a plausible approach to improving representation robustness. The ablations suggest that the complete system benefits from multiple components. [pp. 4-8, Table 2]
Critical weaknesses
  1. The central problem is not sufficiently distinguished from existing failure modes. The manuscript uses "Blind Update" as the main motivation but does not clearly separate it from established confirmation bias and noisy pseudo-label propagation. It also lacks an operational definition of a "verified" update, making the jump from the problem statement to the proposed solution difficult to assess. [pp. 1-2]
  2. The claimed verification is stronger than the implemented gate supports. AVGR tests cosine-distance relationships between prototype representations (Eq. 8), which do not by themselves establish segmentation correctness. Importantly, rejecting a memory candidate does not appear to stop teacher-pseudo-label supervision in the final training objective (Eq. 12). In Eq. 10, the contrastive weighting is 0.2 for accepted candidates and 1 for rejected candidates, further requiring explanation of what is actually filtered. This directly weakens the central claim that erroneous consistency-driven updates are prevented. [pp. 5-6, Eqs. 7-12]
  3. The SAM-derived signal is not established as independent verification. The SAM prompt includes a spatial embedding extracted using the teacher-generated pseudo-label mask for unlabeled images (Eq. 5). Thus, the supposedly complementary signal remains conditioned on the prediction it helps assess. A separately pretrained encoder can provide different features, but it does not guarantee independent prediction errors. The paper does not evaluate correlation or error agreement between branches. [p. 5, Eqs. 5-6]
  4. The experiments do not directly substantiate self-improvement. The final gate improves Dice from 82.81% to 83.30% in Table 2, a gain of only 0.49 percentage points in the reported setting. More importantly, the paper does not show accepted-versus-rejected update correctness, false acceptance, pseudo-label error trajectories, or gate acceptance statistics. Although related SAM-based semi-supervised methods are reviewed, the main comparison omits the closest SAM-based alternatives, making it hard to isolate the new mechanism from the contribution of additional pretrained guidance. [pp. 3, 7-8]
  5. Some methodological details and presentation choices need clarification. The memory is described as initialized from the labeled anchor (p. 4), while Eq. 8 requires strictly smaller distance to that anchor. If the same fixed anchor is used, this condition cannot hold at initialization; if anchors vary, their construction and update schedule should be explicit. The main architecture diagram is dense, component names are not fully consistent, and the plotted momentum axis in Fig. 9 appears inconsistent with the beta values specified in the accompanying text. [pp. 4-5, 9]
Questions for authors

Q1. Scope of the verification mechanism. Which exact gradient-based updates or supervisory signals are blocked when AVGR rejects a candidate? Please reconcile the interpretation of "verified updates" with the continued use of teacher pseudo-label loss and Eq. 10's larger contrastive weight for rejected candidates. Q2. Reliability and gate initialization. Can the authors quantify the true reliability of accepted versus rejected candidates (e.g., segmentation accuracy, error/false-acceptance rates, and acceptance ratios over training)? How is the strict-improvement condition initialized and evaluated if the prototype memory starts from the labeled anchor? Q3. Error dependence of the two branches. Given that the SAM prompt depends on the teacher mask, what supports the claim of complementary or decorrelated verification? Can the authors report error-correlation analysis or disagreement-conditioned accuracy? Q4. Isolating the value of the gate. How does the full framework compare with directly relevant SAM-based semi-supervised baselines, and with simpler confidence/agreement-based gating under matched settings? Can the authors explain the discrepancy between 83.70% Dice in Table 1 and 83.30% in Table 2 for NIH-PAN 10%?

Reviewer_ocazOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To address Blind Update caused by unverified consistency signals in semi-supervised medical image segmentation, this paper proposes SI-CP2SAM, which combines SAM with prototype-based verification to achieve Slighted Update through selective memory refinement. Experiments on three medical imaging datasets show improved segmentation performance over the evaluated baselines.

Critical strengths

The paper addresses an important problem in semi-supervised medical image segmentation and presents a well-motivated framework for learning with limited annotations. The experimental evaluation is substantial, showing consistent improvements across multiple datasets and annotation settings, supported by ablation studies and qualitative comparisons.

Critical weaknesses
  1. The additional contribution of the proposed combination beyond incorporating SAM remains unclear without comparisons to directly relevant frameworks such as CPC-SAM, SemiSAM, and SemiSAM+. Although Figure 6 includes comparisons with MedSAM and MedSAM3D, the experimental settings are insufficiently described to determine how much of the overall improvement comes from incorporating SAM or the proposed methodlogy. For example, were MedSAM and MedSAM3D fine-tuned on the same labeled samples used by the proposed method?
  2. The gate in Eq. (8) checks prototype proximity and agreement, but it is unclear whether these criteria reliably reflect segmentation correctness. The main claim of solving Blind Updata remains unverified. The gating improves segmentation performance in ablation, while comparing the segmentation quality of accepted and rejected candidates would better support the claim.
  3. Minor, providing uncertainty metrics could further strengthen the paper’s contribution.
Questions for authors

See

Reviewer_XqyzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that consistency signals on unlabeled data in semi-supervised medical image segmentation (SSMIS) cannot be verified, and calls this problem “Blind Update.” It proposes SI-CP2SAM, which has three components: AKaPL: A query formed by fusing the labeled prototype and a memory prototype retrieves a “soft” prototype from unlabeled features via cross-attention, and a KAN layer fuses the two into a consensus prototype. CPD: The consensus prototype is fused with a mask-averaged SAM image embedding and used as a sparse prompt for the SAM mask decoder, producing a SAM prediction and a corresponding SAM prototype. AVGR: Three cosine-distance conditions decide whether the unlabeled memory prototype is updated via EMA, and the outcome reweights a prototype contrastive loss.

Critical strengths
  1. The motivation is meaningful: the lack of verification for pseudo-supervision on unlabeled data is a long-standing issue in semi-supervised learning, and using an external foundation model as a complementary signal is a reasonable idea.
  2. The experiments cover three datasets and two label ratios, report both region and boundary metrics, and include a fair number of ablations and sensitivity analyses.
  3. Under extreme label scarcity (NIH-PAN 5%), Dice improves by 4.64 points over the second-best method, which is notable.
Critical weaknesses
  1. The core claim of “verifiable updates” does not match the actual mechanism. It checks prototype similarity, not correctness: The three AVGR conditions only compare cosine distances between prototypes: the distance from the SAM prototype to the labeled prototype, its distance relative to the memory prototype, and the agreement between the soft and SAM prototypes. These conditions test similarity to labeled class means, not whether pixel-level predictions are correct. For example, an under-segmented mask that captures only the organ’s core region may have a prototype even closer to the labeled anchor, making it more likely to be accepted.
  2. Overstated novelty: “The first self-improving SSMIS framework” is an overclaim. Confidence or uncertainty filtering (e.g., UA-MT), curriculum pseudo-labeling, and self-training all essentially filter or check unlabeled signals.
  3. Key baselines are missing. The related work discusses CPC-SAM, SemiSAM, and SemiSAM+, which are SAM-based SSMIS methods, but none of them appears in Table 1. Since the proposed method itself relies on SAM, this is the most direct and necessary comparison. Strong recent SSMIS methods (e.g., ABD, DyCON) are also missing. The method uses copy-paste augmentation; please state whether all baselines use the same augmentation.
  4. The SAM branch is neither independent nor clearly specified. Dependence on pseudo-labels: In CPD, E_grounded is obtained by mask-averaging with the teacher pseudo-label Ỹ^u, and P^u_sam is computed by mask-averaging over the SSMIS model’s own features F^u. The SAM signal thus depends on both the pseudo-labels and the model’s own feature space, so the claim of “decorrelated / independent verification” does not hold. Likewise, AKaPL is described as “pseudo-label-free,” yet it is trained entirely through pseudo-label-based losses.
Questions for authors
  1. Why is w_pro 0.2 for accepted updates but 1 for rejected ones? Does the gate act on L^u_seg and L^u_proto?
  2. Which data split is used for model selection on NIH-PAN?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.