acceptodds
Under review as a conference paper at ICLR 2027

Structured Semantic Consistency for Unsupervised Cross-Domain Nuclei Segmentation

Abstract

Domain shifts in staining, tissue composition, and annotation conventions limit the transferability of nuclei instance segmentation models across histopathology datasets. Prediction consistency alone cannot ensure correct cell-type assignments, making semantic alignment a central challenge in unsupervised domain adaptation. To address this problem, we propose a structured semantic consistency framework that transfers a multi-head nuclei segmentation network from a labeled source domain to an unlabeled target domain. The framework combines source foreground preservation with masked target-view consistency, aligns reliable target features with supervised source-class prototypes, and regularizes inter-class relationships using fixed text embeddings. A class-support constraint is applied only when source and target annotation spaces differ. Training follows a two-stage strategy consisting of joint source–target pre-adaptation and subsequent supervised source refinement. Experiments cover internal domain transfers within PanNuke and cross-dataset adaptation from PanNuke to CoNSeP and Lizard. On CoNSeP and Lizard, class-aware panoptic quality rises from 0.1059 to 0.2335 and from 0.0733 to 0.1127. These results demonstrate the effectiveness of combining prediction consistency, prototype alignment, and semantic relational constraints for cross-domain nuclei instance segmentation without target-domain annotations.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
unsupervised domain adaptation, nuclei instance segmentation, masked image consistency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CcRxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a structured semantic consistency framework for unsupervised domain adaptation (UDA) of nuclei instance segmentation and classification, transferring a multi-head network (NuLite family, FastViT encoder) from a labeled source domain to an unlabeled target domain. The framework combines five components: source foreground feature preservation under appearance changes (SFA); masked target-view consistency, where predictions of a masked view follow confident pseudo-labels from the full view (MIC); alignment of confident target features to source class prototypes (PSA); a loss that matches class-relation matrices between source class centers and fixed CLIP text embeddings (CoPT); and a target class-support constraint that suppresses classes absent from the target label space, used only when label spaces differ (TCC). Training has two stages: joint source-target pre-adaptation, then source-only refinement. The method is evaluated on PanNuke internal transfers (multi-source/multi-target and Breast-centered, with cross-organ comparisons on Breast to Testis, Thyroid, and Bile-duct against UDA baselines) and on adaptation from PanNuke to CoNSeP and Lizard through a unified five-class label space. Class-aware PQ rises from 0.1059 to 0.2335 on CoNSeP and from 0.0733 to 0.1127 on Lizard relative to Source-only, and from 0.4502 to 0.4734 and from 0.4389 to 0.4612 in the two internal settings. Ablations (cumulative addition, component removal, independent addition) are reported, with MIC contributing most. The authors state that target labels are used only for final evaluation and that model selection relies on source validation.

Critical strengths
  1. The method does not use target labels in training, and model selection is done with source-domain validation. This keeps the evaluation consistent with the unsupervised setting the paper targets.

  2. The motivation of the method is clear, and the components are analyzed in several ways (cumulative addition, removal, and independent addition). The paper also reports results that are unfavorable to the proposed method, such as PSA lowering mPQ in the cumulative sequence. This makes the evidence easier to interpret honestly.

Critical weaknesses
  1. The central claim is the combination of existing components, but the evidence that the combination works well is weak. Most components come from prior work: masked-view consistency (MIC, Hoyer et al., 2023), prototype-based alignment (Zhang et al., 2021), and a text-derived class-relation loss described as CoPT-inspired (Mata et al., 2024). Since the integration is presented as the contribution, the paper needs to show that it works well and that the gains are large. It does not. On the external datasets (CoNSeP and Lizard), which carry the headline results, there are no comparison methods, only Source-only. The component analysis is incomplete: the cumulative sequence is run only on Lizard, the independent additions of PSA, CoPT, and TCC give small gains, and MIC alone is never reported, even though removing it drops Lizard mPQ below Source-only. This matters because without this evidence the paper's main contribution cannot be assessed.

  2. The comparison with existing methods is hard to trust. In Tables 1-3, the numbers for Source-only, DA-RCNN, MGA, HT, BAFA, and DA-CC appear identical to those reported in Fan et al. (2024), but the paper gives no source and does not say whether the settings (data, backbone, classes) are the same. Fan et al. use Mask R-CNN, whereas this paper uses NuLite, so the gain cannot be attributed to the method rather than to a stronger backbone. The seven methods in Table 1 are listed by name only, without description or citation, and the correspondence with methods in the related work is not stated. This matters because the within-PanNuke comparison is the only place where the method is compared with other adaptation methods.

Minor: Table 9 does not say how the models were trained or evaluated, shows 11 of the 19 PanNuke tissues, and uses inconsistent bold formatting in the Average row. The Dead class is missing from Tables 1-3.

Questions for authors
  1. Are the numbers in Tables 1-3 for Source-only, DA-RCNN, MGA, HT, BAFA, and DA-CC taken from Fan et al. (2024) or re-run? If taken from it, are the data, backbone, and classes the same as in your setting? What are DA-CC, DDSPSeg, and FAUDA, and where do the DDSPSeg and FAUDA numbers come from? (Clarification)

  2. How were the models in Table 9 trained and evaluated, and why are only 11 of the 19 PanNuke tissues shown? Why is the bPQ gain in Table 9 (about 0.051) more than twice the overall gain in Table 8 (about 0.021)? (Clarification)

  3. Could you report Source-only + MIC, and the removal of SFA and of TCC? Without them it is hard to tell how much each component contributes. (Suggestion)

  4. The reported Source-only on Lizard (bPQ 0.2182, mPQ 0.0733) looks low. In my own experiments with existing state-of-the-art fully-supervised nuclei segmentation and classification models (CellViT, PromptNucSeg, ...) trained on PanNuke and applied directly to Lizard, the PQ was considerably higher. Did you account for the magnification difference between PanNuke and Lizard? Could you report how such fully-supervised models perform on Lizard as a reference? (Clarification and suggestion)

Reviewer_kH8FOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a novel framework for Unsupervised cross-domain nuclei segmentation. In detail, the proposed method uses a fixed text embedding to guide the alignment process for target domain cell type recoginition. Additionally, a class-support constraint is designed for cross annotation space transfer learning. Extensive experiments are conducted on internal domain and cross domain transfer tasks to validate the effectiveness of the proposed method.

Critical strengths
  1. This paper focuses on a highly valuable application scenario in the medical domain.
Critical weaknesses
  1. The overall presentation of the figures is difficult to follow. In Figure 2, six modules are presented in the central part of the diagram, but detailed illustrations are provided for only two of them. Furthermore, the figure does not clearly indicate the order of information flow among these six modules, making it difficult to understand the overall architecture and workflow of the proposed method. In Figure 3, some of the text is too small to read clearly. Additionally, the figure lacks a legend explaining what the different colors and symbols represent. The caption consists of only a single sentence and does not provide sufficient details to help readers understand the figure.
  2. The authors use fixed text embedding as inputs, which may lead to information leak during unsupervised training, as these prompts are relevant to their corresponding labels.
  3. For Figure 5, what are different colors represent for? Also it is hard to tell the difference between different baselines and the proposed method.
  4. For all the equations presented in the paper, none of them provide clear definitions of the mathematical symbols and variables used.
Questions for authors
  1. The authors should provide more clear figures for readers to understand the content of the paper.
  2. The authors should explain whether the prompts used in the method will lead to information leak or not.
  3. The authors should explicitly define each symbol when it is first introduced to improve the clarity and readability of the mathematical formulations.
  4. The manuscript suffers from unclear explanations and a somewhat disorganized logical structure in several places. For example, the baseline methods are not properly introduced, and their corresponding references are missing from the tables. The authors should carefully review and revise the manuscript to improve its overall clarity and presentation quality.
Reviewer_EykKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this manuscript, the authors introduce a structured semantic consistency framework that transfers a multi-head nuclei segmentation network from a source domain to a target domain. The framework combines source foreground preservation, masked consistency, prototype semantic alignment, and text-derived class relations. Furthermore, a target class-prior support constraint is introduced to handle differences between annotation spaces. Experimental results on PanNuke, CoNSeP, and Lizard demonstrate improved segmentation performance across different tissues and datasets.

Critical strengths
  • The proposed framework has demonstrated consistent performance improvements across various cross-organ adaptation tasks. Experimental results on the CoNSeP and Lizard datasets further validate its potential effectiveness under different domain shifts.

  • This manuscript conducts a relatively comprehensive ablation study, including module accumulation and component removal experiments. These experiments clarify the contributions of different consistency components and semantic alignment components.

Critical weaknesses
  • The novelty is relatively limited. Most components are adapted from existing techniques, including masked image consistency, prototype alignment, and text-guided semantic regularization. This manuscript does not clearly establish what new methodological contributions arise from their integration.

  • The cross-dataset experiments on CoNSeP and Lizard lack quantitative comparisons with existing UDA methods. Comparisons with only the Source Only baseline are insufficient to demonstrate the effectiveness of the proposed method over previous work.

Questions for authors
  1. Tables 4, 6, and 7 demonstrate the effectiveness of individual components, but their complementary effects remain unclear. Could the authors further isolate the interactions between PSA and CoPT, for example by comparing their individual and combined effects under the same experimental settings?

  2. The CoPT loss in Eq. (4) only regularizes the source-domain class centers and does not directly align the target-domain features. How does this constraint improve semantic alignment in the target domain?

  3. Why are the UDA methods compared in Tables 1-3 not evaluated on CoNSeP and Lizard? Including these comparisons would help clarify the effectiveness of the proposed method in cross-dataset adaptation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.