acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Logit Debiasing for Unsupervised Cross-Modality Segmentation

Abstract

Unsupervised domain adaptation enables cardiac image segmentation across acquisition domains without target-domain annotations. Existing methods have improved cross-modality transfer through image translation, feature alignment, and prediction regularization, but acquisition-related context can still influence the final segmentation decision. We propose counterfactual logit debiasing (CLD), which compares predictions from observed spatial features and a structural reference under the same global domain context. Both predictions use a shared segmentation head, and their logit difference serves as the final segmentation output. CLD is trained with labeled source images and unlabeled target images. On bSSFP-to-LGE and CT-to-MR cardiac segmentation, it achieves mean Dice scores of 0.823 and 0.778, improving upon the strongest compared UDA baseline by 0.7 and 1.7 percentage points, respectively. Component studies further examine the contributions of the model design and prediction contrast.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
unsupervised domain adaptation, counterfactual logit debiasing, domain shift, cardiac image segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qwzvOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Counterfactual Logit Debiasing (CLD) for unsupervised domain-adaptive cardiac segmentation. A U-Net structure branch produces per-pixel features; a domain branch fuses pooled image features with a learnable domain embedding into a global context vector. A shared head computes factual logits from observed features and counterfactual logits from a broadcast running-mean structural vector under the same context; their difference, inspired by the total direct effect (TDE), forms the final prediction. On bSSFP→LGE (MS-CMRSeg) and CT→MR (MMWHS), CLD reports mean Dice of 0.823 and 0.778, exceeding SLCL by 0.7 and 1.7 points.

Critical strengths
  1. It requires no image translation or multi-stage training and leaves the segmentation backbone unchanged. It is cheap and architecture-agnostic, and would be easy to adopt in existing UDA pipelines. This is the main practical argument for the method.
Critical weaknesses
  1. The counterfactual branch does not perform the claimed structural debiasing. The counterfactual input is spatially constant. Any convolutional or pixel-wise head therefore yields a spatially constant L_cf that carries no image-specific structure. The paper never defines TDE and never tests the mechanism. It gives no analysis of L_cf, no domain probes of X and Z, and no comparison with a learnable class bias or with logit adjustment.

  2. The experiments do not reliably establish an improvement over prior work. All results come from single runs, with no variance estimates or significance tests.

Questions for authors
  1. Please specify f_seg. Because [F̄_loc; Z̃] is spatially constant, any convolutional or pixel-wise head yields a spatially constant L_cf, so L_TDE equals L_fact minus a per-image, per-class offset; with a linear head, Z cancels and inference reduces to a single-branch U-Net.

  2. The target consistency term ||p_fact − p_TDE||_1 vanishes only when L_cf is constant across classes, i.e., when the counterfactual subtraction changes nothing. How do you reconcile this with the debiasing rationale? Does the term help mainly by sharpening L_fact, as an additional entropy-like regularizer?

  3. Please define which local vectors enter the running mean X̄ (source, target or both; background included?), whether you stop gradients through it, and how you initialize it. Please also define Φ^TDE (Eq. 5) and the domain classifier (pixel-wise or image-level).

  4. All baseline values in Table 1 and the baseline row of Table 2 match Gu et al. (MICCAI 2024) to the reported precision. Are they taken from it? If so, please state this, confirm identical splits, folds, preprocessing and backbone, and report your own source-only baseline. How did you obtain the SLCL predictions in Figures 4–5, and if you re-ran SLCL, what scores did it achieve?

  5. Do all results come from single runs? Please report mean ± standard deviation over at least three seeds and paired per-volume tests against SLCL, since the margins are 0.7–1.7 DSC points. Please also report per-class DSC and HD95 for both tasks.

  6. MMWHS uses 25 MR volumes “with automatically generated masks” for training-data augmentation. Do these target-domain masks enter any loss? If so, the setting is not unsupervised. Please also state whether you compute DSC per slice or per volume, which spacing converts HD95 to mm after resizing.

  7. Please provide evidence for the claimed mechanism: visualize L_fact, L_cf and L_TDE on target images; probe how much domain information X and Z each contain.

Reviewer_Nn4KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes counterfactual logit debiasing for unsupervised cross-modality cardiac image segmentation and evaluates it on MS-CMRSeg and MMWHS. It achieves mean DSC scores of 0.823 and 0.778, improving over SLCL by 0.7 and 1.7 percentage points, respectively. However, the gains are limited, and the claimed contributions are not fully supported by the ablation studies.

Critical strengths

Counterfactual modeling is a promising aspect of this work. By replacing structural features while keeping the domain-specific context fixed, the method provides a way to assess the contribution of structural information to segmentation decisions and may reduce spurious correlations caused by differences in imaging domains.

Critical weaknesses
  1. Although the paper presents counterfactual logit debiasing as its main contribution and cites Tang et al.’s TDE-based inference, it does not clearly distinguish the two methods in terms of their causal assumptions, counterfactual reference construction, or specific contributions to medical image segmentation.
  2. In the reported ablation study, introducing the dual-branch architecture raises the mean DSC from 0.529 to 0.796, whereas the final factual–counterfactual logit subtraction provides only an additional improvement from 0.811 to 0.823. This suggests that most of the observed gain may come from the architecture rather than the proposed debiasing operation.
Questions for authors
  1. Please clarify whether SLCL can reasonably be regarded as “the strongest compared UDA baseline” when the comparison only includes methods published up to 2024. Have the authors conducted a systematic survey of recent methods. The results should also be reported as mean ± standard deviation over multiple independent runs to assess the stability of the relatively small gains.
  2. Counterfactual logit debiasing is presented as the main contribution, yet the ablation results suggest that most of the improvement comes from the dual-branch architecture, while the final factual–counterfactual logit subtraction provides only a marginal gain.
Reviewer_FjuTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses unsupervised domain adaptation for cross-modality cardiac segmentation, arguing that domain-related context can still leak into segmentation decisions even after alignment, since structural and global domain context are learned from the same image. The proposed method splits the network into a structure branch and a domain-conditioned global-context branch, computes a counterfactual prediction using a running-mean structural reference under the same fixed domain context, and uses the factual-minus-counterfactual logit difference as the final prediction. On two cardiac cross-modality benchmarks, the method modestly exceeds the strongest reported baseline, with component ablations and a hyperparameter sensitivity sweep included.

Critical strengths

The causal framing motivates a specific, testable architectural intervention rather than a generic regularizer, which gives the method design real explanatory grounding. The component ablation isolates the gain from the two-branch split from the further, smaller gains attributable to the counterfactual contrast specifically, letting the improvement be attributed to the right mechanism. The inclusion of both a supervision-strategy ablation and a multi-factor sensitivity sweep is valuable since it shows the gains are not an artifact of one cherry-picked setting, at least within the ranges tested.

Critical weaknesses

The counterfactual-subtraction technique this method builds on (Tang et al., 2020) was originally designed for a setting where the confounding context is categorical and has a well-defined "neutral" value — in scene graph generation, "no context" is a meaningful baseline to subtract against. Here, the counterfactual reference is a running mean of structural feature vectors across a minibatch of different patients' anatomy, and it is not obvious this quantity is meaningful in the same way. Real cardiac anatomy occupies a highly non-convex manifold of plausible structures; the arithmetic mean of several different patients' structural features is not guaranteed to correspond to any anatomically coherent configuration, and could land in a region of feature space the segmentation head has never seen in a realistic form. If the segmentation head's behavior on this averaged, potentially off-manifold input is unstable or arbitrary, then the "counterfactual" logit being subtracted is not cleanly isolating a domain effect — it could just as easily be injecting noise from an out-of-distribution input, in which case the TDE subtraction's benefit (the smaller of the two gains in Table 2) may be acting more like a regularizer than a principled causal correction. Visualizing or otherwise characterizing what the averaged structural feature actually corresponds to, and checking the segmentation head's stability in that regime, would help establish whether the causal story is doing the work the paper claims.

Related to this, a single global mean conflates very different anatomical structures (left ventricle, right ventricle, myocardium) that have substantially different local feature statistics, since the running-mean reference is computed over the whole image rather than per anatomical structure or per pixel class. A class-conditional reference — a separate running mean per segmentation class — would be a more targeted and more obviously meaningful counterfactual than a single global average, and the paper does not discuss or compare against this alternative despite it being a fairly direct extension of the same idea.

The running-mean statistic is also updated continuously during training, which means early in training it is an average of structural features from a barely-trained, largely uninformative encoder — at that point in training, the "counterfactual" carries essentially no anatomical meaning at all, and applying TDE supervision against it could inject noisy gradient signal before the structure branch has learned anything worth contrasting against. The paper does not describe any warm-up period before the counterfactual loss is applied, nor an ablation over the EMA decay rate governing how quickly the running mean adapts; both would clarify whether this early-training noise meaningfully affects the final result.

The motivating premise — that domain context leaks into segmentation logits in standard UDA pipelines for this task — is argued by analogy to a different task (scene graph generation) rather than demonstrated on the compared baselines directly. A domain classifier's accuracy when probing the compared baselines' pooled/global features, before and after their own training, would show directly whether the leakage this method is designed to fix is actually present in this specific setting, rather than assumed by analogy from an unrelated domain.

Questions for authors

Could the authors confirm whether the sensitivity sweep was conducted on a split distinct from the test set, and report the unreported loss-balancing weight? Could the authors clarify which tensor the domain-adversarial loss acts on, and state whether the running-mean statistic is computed from source, target, or combined features? It would also help to know whether baseline numbers were reproduced under an identical protocol, and whether any seed variance is available.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.