acceptodds
Under review as a conference paper at ICLR 2027

DINO Prior Calibration for Global Context Modulation in Medical Image Segmentation

Abstract

Medical image segmentation requires local details and global context, but unselective global information aggregation may introduce interference in regions with weak boundaries, low contrast, or similar tissues. In medical images, foreground targets and background tissues often have similar feature distributions, whereas DINO priors are mainly learned from natural images with clearer differences between objects and backgrounds. Consequently, directly introducing these priors may cause the model to attend to background regions with similar appearances or weaken its response to the true target and its boundaries. To address these issues, we calibrate DINO priors and supplement them with input image features for medical image segmentation. We propose DPCNet, a DINO Prior Calibration Network for medical image segmentation. DINO Prior Calibration uses coarse segmentation logits to calibrate multi-scale DINO priors. DPCNet then introduces the calibrated DINO priors into GECM as an independent global branch. The calibrated priors also dynamically regulate the structural frequency and selective dependency branches, which supplement the DINO prior with complementary global information from the input image. Experiments on multiple public medical image segmentation datasets demonstrate the effectiveness of DPCNet.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Medical Image Segmentation, Vision Foundation Models, Self-Supervised Learning, Domain Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1N5sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DPCNet, a medical image segmentation framework that calibrates frozen DINOv3 priors using foreground, uncertainty, and boundary cues derived from coarse segmentation predictions. The calibrated priors are incorporated into a Global Expert Collaboration Module (GECM), which combines frequency-based, long-range dependency, and DINO-based features for refined segmentation. Experiments on six medical segmentation datasets demonstrate competitive performance, supported by ablation studies and analyses of prior calibration.

Critical strengths
  1. The paper addresses a relevant problem concerning the transfer of generic DINO representations to medical image segmentation, where irrelevant background responses may interfere with target-specific feature learning.
  2. The experimental evaluation covers diverse medical imaging datasets and includes relatively comprehensive ablations. In particular, the controlled comparisons between raw and calibrated DINO priors provide evidence that the proposed calibration contributes to segmentation improvements beyond the two-stage architecture.
Critical weaknesses
  1. The methodological novelty is not sufficiently established. Using coarse segmentation predictions, uncertainty estimates, and boundary information to guide feature calibration is closely related to existing prediction-guided attention and coarse-to-fine refinement approaches. Moreover, GECM mainly integrates established frequency-domain processing, Mamba, and gated feature fusion mechanisms, making the overall methodological contribution less distinctive.
  2. The generalizability of the proposed calibration mechanism remains unclear. The method is evaluated exclusively with DINOv3 features, without investigating whether similar improvements can be achieved using other pretrained visual encoders. Although the ablation studies demonstrate the effectiveness of DPC within the proposed framework, they do not fully establish whether the benefits reflect a generalizable calibration strategy or are specific to DINOv3 representations.
Questions for authors
  1. What fundamentally distinguishes the proposed DPC mechanism from existing prediction-guided attention, uncertainty-aware feature refinement, and coarse-to-fine segmentation methods? Could the authors provide comparisons with simpler calibration alternatives under the same architecture?
  2. Why is DINOv3 selected as the only pretrained visual encoder? Can the authors evaluate DPC with other representative pretrained models, such as DINOv2, MAE, or CLIP, to demonstrate whether the calibration mechanism generalizes beyond DINOv3?
  3. Given the relatively modest improvement from calibrated DINO priors over raw priors in the controlled ablations, how do the authors justify the additional complexity of the proposed calibration and multi-branch global modeling framework?
2 comments
AuthorsOpenReview ↗

Target-Conditioned Prior Transfer, Encoder Generalization, and Computational Cost

Thank you for your questions. We added simpler alternatives, DINOv2 validation, bridge profiling, and a three-seed comparison.

1. Distinction from attention and refinement methods

Our contribution is target-conditioned selective transfer of natural-image DINO knowledge to medical segmentation, with controlled comparisons showing gains over simpler prior-processing alternatives. The coarse prediction identifies the current medical target, guides adjustment of the external prior, and the adjusted prior then supports fine segmentation. The distinction is target-dependent adaptation of transferred knowledge, beyond prediction-guided weighting of internal segmentation features.

Foreground, uncertainty, and boundary cues condition multiscale residual updates, with Q controlling their magnitude and DINO remaining frozen. The coarse-to-fine structure supplies target cues; GECM combines the prior with local, frequency, and long-range structure. These components support the adaptation and use of the external prior.

Supplementary Table 12 keeps the two-stage network, GECM, and training fixed. Kvasir-SEG Dice is 87.11 with raw priors, 87.44 with foreground-only conditioning, 87.43 with foreground times reliability, and 87.32 with reliability alone; full DPC reaches 87.83. Joint cues therefore improve over the tested simple alternatives.

New experiments replace prior processing with PraNet-style reverse attention or a generic adapter without coarse-prediction conditioning. All use 200 epochs, batch 4, seed 20261108, and the best validation checkpoint; splits are 880/120 for Kvasir-SEG and 517/130 for BUSI fold 0. Kvasir-SEG Dice/Jaccard (%) is 88.21/82.00 for raw priors, 88.04/82.12 for reverse attention, 87.81/81.70 for the adapter, and 88.47/82.26 for DPC. BUSI results are 77.68/70.40, 78.22/71.23, 77.89/70.61, and 79.51/72.55, respectively. DPC has the highest Dice and Jaccard on both datasets. Table 3 also shows foreground/far-background response ratio increasing from 2.02 to 2.84 and far-background share decreasing from 0.79 to 0.54, linking segmentation gains to more selective prior responses.

2. Choice of DINOv3 and encoder generalization

We selected DINOv3 because its spatial representations suit our multiscale prior interface, and the added DINOv2 experiments show that the calibration procedure also works with another pretrained encoder. DPC takes spatial features and target cues; changing the encoder requires adapting extraction layers, spatial sizes, and channel projections, while retaining target-conditioned residual calibration.

Each DINOv2 raw/calibrated comparison fixes the segmentation network, split, training budget, and evaluation. With batch 4 and seed 20261108, 60-epoch Dice improves from 87.35% to 88.32% on Kvasir-SEG and from 78.38% to 78.79% on BUSI fold 0. At 200 epochs, the corresponding changes are 88.50% to 88.62% and 77.86% to 78.35%. The 200-epoch Dice-change 95% CIs, from 20,000 image-paired bootstrap resamples, are [-0.91, 0.95] and [-0.84, 1.90] percentage points (pp). Both include zero: applicability is established, while gain size remains uncertain. MAE and CLIP need separate spatial-feature adaptation and have not been evaluated. We will state the validated encoder scope explicitly.

3. Accuracy gains relative to added complexity

The calibration bridge adds 3.15% measured forward time and about 0.021 GiB peak memory, while improving accuracy in controlled and repeated-seed comparisons. In an independent three-seed Kvasir-SEG experiment, DPC achieves 88.58 +/- 0.15% Dice versus 87.94 +/- 0.24% for raw priors, a mean gain of 0.64 pp (200 epochs, batch 4; mean +/- sample SD across seeds). This complements the single-seed results above.

Supplementary Table 9 isolates calibration while retaining the complete segmentation network and training settings. Dice rises from 87.11 to 87.83 on Kvasir-SEG and from 78.89 to 80.12 on BUSI. We report these original and new 200-epoch comparisons separately because training settings differ.

Paired bridge profiling uses RTX 5090, 512x512 input, batch 1, AMP FP16, 20 warm-ups, and 100 synchronized repetitions. Timing includes frozen DINOv3 and both segmentation stages, excluding I/O, CPU preprocessing, and postprocessing. Enabling DPC adds 1.676M active trainable parameters and 2.986G Conv2d MACs; this MAC count covers bridge convolutions, not the entire network. Peak memory rises from 0.432 to 0.453 GiB. The measured time increase is 3.15%.

GECM's additional value is assessed separately in Table 5: the full branch-and-gating combination improves Kvasir-SEG/BUSI Dice from A+D's 87.19/78.53 to 87.83/80.12. A+D retains CNN anchoring and the calibrated prior. Removing FFT and Mamba reduces branch-experiment latency from 29.27 to 25.34 ms/image (13.41%). These results support calibration and complementary image modeling. We will report their profiling protocols separately.

Reviewer_1N5sOpenReview ↗

Response

My main concern remains the methodological novelty. The response clarifies DPC’s intended role, but its computational distinction from existing prediction-guided and uncertainty-aware refinement remains unclear. The novelty of GECM’s combination of established components is also insufficiently explained. I also remain concerned about generalizability. The DINOv2 gains are small, and both confidence intervals include zero. These results provide limited evidence that DPC offers consistent benefits beyond DINOv3, leaving uncertainty about how broadly the proposed calibration mechanism generalizes.

Reviewer_JZafOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DPCNet, which uses frozen DINOv3 to provide global prior information. The method first generates a coarse segmentation prediction and then calibrates the DINO features through DINO Prior Calibration using foreground, uncertainty, and boundary information. It subsequently uses GECM to integrate multiple contextual features, with the goal of improving medical image segmentation performance. Experiments are conducted on six medical image segmentation datasets, and improvements in segmentation accuracy and boundary quality are reported.

Critical strengths
  1. The paper addresses the potential background interference introduced when transferring visual priors learned from natural images to medical images. The motivation is clear and has practical relevance.

  2. The method performs target-specific modulation while keeping the visual prior frozen, avoiding the training cost of directly fine-tuning a large visual backbone.

Critical weaknesses
  1. Since DINOv3 is pretrained on natural images, the domain gap may limit the practical effectiveness of the transferred prior.

  2. DPC highly depends on the coarse segmentation prediction from the first stage. When the first stage misses the target, the performance of the second stage drops substantially, suggesting that the method has limited ability to correct severe target omissions.

Questions for authors
  1. The paper refers to the core operation as “prior calibration,” but does not directly impose constraints on the properties that the calibrated prior should satisfy. When the module is trained only through the segmentation loss, what evidence shows that it improves the visual prior rather than merely performing coarse-prediction-guided feature adaptation?

  2. Why does the reliability gate use a symmetric form for foreground and background? For high-confidence missed-target regions, how does the model avoid treating the true target as reliable background?

  3. The model contains multiple components, while some components provide only limited gains. Can the authors explain why these components are necessary and whether a simpler combination could achieve comparable performance?

1 comment
AuthorsOpenReview ↗

Prior Calibration Evidence, Symmetric Gating, and Component Contributions

Thank you for recognizing our motivation and target-related modulation. We address calibration, gating, and component value.

1. Domain difference and the meaning of calibration

DPC addresses background interference from natural-to-medical prior transfer, with evidence of improved prior responses and segmentation. Table 3 tests target selectivity; Supplementary Table 9 tests its value while keeping the segmentation network fixed.

Calibration means target-conditioned spatial feature adjustment. Coarse foreground, uncertainty, and boundary cues guide residual updates to frozen multiscale DINO features. Segmentation loss learns the adjustment; target/background separation and boundary responses evaluate its effect, rather than being imposed as separate losses. Thus, calibration denotes improved target selectivity, not probability calibration or complete domain-gap removal.

On 120 held-out Kvasir-SEG images, Table 3 shows that the foreground/far-background response ratio rises from 2.02 to 2.84, the boundary/far-background ratio rises from 1.38 to 1.86, and the far-background response share falls from 0.79 to 0.54. These prior-level changes provide evidence beyond final Dice.

Supplementary Table 9 retains the two-stage network, complete GECM, and training settings, changing whether the prior is calibrated. Kvasir-SEG Dice rises from 87.11 to 87.83, and BUSI Dice from 78.89 to 80.12. Retaining the DPC pathway with a zero prior gives 87.01/78.68 (Table 4). These controls support both DINO information and its adjustment. Feature adaptation implements calibration; improved target selectivity defines its intended effect. We will clarify this distinction.

2. Symmetric gating and confident false negatives

The gate is symmetric because both foreground and background DINO responses need adjustment; the direction of that adjustment remains conditioned on the predicted class and image structure. Q controls residual magnitude, while the original prior and F, U, B determine the learned update. Allowing background updates is important because suppressing irrelevant DINO background responses is a central purpose of DPC.

For example, F=0.05 and F=0.95 can have the same Q under identical boundary conditions. Their distinct foreground cues still allow different residuals, so equal confidence weights do not imply identical calibration.

A confident false negative can receive a high Q, since confidence does not establish correctness. The model addresses such errors through its full information flow: residual addition retains the original DINO prior, and the second stage also receives the image and transferred decoder features. Joint supervision trains localization and refinement through transferred features. Evidence outside the coarse prediction can therefore affect the result.

Supplementary Table 12 supports joint conditioning: full DPC achieves 87.83 Dice, compared with 87.44 for foreground alone, 87.43 for foreground times reliability, and 87.32 for reliability alone. In Table 13, corrupting only the DPC input logits to Dice 1.20 leaves stage two at 75.95, preserving the image and transferred features. This shows segmentation ability under severe prompt corruption, while the 11.88 percentage points (pp) decline identifies severe localization errors as a remaining difficulty. The intervention does not isolate natural confident false negatives. We will clarify its scope.

3. Simpler combinations and component value

We retain the full configuration because its complementary branches and gating improve accuracy on both datasets, with a measured 3.93 ms/image difference when FFT and Mamba are jointly removed. The method follows a coarse-to-fine division of labor: first-stage localization supports second-stage refinement, improving normal Kvasir-SEG Dice from 85.01 to 87.83. The large decline in Table 13 occurs under artificial destruction of the coarse prompt, where the second stage still retains 75.95 Dice.

Table 5 separates the component contributions. CNN anchoring plus the calibrated DINO prior (A+D) achieves 87.19/78.53 Dice on Kvasir-SEG/BUSI. Adding FFT gives 87.38/78.72; adding Mamba gives 87.45/78.86. All branches with fixed summation give 87.58/79.05, and DINO-guided gating raises this to 87.83/80.12. Thus, the complete combination improves over A+D by 0.64/1.59 pp. The larger BUSI gain illustrates the benefit; fixed versus gated fusion isolates the fusion contribution.

Full forward latency in the branch experiment is 29.27 ms/image, compared with 27.66 without Mamba, 27.47 without FFT, and 25.34 without both. Joint removal saves 13.41%; all timings include frozen DINOv3 inference. Independent paired profiling attributes a 3.15% forward-time increase and about 0.021 GiB additional peak memory to the calibration bridge. The full configuration has the highest accuracy in Table 5. We will present accuracy, fusion, and cost together with separate profiling protocols.

Reviewer_VyB8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DPCNet, a prior calibration network designed to mitigate background false activations and prior leakage when transferring natural-image pretrained vision foundation models (DINOv3) to medical image segmentation. The framework adopts a coarse-to-fine two-stage architecture: the first stage produces coarse segmentation logits and extracts foreground, uncertainty, and boundary cues to calibrate multi-scale DINO priors; the second stage dynamically integrates the calibrated priors with convolutional features, frequency features (FFT), and long-range dependencies (Mamba) via a Global Expert Collaboration Module (GECM) for refined segmentation. The effectiveness of the method is validated across six public 2D medical segmentation benchmarks.

Critical strengths
  1. The paper accurately identifies the vulnerability of DINO priors to background leakage in low-contrast medical regions, and the proposed symmetric confidence gating mechanism effectively suppresses irrelevant background interference while enhancing boundary responses.
  2. The supplementary experiments strictly decouple the inherent architectural capacity gains of the two-stage cascaded decoder from the net benefit of prior calibration, while the zero-prior control convincingly demonstrates that the performance gain stems from prior calibration rather than an increased parameter count.
Critical weaknesses
  1. The uncertainty metric and symmetric gating in Equations 1 and 2 strictly rely on the binary probability distribution of a single-channel sigmoid, lacking the theoretical formulation and empirical support required to extend the method to multi-class semantic segmentation.
  2. Concurrently incorporating 2D FFT frequency filtering and Mamba sequence modeling alongside external foundation model priors creates an overly complex architecture, yielding an end-to-end inference latency of 29.27 ms, which is double that of the nnU-Net baseline.
  3. The intervention analysis in supplementary Table 13 reveals that a missed target in the coarse prediction causes the final Dice to drop by 11.88%, demonstrating a heavy dependency on first-stage detection and a lack of self-correction mechanisms to recover false negatives.
Questions for authors
  1. How can the foreground, uncertainty, boundary, and reliability gate terms in Equations 1 and 2 be rigorously reformulated if DPCNet is extended to multi-class semantic segmentation tasks?
  2. What are the performance drops and latency benefits if the computationally intensive Mamba (SDB) and FFT (SFB) branches are removed from GECM, retaining only the CNN backbone and calibrated DINO priors?
  3. When confronting small lesions that are prone to complete omission in the first stage, does the second stage possess any mechanism to recover and recall these false-negative targets?
1 comment
AuthorsOpenReview ↗

Multiclass Extension, Inference Efficiency, and Coarse-to-Fine Refinement

Thank you for your questions. We have tested the multiclass extension, measured branch costs, and clarified the coarse-prompt intervention.

1. Multiclass formulation and ACDC validation

DPC extends to multiclass segmentation, with an explicit definition of all four cues and a completed ACDC validation. Calibration improves Mean Dice by 0.76 percentage points (pp) and Macro IoU by 1.21 pp; both patient-paired 95% confidence intervals are above zero.

For K classes including background, let pk(i)=softmax(z(i))kp_k(i)=\mathrm{softmax}(z(i))_k, with class 0 denoting background. The foreground cue aggregates nonbackground probabilities:

F(i)=1−p0(i)=∑k=1K−1pk(i).F(i)=1-p_0(i)=\sum_{k=1}^{K-1}p_k(i).

Uncertainty uses normalized class entropy, and the boundary cue aggregates horizontal and vertical gradients across classes:

U(i)=−1log⁡K∑k=0K−1pk(i)log⁡pk(i).U(i)=-\frac{1}{\log K}\sum_{k=0}^{K-1}p_k(i)\log p_k(i).

B(i)=clip(12∑k=0K−1∣∇pk(i)∣1,0,1).B(i)=\mathrm{clip}\left(\frac{1}{2}\sum_{k=0}^{K-1}\\|\nabla p_k(i)\\|_1,0,1\right).

Reliability controls the residual-update magnitude:

Q(i)=clip((max⁡kpk(i))ρ(1+βB(i)),0,1).Q(i)=\mathrm{clip}\left((\max_k p_k(i))^\rho(1+\beta B(i)),0,1\right).

ACDC fixes rho=beta=1. The projection receives the full probability vector together with F, U, B, and Q, preserving class identity and boundaries between foreground classes before multiscale residual calibration.

We use background, RV, myocardium, and LV, splitting labeled ED/ES data into 80 training and 20 held-out patients, stratified by pathology. Each patient's slices stay in one set. Raw and calibrated priors share the segmentation network, split, seed, and 60-epoch budget; metrics weight patients equally. Mean Dice rises from 91.05% to 91.81%, with a paired change CI of [0.15, 1.65] pp; Macro IoU rises from 83.93% to 85.14%, CI [0.27, 2.58] pp. RV, myocardium, and LV Dice improve from 90.06/88.28/94.80% to 90.38/89.30/95.75%. Background leakage, the fraction of true background predicted as foreground, falls from 0.181% to 0.138%. CIs use 20,000 patient-paired bootstrap resamples. Thus, the extension has both a defined computation and supporting results.

2. FFT and Mamba: accuracy and efficiency

We retain FFT and Mamba because the complete configuration improves accuracy on both datasets for 3.93 ms/image of additional branch inference time. In Table 5, A+D retains the CNN anchor and calibrated DINO prior, without FFT, Mamba, or the full DINO-guided gate. Its Kvasir-SEG/BUSI Dice is 87.19/78.53, compared with 87.83/80.12 for the full configuration: gains of 0.64/1.59 pp, with the larger benefit on BUSI.

FFT supplies frequency structure, while Mamba supplies long-range context. Adding FFT alone to A+D gives 87.38/78.72 Dice; adding Mamba gives 87.45/78.86. All branches with fixed summation achieve 87.58/79.05, rising to 87.83/80.12 with DINO-guided gating. These results support complementary branches and fusion; the full-versus-A+D gain includes both.

In the RTX 5090 branch experiment (512x512, batch 1), full forward latency is 29.27 ms/image. Removing Mamba gives 27.66 ms (5.49% lower), removing FFT gives 27.47 ms (6.14% lower), and removing both gives 25.34 ms (13.41% lower). Timing includes frozen DINOv3 and both segmentation stages. The reported nnU-Net latency is 14.57 ms; even without the extra branches, our pipeline retains DINO extraction and two-stage inference. Freezing DINO saves training updates, while its forward cost remains included. Separately, paired profiling measures a 3.15% time increase from enabling the DPC bridge. We will distinguish branch costs from the full-system cost.

3. Coarse localization and second-stage refinement

DPCNet is designed to localize coarsely and then refine: normal Kvasir-SEG Dice improves from 85.01 to 87.83, and stage two still achieves 75.95 when the coarse prompt is artificially corrupted to Dice 1.20. The second stage receives the image, transferred decoder features, and calibrated DINO prior, retaining information beyond the coarse mask.

Joint supervision trains localization and refinement:

L=Lrefined+0.2Lcoarse.\mathcal{L}=\mathcal{L}_{\mathrm{refined}}+0.2\mathcal{L}_{\mathrm{coarse}}.

Each term combines Dice and cross-entropy losses. Coarse supervision directly trains target localization; refined supervision also reaches stage one through transferred features. Logits entering DPC are detached, while transferred features retain gradients.

Table 13 replaces only the DPC input logits, preserving weights and transferred features. Thus, Dice 1.20 describes the corrupted prompt, rather than failure of every first-stage representation. The retained 75.95 Dice shows that other information paths support segmentation. The 11.88 pp decline from normal inference also shows that useful localization matters. This intervention assesses severe prompt corruption, not natural small-lesion recovery rates. We will clarify the intervention and the scope of this evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.