acceptodds
Under review as a conference paper at ICLR 2027

SigMI: Sigmoid Contrastive Models Estimate Pointwise Mutual Information

Abstract

Histogram-based approximations of mutual information (MI) have long been used as multimodal similarity measures for medical image registration. Representation learning emerged as an alternative approach, but its connection to mutual information is lost when applied to image registration. We connect both lines of work by showing that sigmoid contrastive models like SigLIP estimate pointwise mutual information (PMI): being trained to distinguish samples from a joint distribution from deranged pairs approximating the product of its marginals, their optimal logits recover PMI up to a correction for the sampling class prior. These sigmoidal mutual information estimators (SigMIs) can estimate calibrated PMI, independent of the training prior. Their estimation error can be described in terms of excess classification risk, and they provide information-theoretic interpretations of SigLIP's temperature and bias parameters. Applying our findings to medical image registration, we propose fully convolutional SigMIs that learn dense image representations whose dot products estimate local PMI. We train our models on domain-randomized synthetic images generated from just five brain segmentation maps. Across three registration algorithms and four datasets, SigMIs outperform other similarities in 8 of 12 settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
mutual information estimation, pointwise mutual information, image registration, medical imaging, multimodal registration, domain randomization, image similarity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_stLvOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper connects the SigLIP loss to an estimator of pointwise mutual information and applies this finding to derive a novel approach for multimodal medical image registration (or rather a plug-in loss usable with existing frameworks). The approach relies on synthetic data following the domain randomization paradigm. It is evaluated on 4 multimodal multi-organ datasets including US/CT/MRI images, on top of 3 registration frameworks. It is then applied and compared end-to-end vs. full registration frameworks, showing promising results.

Critical strengths

The paper presents a rigorous analysis of the SigLIP loss as a pointwise mutual information estimator biased by a class prior directly related to batch size. I found this analysis insightful and clearly presented. Although I am not sure it is entirely novel, I could not clearly tie this analysis to a specific previous paper.

Secondly, the paper connects those insights to a practical registration framework and shows promising results. The authors also make a sensible use of convolutional architectures with known receptive field to train the PMI estimator at patch-level and apply it on full volumes in convolutional manner, which is a sound and practical approach.

Finally, the paper is clear, well structured and easy to follow.

Critical weaknesses

The viability of the resulting domain randomization + PMI estimator registration strategy is not fully established by the paper, but in my opinion this is acceptable given its theoretical contribution.

A weakness of the approach is its reliance on multimodal data with perfect alignment (i.e., known ground-truth displacement). This virtually constrains the approach to in fact follow this domain randomization strategy with synthetic data, and limits its scope.

However at this stage, suspected weaknesses are not substantive and I will move them to open questions instead.

Questions for authors
  1. One of my concerns is that while the SigLIP based PMI estimator is theoretically well-grounded, it would be in practice difficult to train due to the constraint on batch size. Could authors make more precise the typical batch size at which they manage to work on 3D patches? If you extract too few random patches from typically large volumes, minimizing the SigLIP loss (i.e. identifying positives vs. negatives) might be trivial due to the large diversity of appearance of patches (making it likely that all sampled pairs heavily differ from pair to pair). I would assume that if the patch positives vs. negatives can be trivially identified, the approximation quality of the PMI degrades? Can you also comment on the choice to use non-overlapping patches vs. constraining the maximum overlap?

  2. Can authors comment on the ellipsoidal masking step and how it affects the reliability of the loss "matching" anatomical regions cross-modality? The masking seems to introduce irrealistic black regions and boundaries in patches, which would deviate from the data observed in inference? To what extent is the trained model affected by this or its performance upper-bounded by this?

  3. Can the authors comment on the trade-offs in choosing the receptive field for the estimator (here R9 or R17, but many other values could have been considered a priori). Does the performance change gracefully with higher receptive field values, or does it become impractical to train because of patch size or shortcuts? What about values between 9 and 17?

3 comments
AuthorsOpenReview ↗

Author Response (Part 1)

Thank you for your positive and detailed review, and for finding our analysis of SigLIP rigorous, insightful, and clearly presented, and our fully convolutional design sound and practical. Your questions led to three key additions in the revised manuscript: SigMI-R33 in Table 1, a field-of-view masking ablation (Appendix C.6), and a batch-size study against InfoNCE (Section 4.3, Figure 5). Revised text is in blue.

"The viability of the resulting domain randomization + PMI estimator registration strategy is not fully established by the paper, but in my opinion this is acceptable given its theoretical contribution."

Thank you. The revision strengthens this evidence. Table 1 now reports paired significance tests: of the 72 baseline–setting comparisons, 51 are significantly worse than SigMI and 4 significantly better. While the strongest baseline changes with the dataset, SigMI-R9 alone has the best average rank over the 12 settings (1.8, against 2.8 for squared LNCC and 2.7 for MIND-SSC). Against complete registration methods (Section 4.4), SigMI with MultiresSVF has the second-best median on both LuMIR and RESECT, and the best mean TRE on RESECT, without any real training image.

"A weakness of the approach is its reliance on multimodal data with perfect alignment [...]. This virtually constrains the approach to in fact follow this domain randomization strategy with synthetic data, and limits its scope."

We agree that sampling the joint distribution requires perfectly aligned pairs, which is why we train on synthetic data only. We see this less as a limit on scope than as what makes the approach general. Because SigMI scores local patches, the synthetic data only need to be realistic at the patch scale, not as whole volumes, unlike networks trained on full synthetic volumes such as SynthMorph (Appendix B.2). In particular, five brain label maps suffice for an estimator that transfers to real MRI, CT, and ultrasound and to abdominal anatomy, without seeing any real image during training (Table 1). Where aligned real pairs exist, they can be added to the training data without changing the method.

"it would be in practice difficult to train due to the constraint on batch size. Could authors make more precise the typical batch size at which they manage to work on 3D patches?"

The batch size is not a constraint for SigMI. In theory, the prior correction makes the optimal score PMI for any BB (Theorem 2). In practice, memory is not the limit either: a batch of 16,384 R17 patch pairs fits on a single 80 GB H100. We chose B=32B = 32 patch pairs, with gradients accumulated over four batches (Appendix B.2), for training speed: at this batch size, SigMI-R9 and SigMI-R17 each train in under 8 hours on one H100, whereas larger batches are slowed down by loading more image pairs. The new Section 4.3 varies BB from 4 to 64 for SigMI-R17 (Figure 5). Going down to B=4B = 4, its TRE on RESECT and Dice on TRUSTED barely change and its LuMIR Dice drops by less than one point; going up to B=64B = 64 changes these three by at most 0.17 Dice points and 0.09 mm. Abdomen, with only 4 test pairs, varies more across batch sizes.

"If you extract too few random patches [...], minimizing the SigLIP loss [...] might be trivial [...]. I would assume that if the patch positives vs. negatives can be trivially identified, the approximation quality of the PMI degrades?"

Yes, and this concern shaped our design: trivially separable joint and product-of-marginals samples give strong classifiers but poor density-ratio estimates (Rhodes et al., 2020). We make negatives hard in three ways (Section 3.5, Appendix B.1). Each batch draws spatially connected groups of patches from a few image pairs (at most 16 per image), which makes many negatives neighboring, visually similar patches rather than distant anatomy. Small receptive fields keep patches geometrically simple. Field-of-view masking removes the silhouette as a shortcut.

SigMI's advantage over InfoNCE grows with the receptive field: significant wins are tied 4–4 at R9, and 8–0 and 12–0 for SigMI at R17 and R33 (Section 4.3, Table 2). We attribute this to easy negatives: larger patches are easier to tell apart, and easy negatives hurt InfoNCE, which scores each pair relative to its negatives, more than SigMI, which judges each pair independently. Masking likewise helps more as patches grow (Table 7), since larger patches more often fall on a field-of-view border.

(continued below)

AuthorsOpenReview ↗

Author Response (Part 2)

"Can you also comment on the choice to use non-overlapping patches vs. constraining the maximum overlap?"

Patches that share voxels are dependent and cannot be samples from the product of marginals. Labeling them as negatives would thus hurt the estimator. Labeling them as positives would make the score saturate before alignment is reached. A maximum overlap would trade a small bias for harder negatives, and our augmentations already allow occasional small overlaps, which we accept as minor training noise (Appendix B.1). Graded supervision by overlap would go beyond PMI estimation, and we leave it for future work.

"Can authors comment on the ellipsoidal masking step [...]? The masking seems to introduce irrealistic black regions and boundaries in patches, which would deviate from the data observed in inference? To what extent is the trained model affected by this or its performance upper-bounded by this?"

The masks mimic field-of-view borders, which do occur at inference, for example at the edge of the ultrasound cone in RESECT and TRUSTED, or with partial coverage in CT and MRI. Because each patch receives an independent, smoothly deformed ellipsoidal mask, training covers pairs in which one patch shows anatomy that the other only partly shows, as at a field-of-view border during registration. Masks are applied with probability 0.7, leaving 30% of training pairs unmasked (Appendix B.1).

The new ablation retrains all three variants without masking (Appendix C.6, Table 7). Masking never significantly hurts, and its benefit grows with the receptive field: at R9 it changes results by at most 0.09 Dice points and 0.16 mm, while at R33 it significantly improves Abdomen by 11.06 Dice points, RESECT by 4.54 mm, and TRUSTED by 0.39 Dice points. On LuMIR, where both modalities cover the same anatomy, it changes R9 and R33 by at most 0.03 Dice points and significantly improves R17 by 0.26. We therefore see no sign that the masked regions limit performance on unmasked data.

"Can the authors comment on the trade-offs in choosing the receptive field for the estimator [...]? Does the performance change gracefully with higher receptive field values, or does it become impractical to train because of patch size or shortcuts? What about values between 9 and 17?"

The receptive field trades shared information against local precision. A larger window captures more information shared between modalities, consistent with the learned PMI range widening with R: its upper end grows from 6.1 nats at R9 to 9.6 and 10.6 nats at R17 and R33 (Table 3). This context helps under large misalignment, but each estimate then averages over a larger region and localizes deformations less precisely. Small windows also make negatives harder and reduce the gap between synthetic and real images (Appendix B.2). The new SigMI-R33 results in Table 1 reflect this trade-off. The best SigMI variant is R9 in 6 of the 12 settings, including all three on LuMIR, R17 in 3, and R33 in 3: on Abdomen, which has the largest initial misalignment (32.65% Dice before registration), with MultiresSVF and ConvexAdam, and on RESECT with ConvexAdam. R33 remains practical to train (2.1M parameters, Table 3), but it is more prone to shortcuts, which is where the sigmoid objective and masking matter most (Tables 2 and 7, discussed above).

Values between 9 and 17 would require different kernel sizes per stage: with the same kernels in all four stages, R=1+8mR = 1 + 8m (Appendix B.2), which gives 9, 17, 25, and 33. Given this trade-off, and since MultiresSVF and FireANTs SyN are multi-resolution, a natural extension is to use larger receptive fields at coarse levels and smaller ones at fine levels.

Thank you again for your questions, which improved the paper. If our answers address them, we would be grateful if you would consider raising your score, and we would be glad to discuss any further questions.

Many thanks to the authors for taking the time to answer my comments and questions. I am satisfied with these responses and updates, which address the points I raised. On the other hand, my initial review underestimated a bit the overlap of the theoretical part with existing papers, as pointed out by other reviews. All put together, I will maintain my initial rating as my assessment is that the paper makes contributions of interest to the community in the context of multimodal image registration.

Reviewer_KF7SOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that the sigmoid loss used in SigLIP can be interpreted as a batch-size-independent estimator of pointwise mutual information (PMI), which the authors call SigMI. They train a fully convolutional 3D encoder using only synthetic images generated from five brain templates. They then evaluate the method for image registration across four datasets and three solvers, comparing eight similarity measures, including their two proposed measures.

Critical strengths
  1. A clear reinterpretation of SigLIP with practical implications. The prior correction comes at no additional cost and is validated against ground-truth PMI (Fig. 3).
  2. Robust transfer across domains and solvers. A single encoder trained entirely on synthetic data transfers to abdominal and ultrasound image registration across three solvers.
Critical weaknesses
  1. The theoretical contribution is overstated relative to prior work. The identity "logit = PMI − log(something)" has appeared in NCE, Levy & Goldberg (2014), DEMI (2020), and Tsai et al. (2020). Uchiyama et al. (CVPR 2026) [1] also discuss this connection specifically for SigLIP.
  2. The registration results provide weaker evidence than claimed and do not directly test the paper’s central claim. The reported "8 of 12" combines wins from two variants, each of which wins only 6 of 12 settings. The test sets are small (18/4/14/47 pairs), and no statistical significance tests are reported.
  3. Several ConvexAdam results need discussion. On RESECT, four baselines perform substantially worse than the unregistered images (18.8–24.6 vs. 5.60 mm). On TRUSTED, every method, including SigMI, scores below the unregistered baseline (84.74). These results are left unexplained. [1] Fumiya Uchiyama et al. "CLIP-like Model as a Foundational Density Ratio Estimator." CVPR 2026, pp. 15784–15793. arXiv:2506.22881.
Questions for authors
  1. Could you explain the anomalous ConvexAdam results: RESECT TRE, and all TRUSTED results falling below the unregistered baseline? These results raise concerns about the reliability of the evaluation. Should these settings count toward the claimed "8 of 12" wins?
  2. Were the Section 4.3 variants (R9 for LuMIR and R17 for RESECT) selected using validation or test data? What was the rationale for these choices?
2 comments
AuthorsOpenReview ↗

Author Response (Part 1)

Thank you for your review, and for noting that the reinterpretation of SigLIP is clear and has practical implications, that the prior correction is validated against ground-truth PMI, and that transfer across domains and solvers is robust. Your comments led us to fix our TRUSTED ConvexAdam setup, add paired significance tests to Table 1, add a matched InfoNCE comparison (new Section 4.3), and report the "8 of 12" result per variant. Revised text is in blue.

"The theoretical contribution is overstated relative to prior work. The identity "logit = PMI − log(something)" has appeared in NCE, Levy & Goldberg (2014), DEMI (2020), and Tsai et al. (2020). Uchiyama et al. (CVPR 2026) [1] also discuss this connection specifically for SigLIP."

We agree, and thank you for the reference to Uchiyama et al. (2026). We have corrected this overclaim: Theorem 2 is now stated "as in" Gutmann & Hyvärinen (2012), Levy & Goldberg (2014), Liao et al. (2020), and Tsai et al. (2020), Uchiyama et al. are cited at Contribution (ii) and Corollary 4, and the abstract and introduction now say that we "make explicit" that SigLIP estimates PMI. The revision instead highlights our new theoretical contributions: a bound on the pointwise PMI error at finite capacity, with an explicit 1/π1/\pi factor for SigLIP's class imbalance (Corollary 6), and an information-theoretic interpretation of SigLIP's temperature and bias as the half-width and center of the estimator's range (Section 3.3).

"The registration results provide weaker evidence than claimed and do not directly test the paper's central claim."

The central claim has two parts, and the revision tests each directly. That the prior-corrected SigLIP logit is PMI can only be checked where the true PMI is known, which is why Section 4.1 compares against ground truth on 13 BMI tasks (Figure 3). That this estimator matters for registration is now tested in Section 4.3, with InfoNCE models that differ from SigMI only in the objective (same architecture, synthetic data, batch construction, B=32B = 32, and hyperparameter search). With paired tests over the 12 dataset–solver settings, the two objectives are on par at receptive field R9 (4 significant wins each), while SigMI wins 8 and 12 settings at R17 and R33 with no significant loss. Retraining both R17 models with BB from 4 to 64 shows that small batches hurt InfoNCE more than SigMI (Figure 5).

"The reported "8 of 12" combines wins from two variants, each of which wins only 6 of 12 settings."

You are right and we have clarified this point. The abstract reads "two SigMI variants each win in 6 of 12 settings, together in 8", and the contributions, Section 4.2, and the conclusion give the per-variant counts. For a single estimator, SigMI-R9 has the best average rank against the six baselines over the 12 settings (1.8, against 2.8 for squared LNCC and 2.7 for MIND-SSC) and never ranks below fourth, whereas the strongest baseline changes with the dataset (Section 4.2).

"The test sets are small (18/4/14/47 pairs), and no statistical significance tests are reported."

We agree the test sets are small due to the annotation cost. These are well-established public benchmarks covering four modality pairs (T1–T2 MRI, MRI–CT, MRI–ultrasound, and CT–ultrasound). As suggested, Table 1 now marks baselines significantly worse (†) or better (∗) than the best SigMI variant per solver and dataset: two-sided exact Wilcoxon signed-rank tests paired per annotation, Holm-corrected over the six baselines per cell, α=0.05\alpha = 0.05. With only 4 Abdomen test pairs, a case-level exact test cannot reach p<0.05p < 0.05, hence annotation-level pairing (labels, landmarks, or kidneys), applied to all datasets. Of the 72 baseline–setting comparisons, 51 are significantly worse and 4 significantly better.

(continued below)

AuthorsOpenReview ↗

Author Response (Part 2)

"Several ConvexAdam results need discussion. On RESECT, four baselines perform substantially worse than the unregistered images (18.8–24.6 vs. 5.60 mm). On TRUSTED, every method, including SigMI, scores below the unregistered baseline (84.74)." and "Should these settings count toward the claimed "8 of 12" wins?"

RESECT. These results reflect limitations of the existing measures rather than our configuration. We use the convex-stage parameters that ConvexAdam's authors used for the CuRIOUS challenge, whose public data is RESECT, and, as in the ReMIND2Reg 2025 ConvexAdam baseline, we disable the Adam stage on ultrasound (Appendix C.3). Nothing is then tuned per measure: each displacement is the best offset in the measure's local cost volume, followed by smoothing. In this setting, all six baselines end above the initial 5.60 mm TRE (from 6.06 mm for MIND-SSC to 24.61 mm for Mattes MI), and only SigMI improves on it (3.52 to 5.50 mm). The four measures you mention also barely improve with the other solvers (4.34 to 5.29 mm with MultiresSVF and FireANTs SyN), where tuned regularization keeps them close to the initial alignment. Section 4.2 now explains this, and our limitations note that tuning the search range per measure could help some baselines.

TRUSTED. Thank you, this was an error on our side. We had reused RESECT's convex-stage spatial parameters in voxels, although TRUSTED's voxels are about twice as large. With the grid spacing and displacement half-width rescaled to 2 and 3 voxels (Appendix C.3), and all TRUSTED ConvexAdam results rerun with the same parameters for every measure, SigMI-R9 (85.81), SigMI-R17 (85.24), and MIND-SSC (85.16) now improve on the unregistered 84.74 Dice.

Counting. We believe these settings should count: they apply the same fixed parameters to every measure, which makes them the most direct comparison of the measures. Our conclusion does not depend on them, though. Excluding both ConvexAdam ultrasound settings, SigMI-R9 still has the best average rank over the remaining 10 (2.0, against 2.3 for squared LNCC and 2.8 for MIND-SSC).

"Were the Section 4.3 variants (R9 for LuMIR and R17 for RESECT) selected using validation or test data? What was the rationale for these choices?"

Thank you for raising this: the choice relies on validation data. The receptive field was chosen with the best validation performance using MultiresSVF: SigMI-R9 on LuMIR and SigMI-R17 on RESECT. Section 4.4 (Section 4.3 in the submission) now describes this protocol. Our rationale is to compare our measure, plugged into an off-the-shelf solver, against methods whose measure and solver were designed together, and whose hyperparameters were tuned with a larger budget (50 trials plus 3 seeds for the top 3 candidates, against 20 trials for SigMI; Appendix C.5).

We hope the repositioned theory, the corrected TRUSTED setup, and the per-variant reporting address your concerns, and we would be glad to discuss any remaining questions.

Reviewer_3RcMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper proposes the SIGMI estimator for pointwise mutual information (PMI) for medical image registration, which is the SigLIP score but subtracted a class prior correction constant that depends on the batch size. The paper theoretically formulates the SIGMI estimator and provides some theoretical bound of the error between true PMI and the SIGMI estimator. The paper then instantiate the proposed estimator on a 3D convolutional network trained using synthetic images from five brain templates. For validation, the paper then evaluates two variants of the proposed model against baselines on real data across different anatomies, modalities and registration solvers. Additionally, the paper then evaluates the proposed estimator against full registration methods on two datasets, using the best variant of the proposed estimator on each dataset. To quantify the accuracy of the proposed estimator, the paper also evaluates the accuracy of SigMI vs other baselines (e.g. NWJ, InfoNCE, SigLIP) on the BMI v2 benchmark, whose PMI ground truth is known.

Critical strengths
  1. The theory looks correct and the paper engineering contribution is good: The paper implements a 3D convolution model that predict pointwise mutual information only using synthetic training images on one organ templates and shows it to be able to perform competitively compared to other baselines across different anatomies, modalities and solvers. The author also implemented a custom CUDA kernel for parallel cost volume construction for the solvers.

  2. The paper has elaborate evaluate scheme: The paper has many qualitative and quantitative evaluations. The evaluation scheme of the paper is elaborate across different settings and against different baselines. This provides a broad empirical assessment of the proposed estimator.

Critical weaknesses
  1. Limited theoretical novelty: One of the paper contribution claims is: “(i) show that the SigLIP objective is a two-step MI estimator that recovers pointwise MI under any class prior;” However, the paper’s main equation (Eq. 8) is a special case of Eq. (8) in “DEMI: DISCRIMINATIVE ESTIMATOR OF MUTUAL INFORMATION” by Ruizhi Liao et. al. (https://arxiv.org/pdf/2010.01766), which the paper cites. DEMI allows a generic binary classifier and arbitrary ratio while SigMI sets the binary classifier to SigLIP and sets the ratio pipi to be dependent to the batch size (pipi=1/B). Addtionally, mutual information for SigLIP and CLIP-like models have been studied in “CLIP-like Model as a Foundational Density Ratio Estimator” by Fumiya Uchiyama et. al. (published at CVPR, version with appendix at https://arxiv.org/pdf/2506.22881). Eq. 8 of SigMI follow directly by taking logarithm of Eq. 28 in the appendix of Fumiya Uchiyama et. al. The work by Fumiya Uchiyama et. al. should be cited. Regarding the other theoretical contribution claim: the other theoretical derivation looks correct however it is thin. Consequentially, I find the theoretical contribution limited.

  2. Empirically, it is unclear what is the effect of the proposed SigMI estimator: First, regarding Table 1 in the main text, SigMI variants outperform baselines very marginally in many settings. In many cases, the baseline Squared LNCC perform better or only a tiny bit worse than the SigMI variants. Therefore, I find some paired statistical comparisons or multi seeds training and evaluations is needed to establish SigMI outperformance compared to the baselines. Second, regarding Table 3,4,5 in section C of the Appendix, InfoNCE outperforms SigMI in most tasks compared in term of PMI estimation error. Because the InfoNCE and SigLIP/SigMI are very closely related, it would help to add an InfoNCE trained model with the same network architecture and training settings as a baseline to the Table 1 in the main text. This would help establishing whether SigMI has any advantage over other contrastive learning methods. Additionally, the experiments don’t adequately disentangle the contribution of factors such as the original contrastive sigmoid loss, or the network architecture choices. Therefore, I find the experiment results in the paper inadequate to establish clear advantage of the proposed SigMI method compared existing approaches.

Questions for authors

Could the authors clarify the contribution of different components of the SigMI estimators to the performance gain in the paper? For example, is it due to the sigmoid loss, network architecture, training procedure choices, etc...?

Additionally, could the authors compare SigMI registration accuracy to another contrastive loss trained model (e.g. InfoNCE) with otherwise identical settings?

2 comments
AuthorsOpenReview ↗

Author Response (Part 1)

Thank you for your careful review, and for noting that the theory is correct, that the engineering contribution is good, and that the evaluation across anatomies, modalities, and solvers is elaborate. Your comments led directly to the paired significance tests in Table 1 and to the matched InfoNCE comparison in the new Section 4.3. Revised text is in blue.

"the paper's main equation (Eq. 8) is a special case of Eq. (8) in [DEMI] [...] Eq. 8 of SigMI follow directly by taking logarithm of Eq. 28 in the appendix of Fumiya Uchiyama et. al. The work [...] should be cited."

We agree, and thank you for pointing us to recent work from Uchiyama et al. The revision no longer presents this identity as ours: Theorem 2 is now stated "as in" DEMI (Liao et al., 2020) and related work (Gutmann & Hyvärinen, 2012; Levy & Goldberg, 2014; Tsai et al., 2020), and Uchiyama et al. are cited at Contribution (ii) and Corollary 4.

"the other theoretical derivation looks correct however it is thin."

Our theoretical contribution lies in what this identity alone does not establish:

  1. A finite-capacity, pointwise error bound in SigLIP's regime. DEMI's analysis is asymptotic (its authors "leave theoretical analysis [...] for future work"), Tsai et al. bound the global MI estimate, and Uchiyama et al. assume ideal modeling. Using the Bregman form of the excess risk (Theorem 5; Reid & Williamson, 2010), Corollary 6 bounds the mean squared PMI error under the joint by the excess classification risk, with an explicit 1/π1/\pi factor. This factor is the price of SigLIP's extreme imbalance (π=1/32\pi = 1/32 in our models, whereas DEMI's experiments use α∈{0.25,0.5,0.75}\alpha \in \{0.25, 0.5, 0.75\}).
  2. An information-theoretic reading of SigLIP's temperature and bias (Section 3.3). Estimates lie in [b~−t,b~+t][\tilde b - t, \tilde b + t], and since PMI's range is not centered at zero, the bias must be learned. Our models confirm this, with learned centers b~=−2.35\tilde b = -2.35, −3.22-3.22, and −4.10-4.10 nats (Table 3). The analysis also explains Zhai et al.'s (2023) own measurements: in their Fig. 6, randomly masking negatives at a fixed batch of 16k raises the learned bias by about 9 nats as the ratio goes from 1:16k to 1:1.6, while the prior-corrected b~=b+log⁡ν\tilde b = b + \log \nu stays near −2.8-2.8 nats at every ratio (read off their figure). The class prior moves bb.
  3. Validation of calibration against ground-truth PMI (Section 4.1). On 13 BMI tasks with 10 seeds, the uncorrected SigLIP error exceeds 5.5 nats at B=256B = 256, whereas SigMI stays within 0.09 nats of InfoNCE (see below). Uchiyama et al. use the ratio up to a constant or normalize it over a sample set, and their ground-truth toy experiment found SigLIP's ratios less accurate than CLIP's (their supplementary Table 3).
  4. A dense dot product estimator. DEMI and Tsai et al. score each pair with a single network that takes the two samples concatenated, i.e. one forward pass per patch pair. Our dual-encoder form of Eq. 9 instead gives local PMI everywhere as a dot product after one forward pass per image, which is what makes it usable for registration.

To our knowledge, the error bound and the range analysis are new for SigLIP. Finally, we turn the estimator into a dense similarity measure and validate it on a real-world application, multimodal medical image registration.

"regarding Table 3,4,5 [...] InfoNCE outperforms SigMI in most tasks compared in term of PMI estimation error."

You are right that InfoNCE's error is lower on most synthetic MI tasks (now Tables 4–6). We clarify that InfoNCE's PMI estimate is only defined after normalizing its score over a reference set, for which we gave it the full test set of 2,500 samples (Appendix C.1), while SigMI scores each pair on its own. With this advantage, InfoNCE's error is lower by at most 0.09 nats at B=256B = 256, and SigMI's is lower on all three 25-dimensional tasks. With fewer than about 32 reference samples, InfoNCE's estimate of a given pair varies by several tenths of a nat across reference sets (Figure 7). Importantly, image registration scores pairs one at a time, without a reference set, which is the setting SigMI is designed for.

(continued below)

AuthorsOpenReview ↗

Author Response (Part 2)

"it would help to add an InfoNCE trained model with the same network architecture and training settings as a baseline [...]" and "could the authors compare SigMI registration accuracy to another contrastive loss trained model (e.g. InfoNCE) with otherwise identical settings?"

As suggested, we trained InfoNCE models that differ from SigMI only in the objective (same architecture, synthetic data, batch construction, B=32B = 32, and hyperparameter search) at all three receptive fields. Results are reported in the new Section 4.3 (Table 2, Figure 5). For ease of reference, significant wins over the 12 dataset–solver settings (paired tests as in Table 1) are:

Receptive fieldSigMI significantly betterInfoNCE significantly better
R94 / 124 / 12
R178 / 120 / 12
R3312 / 120 / 12

At R9 the objectives are on par. SigMI's advantage grows with the receptive field and smaller batch sizes. Retraining both R17 models with BB from 4 to 64 shows that small batches hurt InfoNCE more than SigMI (Figure 5), consistent with Zhai et al. (2023) at much larger batch sizes. Taking each objective's best receptive field per setting, as Table 1 does for SigMI, SigMI has the better mean in 10 of 12 settings.

The two scores also differ in meaning: InfoNCE identifies PMI only up to a normalizer that depends on the conditioning sample (Uchiyama et al., 2026, Eq. 1), whereas SigMI's score is a PMI estimate in nats obtained with a dot product (Figure 4).

"SigMI variants outperform baselines very marginally in many settings [...] Squared LNCC perform better or only a tiny bit worse [...] some paired statistical comparisons or multi seeds training and evaluations is needed"

As suggested, Table 1 now marks baselines significantly worse (†) or better (∗) than the best SigMI variant per solver and dataset: two-sided exact Wilcoxon signed-rank tests paired per annotation, Holm-corrected over the six baselines per cell, α=0.05\alpha = 0.05. With only 4 Abdomen test pairs, a case-level exact test cannot reach p<0.05p < 0.05, hence annotation-level pairing, applied to all datasets. Of the 72 baseline–setting comparisons, 51 are significantly worse and 4 significantly better, highlighting the effectiveness of SigMI.

For Squared LNCC, the outcome depends on the modality pair. LNCC assumes a locally affine intensity relation, which roughly holds on LuMIR (two thirds of its pairs are T1–T1). With ultrasound, where it does not, LNCC is significantly worse in 5 of 6 settings, by up to 15.3 mm TRE (ConvexAdam on RESECT). SigMI, in contrast, handles all four modality pairs (T1–T2 MRI, MRI–CT, MRI–ultrasound, and CT–ultrasound) with one estimator per receptive field. While the strongest baseline changes with the dataset (LNCC on LuMIR and most TRUSTED settings, MIND-SSC on Abdomen and RESECT), SigMI-R9 has the best average rank over the 12 settings (1.8, against 2.8 for LNCC and 2.7 for MIND-SSC) and never ranks below fourth.

"the experiments don't adequately disentangle the contribution of factors such as the original contrastive sigmoid loss, or the network architecture choices." and "Could the authors clarify the contribution of different components [...]?"

We agree this was missing. The revision isolates our two main design choices, all else fixed: the training objective, through the InfoNCE comparison above (Section 4.3), and the field-of-view masking, by retraining all three variants without it (Appendix C.6, Table 7). Masking never significantly hurts, and it significantly improves 2 and 3 of the 4 datasets at R17 and R33. Like the objective, it matters more as the receptive field grows, consistent with larger patches making negatives easier to separate. The encoder is a standard residual CNN whose main design parameter, the receptive field, is varied in Table 1.

We hope the repositioned theory, the paired tests, and the matched InfoNCE comparison address your two main concerns, and we would be glad to discuss any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive reviews. All changes in the revised manuscript are marked in blue.

We are glad the reviewers found our analysis of SigLIP as a PMI estimator rigorous, insightful, and correct [stLv, 3RcM], with practical implications [KF7S]; the fully convolutional design sound and practical [stLv]; the transfer from synthetic data across anatomies, modalities, and solvers robust [KF7S, 3RcM]; and the paper clear [3RcM, KF7S, stLv].

The main changes are:

  1. Positioning of the theory [3RcM, KF7S]. The prior-corrected logit is now credited to prior work, including DEMI and Uchiyama et al. (Theorem 2, Corollary 4). We claim what it does not provide on its own: a finite-capacity pointwise error bound (Corollary 6), the interpretation of SigLIP's temperature and bias (Section 3.3), and a dense estimator for registration.
  2. Matched InfoNCE comparison [3RcM, KF7S]. In the new Section 4.3, SigMI's advantage grows with the receptive field and as the batch size shrinks.
  3. Significance tests [3RcM, KF7S]. In Table 1, 51 of 72 baseline comparisons are significantly worse than SigMI and 4 significantly better.
  4. ConvexAdam [KF7S]. We fixed the TRUSTED setup and explain the RESECT results (Section 4.2, Appendix C.3).
  5. Reporting [KF7S]. Win counts are now given per variant, and Section 4.4 states how its variants were selected.
  6. Ablations [stLv, 3RcM]. Field-of-view masking (Appendix C.6) and a larger receptive field, SigMI-R33 (Table 1).

All other points are addressed in the individual responses. We welcome any further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.