acceptodds
Under review as a conference paper at ICLR 2027

Contrast Transformation-Guided Multimodal Medical Image Registration

Abstract

Multi-modal medical image registration, especially MRI/CT, is challenging due to inconsistent appearance and large nonlinear deformations. Methods based on hand-crafted metrics are noise-sensitive and require expensive iterative optimization, while deep models still struggle to separate modality-induced intensity differences from true misalignment. Recent translation-based methods normalize appearance before registration, but weak coupling between synthesis and alignment and possible geometric distortion can introduce spurious deformations. We propose an end-to-end framework with a contrast transformation branch that aligns gray-level distributions, mapping CT to MR-like contrast without changing anatomy, directly guided by registration loss. On two public CT-MR datasets, our approach achieves state-of-the-art performance, surpassing existing methods by 7.34% and 4.29%. The proposed contrast transformation network improves seven established models, with DSC gains up to 11.9% and 8.0%. These results demonstrate state-of-the-art accuracy, strong generalization, and easy integration into dual-stream registration frameworks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multi-modal image registration, deformable image registration, CT-MR registration, contrast transformation, medical image analysis, anatomy preservation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2fP3OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an end-to-end framework for unsupervised CT–MR registration using a histogram-conditioned contrast transformation network. The transformed CT, original CT, and target MR are processed by a shared encoder, with dynamic mixing of the two CT feature streams before deformation estimation. The transformation and registration networks are jointly optimized using image similarity, histogram matching, content preservation, and deformation smoothness objectives. Experiments on two abdominal CT–MR settings demonstrate improved registration performance across seven backbone architectures.

Critical strengths
  1. Consistent improvements across multiple registration architectures. Adding TransNet improves DSC for all seven evaluated backbones on both datasets. This is a strong empirical evidence in the paper and supports the module’s practical utility beyond a single selected architecture.

  2. A well-motivated integration of contrast transformation and registration. Using the target MR histogram rather than the spatial MR image is a reasonable design choice for limiting the direct introduction of target-specific spatial information.

  3. Evaluation considers both alignment accuracy and deformation regularity. The main RDP-based configuration improves DSC, HD95, and ASSD on both datasets while reporting lower non-positive Jacobian percentages than the RDP baseline.

Critical weaknesses

1. The advantage of histogram-conditioned transformation over image synthesis strategies is insufficiently established.

The paper motivates its method by contrasting it with conventional image synthesis, but does not provide a controlled comparison between histogram-conditioned TransNet and a conventional CT-to-MR synthesis module within the same registration framework. INNReg is evaluated, but comparing two complete pipelines does not isolate the contribution of the transformation strategy. Table 3 shows that adding histogram conditioning improves DSC from 0.649 to 0.658; however, this only establishes an incremental benefit within the chosen architecture. A direct histogram-matching baseline is also absent. Consequently, it remains unclear whether the proposed learned transformation offers a meaningful advantage over either conventional synthesis or a much simpler intensity-mapping operation。

2. The intended role of TransCT is not sufficiently validated through independent quantitative evaluation.

Figures 3–4 provide checkerboard visualizations and a sampled gradient-profile comparison, so the paper does offer some evaluation of TransCT itself. However, these examples do not establish, across the test set, that TransCT simultaneously preserves source anatomy and reduces the appearance discrepancy with the target MR relative to the original CT. End-to-end training is not inherently problematic, but improved downstream registration alone does not isolate the claimed contrast-normalization mechanism from other changes introduced by the auxiliary branch, feature mixing, and additional losses. The reported quantitative metrics primarily assess final registration and deformation regularity, rather than the intermediate transformation. Independent evaluation is therefore needed to support the claim that the method improves registration specifically by reducing the modality gap without introducing anatomical distortions.

3. The state-of-the-art claims are not adequately supported by the comparison scope.

The experiments do not include comparisons with recent methods such as OTMorph and TransReg, which share conceptual similarities with the proposed approach. As a result, the proposed method’s performance relative to these alternatives remains unclear. TransReg is already cited in the manuscript but is not experimentally evaluated. Furthermore, the main RDP+TransNet configuration achieves 0.658 DSC on Learn2Reg, whereas the unmodified ModeT and PCNet baselines in Table 2 achieve 0.688 and 0.692, respectively. Although ModeT+TransNet reaches 0.704, the manuscript should distinguish the incremental benefit of the module from the overall ranking of a particular instantiated model. These issues do not invalidate the reported within-backbone improvements, but they weaken the broader claim of superiority over existing approaches.

Questions for authors
  1. How does histogram-conditioned TransNet compare with conventional CT-to-MR synthesis under a matched registration setup? This would help determine whether the proposed transformation strategy itself offers an advantage, rather than differences between complete registration pipelines.

  2. How much improvement requires a learned transformation rather than direct histogram matching? Could you replace TransNet with direct CT-to-target-MR histogram matching and retrain the registration network? Please also clarify whether the “w/o MR-hist” variant removes only histogram conditioning or additionally removes the histogram-based EMD loss.

  3. Can you quantitatively validate TransCT independently of the final registration output? In particular, does TransCT exhibit a smaller appearance discrepancy with the target MR than the original CT, while retaining the source CT’s anatomical locations and boundaries?

  4. Can you establish the method’s relative performance against OTMorph and TransReg and reconcile the SOTA claims across Tables 1 and 2?

  5. How well does TransCT resolve tissue-specific contrast differences, particularly in bone regions? In the examples shown in Figure 3, cortical bone appears to remain relatively bright in TransCT, resembling the original CT rather than the low-signal appearance in the MR images. This suggests that important tissue-specific contrast discrepancies may remain unresolved despite global histogram alignment.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 2fP3 (1)

Thank you for focusing on the evidence needed to distinguish the transformation mechanism from the overall pipeline. We agree that the strongest test combines comparisons with a common registration backbone and evaluation of the intermediate image, while keeping the claims within the scope of each measurement. We group overlapping weaknesses and questions below to avoid repeating the same result.


W1, Q1, and Q2. Synthesis and direct histogram matching with the same registration backbone.

Response.{\color{green}\mathbf{Response.}} We compared raw CT, direct CT-to-target-MR histogram matching, a CycleGAN-based transformation, and TransNet using RDP as the common registration backbone. For direct matching, the registration network was retrained with the matched images. For CycleGAN, the synthesized images replace TransCT as the auxiliary image input.

Input/transformation configuration with RDPDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Raw CT0.59712.0994.6200.052
Direct histogram matching0.60012.2004.7100.051
CycleGAN-based transformation0.61012.0404.5500.058
TransNet0.65811.7404.3240.030

On Learn2Reg, TransNet exceeds the tested direct-matching and CycleGAN configurations by 0.058 and 0.048 DSC, respectively, and has lower HD95, ASSD, and NJD. Direct histogram matching only changes DSC from 0.597 to 0.600. This is evidence that simple matching of marginal intensity distributions is insufficient to reproduce the gain in this setting.

These comparisons hold the registration backbone fixed, but compare complete transformation configurations with different training objectives. They support an advantage over the tested alternatives, not an isolated causal attribution to histogram conditioning or a claim that all synthesis approaches are inferior. We will specify the feature paths, registration losses, transformer training/freeze policy, and training budgets for each configuration so that the extent of matching is explicit. To answer the ablation question directly, the “w/o MR-hist” variant removes only the histogram input and retains EMD. The additional ablations are:

ConfigurationMR histogram inputEMDMIDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Ours Full configuration✓✓✓0.65811.7404.3240.030
No MR histogram input×✓✓0.64912.0634.5770.034
No EMD✓×✓0.64212.0204.5400.039
No MI✓✓×0.65111.9504.4800.035
Neither MR histogram input nor EMD××✓0.64112.0554.5600.038

The results separate conditioning from histogram-based supervision: conditioning contributes an additional 0.009 DSC when EMD is retained, while removing EMD with conditioning retained decreases DSC by 0.016. They support the tested combination without implying that all of its gains arise from conditioning alone.

AuthorsOpenReview ↗

Response to Reviewer 2fP3 (2)

W2 and Q3. What the pre-registration evaluation establishes.

Response.{\color{green}\mathbf{Response.}} We agree that final registration accuracy and deformation regularity cannot, by themselves, validate the intermediate transformation. We therefore measured TransCT before applying the predicted deformation on the Learn2Reg test set:

Image before registrationWhole‑volume Wasserstein‑1 to MR ↓Detected‑edge ASSD to CT (mm) ↓Detected‑edge HD95 to CT (mm) ↓
Original CT0.18240.000.00
Direct histogram matching0.009110.3242.66
CycleGAN0.046528.5379.16
TransCT0.03161.245.34

The reduction from 0.1824 to 0.0316 shows a smaller discrepancy in the global intensity distribution relative to the original CT. Notably, direct histogram matching achieves a smaller Wasserstein-1 distance than TransNet but substantially lower downstream DSC, 0.600 versus 0.658. Thus, global histogram agreement alone does not explain the registration results.

We also qualify the structural interpretation. Edge-set distances depend on contrast and edge-extraction thresholds: pointwise histogram matching can change those sets without moving voxel coordinates. The table therefore reports detected-edge agreement under the implemented measurement, not anatomical displacement. Furthermore, Wasserstein-1 is related to our EMD training objective, so this is an evaluation outside the final registration output, not an independent validation of every proposed mechanism.

The histogram input omits the spatial arrangement of the target MR, the MI term encourages dependence on the source CT, and feature mixing retains an original-image pathway. These are design choices intended to reduce transformation-induced errors; none guarantees anatomical invariance. We will remove the categorical “without changing anatomy” claim and expand the qualitative examples using consistent sampling across all held-out cases. The present evidence supports useful contrast transformation with favorable observed edge agreement, while anatomical preservation remains a qualified empirical claim.

AuthorsOpenReview ↗

Response to Reviewer 2fP3 (3)

W3 and Q4. OTMorph, TransReg, and the overall ranking.

Response.{\color{green}\mathbf{Response.}} We agree that the main RDP configuration and the best augmented backbone must not be conflated. The additional OTMorph comparison is:

SettingMethodDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Learn2RegOTMorph0.63012.2614.6960.050
Learn2RegRDP + TransNet0.65811.7404.3240.030
AbdomenCT-1K CT/AMOS MROTMorph0.50311.4904.3120.075
AbdomenCT-1K CT/AMOS MRRDP + TransNet0.58310.1923.6460.039

These results favor RDP+TransNet over the evaluated OTMorph implementation under our experimental settings. They do not establish a universal ranking across datasets or implementations.

Our current experiments do not include a verified implementation of TransReg, so we cannot claim superiority over it. We will explicitly identify this missing comparison. We will also discuss its conceptual overlap: TransReg combines original-pair and translated-pair registration pathways, whereas our module mixes features before an existing deformation decoder. This distinction describes the design; it is not a substitute for a head-to-head result.

For Learn2Reg, unmodified ModeT and PCNet score 0.688 and 0.692 DSC, respectively, both above RDP + TransNet at 0.658. Adding TransNet improves them to 0.704 and 0.703. In the second setting, unmodified CorrMLP also has better HD95/ASSD than RDP + TransNet. We will include these stronger configurations in the consolidated comparison and remove the blanket SOTA claim rather than transfer it to whichever configuration has the largest DSC. The supported finding is consistent within-backbone improvement, with ModeT + TransNet achieving the highest Learn2Reg DSC among our evaluated configurations.


Q5. Residual tissue-specific differences, particularly cortical bone.

Response.{\color{green}\mathbf{Response.}} Your observation is valid: globally similar histograms do not guarantee that corresponding tissues acquire the same contrast, and the examples indicate that cortical bone can remain relatively bright. A global histogram specifies intensity frequencies but does not assign those intensities to tissue identities. Our current evaluation therefore does not establish tissue-specific CT-to-MR equivalence or successful normalization of cortical bone.

The positive organ-registration results show that the representation can be useful despite residual contrast differences. They do not prove that bone contrast is correct, that bone boundaries are preserved, or that bone-driven registration would improve. We will explicitly state this limitation and describe TransCT as a registration-oriented contrast transformation rather than an anatomically and radiologically equivalent MR synthesis. Tissue-specific assessment is needed before making a stronger claim about bone or other structures outside the reported organ evaluation.

Reviewer_czjnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a contrast transformation branch, integrated with a dual-encoder registration backbone, for multimodal medical image registration. The module uses the MR intensity histogram to transform CT contrast and then combines the original and transformed CT features via a dynamic feature-mixing strategy before registration. The method is evaluated on two abdominal MR-CT registration settings and reports strong quantitative performance across several registration backbones.

Critical strengths
  1. The proposed transformation branch is designed as a plug-and-play module and is evaluated with several dual-stream registration backbones, demonstrating good architectural flexibility.
  2. The paper reports strong quantitative improvements across the evaluated settings and backbones.
Critical weaknesses

Weaknesses:

  1. Technical novelty appears to be limited. The core contrast-transformation mechanism is closely related to HueNet (Avi-Aharon et al., 2023), including histogram conditioning and the EMD/MI objectives. The paper should more clearly distinguish its new contributions from prior work.
  2. The conceptual description in the Introduction does not fully match the actual architecture. The paper suggests that contrast normalisation precedes shared feature extraction, whereas the original CT is also processed by the shared encoder and directly contributes to registration through dynamic feature mixing.
  3. The ablation study is insufficient. Key components, including dynamic feature mixing and the loss terms, are not validated. In addition, comparisons with transformed-feature-only, fixed-feature mixing, a two-stage transformation-registration pipeline, and shuffled MR-histogram conditioning would be helpful.
  4. The dataset construction and experimental protocol are not clear. For Learn2Reg, it is unclear how the 200 randomly constructed unpaired pairs are used. For the AbdomenCT-1K-based experiment, the source of the reported 50 labelled MR volumes is not specified. The exact train/test pairing protocol should also be clarified.
  5. The implementation of INNReg is unclear. INNReg was originally proposed as a 2D registration framework, while this paper performs 3D registration.
  6. The evaluation is relatively limited. Both experiments focus on abdominal MR-CT registration; the test sets are small, with only 5 or 6 testing samples.
  7. The computational overhead is not reported.

Minor Weaknesses:

  1. AC-CAR is not a translation-based method, correct?
  2. The statement that “the use of shared weights imposes a common latent space” is overstated. Weight sharing alone does not guarantee modality-invariant or aligned feature representations.
  3. The Related Work is somewhat repetitive and could be reorganised more clearly, with better coverage of closely related histogram/intensity-transformation and appearance-normalisation methods.
Questions for authors

See my comments on weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer czjn (1)

Thank you for identifying the novelty, protocol, and ablation issues. We agree that the paper must distinguish the established histogram-transformation components from the proposed integration, and that architectural flexibility should not be confused with broad external validation.


W1. Relationship to HueNet and the scope of the contribution.

Response.{\color{green}\mathbf{Response.}} We agree that histogram conditioning and the EMD/MI formulation are closely related to HueNet. We will explicitly credit this foundation and do not claim a new histogram loss or a new general-purpose image-translation principle [1].

Our contribution concerns how this transformation is integrated into deformation estimation. The CT-derived transformation is optimized together with registration; the original CT and transformed CT features are both available to the registration decoder through schedule mixing; and the same feature extractor processes CT, TransCT, and MR while the existing deformation decoder is retained. Thus, the transformed image is an intermediate representation for correspondence estimation, rather than an endpoint evaluated primarily for visual synthesis quality. Neither end-to-end training nor using original and synthesized information is, by itself, claimed as a new idea.

The empirical basis for this integration is improvement within seven different registration backbones in both evaluated settings, together with the mixing, loss, and training-schedule comparisons below. The fixed-total-epoch sequential results provide supporting evidence for the chosen training configuration, but do not establish superiority over a fully optimized HueNet-based registration pipeline. We will present the contribution at this supported level.

Reference: [1] Avi-Aharon et al. Differentiable Histogram Loss Functions for Intensity-based Image-to-Image Translation.


W2. Actual information flow and the original CT path.

Response.{\color{green}\mathbf{Response.}} You are correct that the Introduction describes an overly simple “normalize first, then encode” sequence. With E denoting the shared encoder, T the transformation decoder, and H the target histogram embedding, the information flow is:

  • F_CT = E(CT).
  • TransCT = T(F_CT, H(MR)).
  • At each pyramid level, F_MixCT = λ F_CT + (1-λ) F_TransCT.
  • The registration decoder predicts the deformation from F_MixCT and F_MR, and that deformation is applied to the original CT.

The original CT features therefore serve two purposes: producing TransCT and directly contributing to registration through the mixture. We will make both paths explicit in the Introduction and Methods. “Dual-stream” refers to the two inputs to deformation estimation; there are three image passes through the shared encoder. The MR histogram excludes the target image’s spatial arrangement from the direct transformation input, but this architectural choice is not a mathematical guarantee against source-structure changes.

AuthorsOpenReview ↗

Response to Reviewer czjn (2)

W3. Mixing, transformation losses, training schedules, and shuffled histograms.

Response.{\color{green}\mathbf{Response.}} Feature mixing. The Learn2Reg results distinguish training-time mixing from a test-time intervention:

Training scheduleInference λDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Fixed λ = 0.50.50.64312.0704.5220.037
Fixed λ = 0.10.10.63612.2404.6630.044
Anneal λ from 1 to 000.64711.9864.4760.039
Anneal λ from 1 to 0.1; TransCT features only at inference00.65011.9134.4580.036
Anneal λ from 1 to 0.10.10.65811.7404.3240.030

The default schedule performs best among these tested variants. The inference-only experiment tests reliance on the original-feature path after training with that path; it is not equivalent to training with transformed features alone throughout. The anneal-to-zero experiment also retains original features during earlier training. We do not present either experiment as a fully transformed-feature-only training control.

Transformation losses and conditioning. We separately removed the MR histogram input, the EMD loss, and the MI loss while retaining the registration similarity and smoothness terms:

ConfigurationMR histogram inputEMDMIDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Full configuration✓✓✓0.65811.7404.3240.030
No MR histogram input×✓✓0.64912.0634.5770.034
No EMD✓×✓0.64212.0204.5400.039
No MI✓✓×0.65111.9504.4800.035
Neither MR histogram input nor EMD××✓0.64112.0554.5600.038

In particular, “w/o MR-hist” removes only conditioning, not the EMD loss. Removing conditioning decreases DSC by 0.009; removing EMD decreases it by 0.016. These are empirical contributions within this configuration, not evidence that either component is universally indispensable. The MI ablation assesses downstream registration; it does not by itself quantify structural preservation by the intermediate image.

Sequential training. Registration-first and transformation-first schedules of 50 + 50 epochs obtain DSC 0.631 and 0.634, respectively, compared with 0.658 for 100 epochs of joint learning. Their corresponding HD95/ASSD/NJD values are 12.236/4.732/0.040 and 12.181/4.657/0.045, compared with 11.740/4.324/0.030 for joint training. These results favor the joint schedule under this total-epoch budget. We will specify the shared-encoder freezing policy, since matching epoch totals does not match the updates received by each component.

Shuffled histogram conditioning. We agree that this is a distinct control. Randomly constructing inter-subject CT-MR pairs does not implement it: each constructed pair still supplies the histogram of its actual registration target. A genuine shuffled-histogram control must keep the moving CT and fixed MR unchanged while supplying a different MR histogram to the conditioning branch. Our current results do not contain that control, so we do not claim that the specific target histogram is necessary, rather than useful modality-level intensity information. The no-conditioning ablation supports the contribution of the input pathway but does not resolve this more specific question.

AuthorsOpenReview ↗

Response to Reviewer czjn (3)

W4. Dataset provenance and pairing protocol.

Response.{\color{green}\mathbf{Response.}} Thank you for identifying the dataset-description error. AbdomenCT-1K is a CT dataset; the MR scans in our second setting come from AMOS. We will consistently name this experiment “AbdomenCT-1K CT/AMOS MR,” rather than describing MR scans as part of AbdomenCT-1K or labeling the entire experiment simply “AMOS.”

For Learn2Reg, we use a 3/5 train/test subdivision of eight paired cases available to our experiments. The 200 randomly constructed CT-MR combinations are drawn before training from the additional unpaired pools and remain fixed across experiments; they are used for training, not as 200 independent test cases. Our 3/5 subdivision is a study-specific split, not the official challenge train/test split, and the eight cases should not be described as the entire Learn2Reg paired dataset.

In the second setting, the protocol selects 30 annotated volumes per modality, allocating 24 per modality to training and six per modality to testing. The training set contains 224 fixed inter-subject CT-MR combinations, including 200 additional combinations constructed using the remaining 20 MR volumes and the CT training pool. The six held-out volumes per modality are combined to form 36 CT-MR test pairs. These counts distinguish individual volumes from CT–MR pairings and should not be conflated. Anatomical labels are used for evaluation rather than the unsupervised training loss.

We agree that these counts must be accompanied by the actual pair and split definitions. We will report the modality-specific scan identifiers, the construction of the primary and auxiliary training pools, the exact test pair list, evaluated organs and label mapping, and the model-selection rule. This will make the subject separation and the number of independent evaluation units explicit instead of relying on a statement that the data were randomly paired.


W5. INNReg adaptation to volumetric registration.

Response.{\color{green}\mathbf{Response.}} Our comparison is an adaptation, not the original 2D INNReg pipeline. We retain its invertible translator, apply it slice by slice, and stack the translated slices into a volume. A 3D VxmDense registration network then replaces its 2D registration network and predicts a volumetric displacement field.

We will label this row “INNReg-derived 3D adaptation” and report the slice direction, preprocessing, training objectives, and volumetric warping settings. The comparison evaluates this particular adaptation; it does not establish the performance of an official 3D INNReg implementation or guarantee the best possible adaptation of the method.


W6. Small test sets and limited evaluation scope

Response.{\color{green}\mathbf{Response.}} We agree. The two experiments cover abdominal CT–MR registration: the first uses five paired evaluation cases, while the second evaluates all 36 CT–MR pairings formed from six held-out volumes per modality. Testing seven backbones addresses architectural applicability, but cannot substitute for evaluation on more subjects or other anatomical regions. Similarly, three training seeds address run-to-run variation, not population-level uncertainty.

The three-seed mean DSC improves from 0.5980 ± 0.0046 to 0.6583 ± 0.0035 on Learn2Reg and from 0.5590 ± 0.0030 to 0.5840 ± 0.0036 on the AbdomenCT-1K CT / AMOS MR setting. These SDs are across run-level means. We will limit our conclusions to the evaluated abdominal settings, state the small-sample limitation explicitly, and distinguish repeated evaluation of the same subjects from additional independent evidence. We will also provide individual-case results so that improvements and failures are visible rather than represented only by aggregate means.

Reviewer_ZLSROpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses deformable CT → MR registration, where modality-induced intensity differences make it hard for the network to separate appearance change from true misalignment. The authors design a contrast transformation branch ("TransNet") that can be injected into any dual-stream pyramid registration network: the MR image is represented only by its intensity histogram (encoded by an MLP), which is fused with the CT features to decode an MR-like CT image (TransCT). The whole system is trained end-to-end and unsupervised with four equally weighted losses (LNCC on the warped TransCT, histogram EMD, mutual information with the original CT, and a smoothness term). This design is feasible and easy to use. The validation covers two abdominal CT–MR datasets, a comparison with six learning-based baselines (Table 1), integration into seven dual-stream backbones (Table 2), and a two-row ablation (Table 3). In my reading, what is actually learned is closer to a registration-driven, spatially-aware contrast normalization than a full modality translation, which I regard as a sensible and honest scope for the idea. Despite some noticeable issues, I think it is a good piece of work.

Critical strengths
  1. The writing is clear from the very beginning of the paper. It is easy to follow and understand.
  2. The design targets the real failure mode of translation-based registration, namely that a translator optimized for realism may alter anatomy and force the registration network to compensate. Because the decoder is untouched, the module is cheap to adopt in any dual-stream backbone. This matters to my assessment because the contribution is conceptual and reusable rather than a one-off architecture.
  3. Across seven backbones (convolutional, Transformer and MLP decoders) and two datasets, adding TransNet improves DSC in 14/14 cases, lowers ASSD in 14/14 and lowers NJD in 14/14, with HD95 improving in 13/14.
Critical weaknesses
  1. The paper claims state-of-the-art performance based on Table 1. However, Table 1 omits ModeT and PCNet on Learn2Reg (DSC 0.688 and 0.692 vs. 0.658 for “Ours”, and also better HD95/ASSD) and CorrMLP on AbdomenCT-1K (HD95 8.815 / ASSD 3.338 vs. 10.192 / 3.646), even though all three were trained by the authors under the same protocol in Table 2. Please explain why these were left out, add them to Table 1, and revise the SOTA statements in the abstract and Sec. 5 accordingly (note that ModeT + TransNet at 0.704 is in fact the best number in the paper). I regard this as a framing problem rather than a methodological one, but it must be fixed.
  2. The claim that TransNet changes contrast without changing anatomy is only shown qualitatively, and with very limited examples (Figures 3–4: two slices and one gradient profile). It is hard to know whether these are selected cases or whether they reflect the method in general. Please show more examples, and preferably also report one simple quantitative measure over the whole test set (for example, how much TransCT deviates from the original CT in structure, or how similar TransCT becomes to MR), so that the claim does not rest on a few figures alone.
  3. λ is a critical parameter. However, there is no discussion of how the annealing schedule affects the final results. What about setting λ to a fixed number, for example 0.5 or 0.1, or annealing to 0? Please also state the value of λ used at inference.
Questions for authors

Please regards all weakness as questions first. Here are additional ones:

  1. Can you report multi-seed results with standard deviations?
  2. Does end-to-end training matter? A two-stage version (train the backbone, freeze it, then train TransNet) would show whether the gain comes from joint training or just from the extra branch.
6 comments
AuthorsOpenReview ↗

Response to Reviewer ZLSR (1)

Thank you for the constructive review and for recognizing the value of a reusable registration module. We agree that the ranking, structural-preservation claim, and training choices require clearer evidence and more precise wording.


W1. Incomplete comparison table and overstated SOTA claims.

Response.{\color{green}\mathbf{Response.}} You are correct. Separating the backbone-integration experiment from the main comparison does not justify excluding stronger models from the overall ranking. We will include the relevant baseline and augmented configurations in a consolidated comparison and remove the unqualified SOTA statements from the abstract, Results, and Conclusion.

The Learn2Reg DSC results illustrate the distinction:

ConfigurationDSC
RDP0.597
RDP + TransNet0.658
ModelT0.688
ModelT + TransNet0.704
PCNet0.692
PCNet + TransNet0.703

Thus, RDP + TransNet improves its own backbone by 0.061 DSC, but does not outperform unmodified ModelT or PCNet on this dataset. ModelT + TransNet has the highest Learn2Reg DSC among the evaluated configurations; this is a configuration-specific results, not a claim of superiority over all existing methods. We also acknowledge that, in the second abdominal setting, unmodified CorrMLP has lower HD95 and ASSD than RDP + TransNet: 8.815 vs 10.192 mm and 3.338 vs 3.646 mm, respectively.

The supported general observation is the within-backbone improvement: DSC, ASSD, and NJD improve in all 14 backbone-setting combinations, and HD95 improves in 13 of 14. We will also correct the statement that every metric improves in every combination. These comparisons support the module’s usefulness across the tested architectures without obscuring the stronger underlying backbones.


W2. Quantitative assessment of TransCT before registration.

Response.{\color{green}\mathbf{Response.}} We agree that the selected slices and gradient profile are insufficient to establish anatomical preservation across the test set. We evaluated the intermediate images on the Learn2Reg test set before applying any deformation. The following measurements compare global intensity distributions with the target MR and detected edge sets with the source CT:

Image before registrationWhole-volume Wasserstein-1 ↓Edge ASSD (mm) ↓Edge HD95 (mm) ↓
Original CT0.18240.000.00
Direct histogram matching0.009110.3242.66
CycleGAN0.046528.5379.16
TransCT0.03161.245.34

TransCT has a smaller global intensity-distribution discrepancy than the original CT and lower detected-edge discrepancies than the other transformed images under the implemented evaluation. However, these two measurements have important limits. Wasserstein-1 assesses the marginal intensity distribution, which is also targeted by the EMD loss. Edge detection is contrast-dependent: even pointwise histogram matching can change the detected edge set without moving voxel coordinates. The edge distances therefore cannot be interpreted as a direct measurement of anatomical displacement or as proof that histogram matching geometrically deforms anatomy.

Accordingly, we will replace “without changing anatomy” with a qualified description of a transformation designed to retain source structural information. We will also expand the qualitative presentation to cover every held-out case using a consistent slice-selection rule. The present measurements support global contrast normalization and detected-edge agreement, but not a guarantee of anatomical invariance.

AuthorsOpenReview ↗

Response to Reviewer ZLSR (2)

W3. Feature-mixing schedule and inference value.

Response.{\color{green}\mathbf{Response.}} At inference, the original CT features have weight 0.1 and the transformed CT features have weight 0.9. We distinguish changing the training schedule from removing the original features only at inference:

Training scheduleInference λDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Fixed λ = 0.50.50.64312.0704.5220.037
Fixed λ = 0.10.10.63612.2404.6630.044
Anneal λ from 1 to 000.64711.9864.4760.039
Anneal λ from 1 to 0.1; remove original features only at inference00.65011.9134.4580.036
Anneal λ from 1 to 0.1 (Ours)0.10.65811.7404.3240.030

The comparison with fixed λ=0.1 keeps the inference mixture unchanged and supports the usefulness of the annealing schedule in this setting. Removing the original CT feature contribution only at inference changes DSC from 0.658 to 0.650; annealing to zero during training gives 0.647. These are different interventions and should not be conflated. The results support the chosen schedule among the tested alternatives, rather than establishing that it is universally optimal or that the residual original-feature path guarantees structural preservation.


Q1. Variation across training seeds.

Response.{\color{green}\mathbf{Response.}} We Compared RDP and RDP+TransNet using seeds 42, 43, and 44. The second setting combines Abdomen-CT-1K CT scans with AMOS MR scans; we use that explicit description instead of calling it simply “AMOS”.

SettingMethodSeed 42Seed 43Seed 44DSC mean ± sample SD
Learn2RegRDP0.5970.6030.5940.5980 ± 0.0046
Learn2RegRDP+TransNet0.6580.6550.6620.6583 ± 0.0035
AbdomenCT-1K CT / AMOS MRRDP0.5590.5560.5620.5590 ± 0.0030
AbdomenCT-1K CT / AMOS MRRDP+TransNet0.5830.5880.5810.5840 ± 0.0036

The mean absolute improvements are 0.0603 and 0.0250 DSC, respectively: 6.03 and 2.50 percentage points when DSC is expressed as a percentage. The gain is positive in each paired-seed comparison. The SD is calculated across the three run-level test means, with denominator n-1; it is not a standard deviation across subjects. These results address optimization variability on the existing splits, but do not increase the number of independent test subjects.


Q2. Joint versus sequential training.

Response.{\color{green}\mathbf{Response.}} We compared joint optimization with two sequential schedules using the same auxiliary branch:

Training strategyEpochsDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
Train registration backbone, freeze it, then train TransNet50 + 500.63112.2364.7320.040
Train TransNet, freeze it, then train registration backbone50 + 500.63412.1814.6570.045
Joint training1000.65811.7404.3240.030

Joint training exceeds the two sequential schedules by 0.027 and 0.024 DSC. This supports the practical benefit of the joint schedule under the reported epoch allocation. Equal total epochs do not imply equal update counts for every parameter, however, so these results do not isolate joint optimization as the sole cause of improvement. We will document which parameters are frozen in each stage, including the shared encoder, and describe these as schedule comparisons rather than a parameter-update-matched causal test.

I appreciate the authors’ response, which addresses most of my concerns. However, considering the other reviewers’ comments, and especially noting that my favorite part of this paper, the core contrast-transformation mechanism using histograms, is too similar to what HueNet (Avi-Aharon et al., 2023) proposed, I will not lower my score, but my assessment is now more inclined toward average (2.5).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised manuscript: completed controls and clarified conclusions

Thank you all for careful and detailed reviews. We have uploaded a revised manuscript incorporating the completed training controls in the main text (Section 5, Table 4) and the inference-time conditioning test. The revision also retains the clarifications to the architecture, dataset provenance, comparison configurations, computational cost, and attribution to prior work.

Training controls. With RDP on the evaluated Learn2Reg setting, the newly completed controls give:

ConfigurationDSC ↑HD95 (mm) ↓ASSD (mm) ↓NJD (%) ↓
RDP baseline0.59712.0994.6200.052
TransNet, detach0.65111.7664.4340.028
Sequential (50+50)0.63412.1814.6570.045
Sequential (100+100)0.64512.0304.5400.039
Joint (100; full)0.65811.7404.3240.030

In the sequential controls, transformation training precedes registration training, and the learned transformation is frozen before the latter stage. Extending both stages improves performance, but the evaluated joint configuration remains better on all four reported metrics than the longer sequential configuration. These are training-configuration comparisons; epoch totals do not imply matched computation.

The detach control blocks direct registration gradients to transformation-specific parameters while retaining EMD/MI and the task-trained shared encoder. The full model improves the three alignment metrics, whereas detach has slightly lower NJD (0.028% versus 0.030%). Thus, direct registration feedback contributes an increment in this comparison; it does not explain all improvement over the backbone, nor is this control fully independent of registration through the shared encoder.

Conditioning at inference. With the trained model and registration image pairs unchanged, shuffling the MR-histogram input changes DSC/HD95/ASSD from 0.658/11.740/4.324 to 0.654/12.297/4.609. The corresponding HD95 and ASSD increases are 0.557 and 0.285 mm. This provides evidence of sensitivity to the conditioning input in the evaluated setting. It does not show that conditioning accounts for the entire gain.

Contribution and limits. We explicitly credit HueNet for the histogram-based foundations. The contribution evaluated here is their registration-specific integration with direct task feedback, scheduled original/transformed feature mixing, and an unchanged deformation decoder; neither histogram conditioning nor joint training alone is claimed as a new principle. In the existing cross-backbone evaluation, mean DSC, ASSD, and NJD improve in all 14 backbone-setting combinations, and HD95 improves in 13. This supports applicability across the evaluated backbones without implying universal state-of-the-art performance.

The small abdominal test cohorts, absence of repeated-run uncertainty estimates for the new controls, and lack of an independent anatomical-invariance guarantee remain limitations. The controls use our transformation branch and are not an official HueNet reproduction. We hope the completed revisions and these explicit boundaries provide a clearer basis for the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.