acceptodds
Withdrawn from ICLR 2027

Geometry-Aware Spectral Conditioning for Biplanar X-Ray-to-CT Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstructing a 3D computed tomography (CT) volume from two orthogonal X-ray projections is severely ill-posed because projection measurements collapse depth information and superimpose anatomical structures. We propose Frequency-Enhanced Dual-Encoding ControlNet (FEDE-ControlNet), a framework that adapts a frozen 3D latent diffusion model for biplanar X-ray-to-CT reconstruction through complementary geometric and spectral conditioning. A geometric pathway back-projects features extracted by a frozen X-ray encoder into a shared volumetric grid, establishing explicit detector-to-volume correspondence. In parallel, a dual-encoding pathway integrates spatial features with global and patch-local Fourier representations through frequency-domain cross-attention, then geometrically lifts the fused features into multi-scale volumetric conditions. A trainable control branch incorporates both pathways through zero-initialized residual connections while keeping the pretrained denoiser and CT autoencoder frozen. To complement latent-space denoising, multi-level volumetric feature supervision aligns decoded predictions with reference CT volumes using hierarchical representations from an independent frozen encoder. This auxiliary supervision is applied only during training and adds no inference-time computation. Experiments demonstrate that the proposed FEDE-ControlNet achieves superior reconstruction fidelity and structural similarity compared with the evaluated state-of-the-art methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Frequency-Aware Representation Learning, Biplanar X-Ray-to-CT Reconstruction, Geometry-Aware Conditioning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9EYUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FEDE-ControlNet for reconstructing 3D CT volumes from two orthogonal X-ray projections. The framework adapts a pretrained and frozen 3D MedDiffusion model through a trainable ControlNet branch. It combines geometry-aware back-projection of X-ray features with a multi-scale spatial-frequency encoder (FEDE) to provide volumetric conditioning features. Additionally, a multi-level volumetric feature matching loss is introduced to supervise the reconstructed CT volumes during training.

Critical strengths

Controlled comparisons with the same pretrained generative prior. The paper includes a reasonably controlled comparison with C-MedDiffusion, which shares the same pretrained diffusion backbone and geometric back-projection conditioning. The factorial ablation also evaluates the individual and combined contributions of FEDE and volumetric feature supervision. These results provide evidence that the proposed conditioning pipeline can improve reconstruction fidelity under the evaluated synthetic acquisition setting.

However, this empirical improvement does not fully address my concerns about methodological novelty and the practical significance of the reconstructed volumes.

Critical weaknesses

1. Limited methodological novelty.

The proposed framework appears to be primarily an integration of existing techniques. In particular, FEDE, which contributes most of the performance gains according to Table 2, closely follows the Frequency-Enhanced Dual-Encoding architecture in DuFal (Tran Van et al., 2026). The global and local Fourier operators, spectral-channel factorization, Galerkin attention, and frequency-domain cross-attention have direct counterparts in DuFal, including the mathematical formulations in Eqs. (6)–(10).

Although the authors acknowledge DuFal, the overlap extends to the core design of FEDE. The main adaptation appears to be lifting these features into multi-scale volumetric conditions for a pretrained diffusion model, while the ControlNet-based adaptation itself is also established. Consequently, the paper offers limited methodological novelty and does not provide substantial new insights into biplanar X-ray-to-CT reconstruction.

2. Insufficient motivation and ablation for spectral conditioning.

The motivation for using Fourier Neural Operators in this task is not sufficiently established. What specific limitations of conventional spatial features are addressed by frequency-domain processing? More fundamentally, how does extracting additional spectral information from two X-ray projections help resolve the severe depth ambiguity of 3D reconstruction?

The ablation studies only demonstrate the benefit of adding the entire FEDE module. They do not establish whether the improvements arise from Fourier processing, increased model capacity, or richer multi-scale conditioning. Comparisons with parameter-matched spatial encoders and ablations of the individual frequency components are necessary to support the proposed design.

3. Limited practical significance and lack of downstream validation.

Biplanar X-ray-to-CT reconstruction is an extremely underdetermined problem. In such settings, the practical value of reconstruction often depends on recovering specific clinically relevant anatomical information, rather than merely improving overall image similarity. For example, applications such as scoliosis assessment, anatomical segmentation, and image-guided interventions require accurate reconstruction of particular structures, geometries, or organ volumes.

However, the paper evaluates reconstruction quality almost exclusively using PSNR, SSIM, and LPIPS. The qualitative results still exhibit noticeable anatomical inaccuracies, and it remains unclear whether the generated volumes are sufficiently reliable for downstream applications. Given the thoracic CT setting, evaluations such as lung/airway segmentation, organ-volume estimation, or lung-nodule preservation would provide more meaningful evidence of the method's utility.

Questions for authors

1. Spectral conditioning. Why is Fourier-domain processing particularly important for biplanar reconstruction? Can the authors provide comparisons with parameter-matched spatial encoders and ablations of the global/local Fourier branches and frequency-domain attention to isolate the benefits of spectral processing?

2. Potential pretraining–test data leakage. A serious concern is that the pretrained 3D MedDiffusion model was developed using datasets including LIDC-IDRI, which is also used for training and testing in this paper. Specifically, the original 3D MedDiffusion paper lists 1,010 LIDC-IDRI scans among its pretraining data sources, although the final training collection underwent additional filtering. The authors do not clarify whether the LIDC-IDRI test cases overlap with the pretraining data of the frozen diffusion backbone or CT autoencoder.Although the comparison with C-MedDiffusion partially mitigates this concern because both methods share the same pretrained prior, potential overlap could still affect the absolute reconstruction performance and the fairness of comparisons with other methods.

3. Downstream validity. What is the intended practical application of the reconstructed CT volumes? Can the authors demonstrate anatomical fidelity using downstream tasks such as segmentation, organ-volume estimation, or lesion preservation?

4. Reconstruction resolution and generalizability. Most experiments are conducted at only 128³ resolution, raising concerns about the preservation of fine anatomical structures. Although a 256³ experiment is included, it evaluates only FEDE without the proposed volumetric feature supervision and compares against C-MedDiffusion. Moreover, the Volume PSNR improvement decreases from approximately 0.913 dB at 128³ to 0.243 dB at 256³. While this does not conclusively establish poor scalability, it raises questions about whether the reported gains remain meaningful at higher spatial resolutions.

Additionally, all experiments use synthetic X-ray projections generated under a fixed projection geometry. Even the external LCTSC evaluation follows the same synthetic protocol. How well would the proposed method generalize to real clinical radiographs, different acquisition geometries, and potential geometric calibration errors? The current experiments provide limited evidence in this regard.

6. Completeness of comparisons. Why are closely related diffusion-based methods such as DiffuX2CT and DX2CT discussed but not included in the quantitative comparisons? Could the authors also report the additional computational cost of FEDE and the inference time of the complete framework, particularly given its 1,000-step DDIM sampling?

Reviewer_QEzGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a pipeline, including FEDE-ControlNet for reconstructing 3D chest CT volumes from two orthogonal X-rays (PA and lateral). It adapts a frozen 3D MedDiffusion [1] model using a trainable ControlNet with two conditioning pathways:

  • Geometric pathway: SDXL-VAE features from each X-ray are back-projected into a shared 3D grid.
  • FEDE pathway: Spatial convolution and Fourier Neural Operator (FNO) branches extract spatial-frequency features, which are fused through frequency-domain cross-attention and lifted into multi-scale 3D conditions.

The model also uses multi-level feature supervision from a frozen MAISI encoder to guide the reconstructed CT volumes during training.

Critical strengths

The method consistently outperforms the evaluated baselines in CT reconstruction and projection consistency metrics (Tables 1 and 3). The ablation study also supports the overall contributions of key components (Table 2; Figure 3).

Critical weaknesses

1. Limited novelty

  • Contribution 1: The original 3D MedDiffusion [1] already uses ControlNet for downstream tasks, including sparse-view CT reconstruction. Although the current paper focuses on biplanar inputs, it is unclear what is methodologically new in adapting the pretrained model, how the implementation differs, and whether these differences are non-trivial.
  • Contribution 2: The FEDE design in Section 3.2 appears to overlap substantially with DuFal [7], particularly the global/local Fourier processing and frequency-domain cross-attention. The main difference seems to be adapting these features for multi-scale volumetric conditioning. The authors should clarify what is newly proposed and why this adaptation is non-trivial.

2. Insufficient ablation studies

Table 2 shows the overall contributions of FEDE and VFM supervision, but several design choices remain unverified:

  • FEDE: No ablations of the global/local Fourier branches or frequency-domain cross-attention, despite their importance to Contribution 2.
  • Multi-level supervision: No comparison between multi-level and single-level supervision (e.g., using only the last selected layer), or between different feature extractors, including non-VFM models.

3. Missing baselines

Several relevant methods for the same task, including DiffuX2CT [2], DX2CT [3], DCT-Net [4], and XTransCT [5], are discussed but not included in Table 1. The authors should justify their omission and clarify whether comparisons under the same protocol are possible.

Without these comparisons, the claim of superior reconstruction performance in the abstract is not sufficiently supported.

Questions for authors
  1. Could the authors clarify the methodological differences between FEDE and DuFal [7], and ablate the global/local Fourier branches and frequency-domain cross-attention?
  2. Could the authors compare multi-level with single-level feature supervision, and evaluate different feature extractors? Also, how does the proposed loss differ from the perceptual loss used in PerX2CT [6], other than using a different model (VFM)?
  3. Could the authors include additional baselines [2–5], or explain why they cannot be compared?
  4. Could the authors evaluate the reconstructed CT volumes on downstream tasks, such as segmentation or lesion detection, to better validate their anatomical fidelity?

References

  1. Wang et al. (2025). 3D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-Quality Medical Image Generation. IEEE TMI.
  2. Liu et al. (2024). DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays. ECCV.
  3. Jeong et al. (2025). DX2CT: Diffusion Model for 3D CT Reconstruction from Bi or Mono-planar 2D X-ray(s). ICASSP.
  4. Zhang et al. (2025). DCT-Net: Dual-Branch CT Reconstruction from Orthogonal X-Rays with Diffusion Model and Contrastive Learning. MICCAI.
  5. Zhang et al. (2024). XTransCT: Ultra-Fast Volumetric CT Reconstruction Using Two Orthogonal X-Ray Projections. Physics in Medicine & Biology.
  6. Kyung et al. (2023). Perspective Projection-Based 3D CT Reconstruction from Biplanar X-Rays (PerX2CT). ICASSP.
  7. Van et al. (2026). DuFal: Dual-Frequency-Aware Learning for High-Fidelity Extremely Sparse-view CBCT Reconstruction. TMLR.
Reviewer_gsLnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FEDE-ControlNet for reconstructing 3D CT volumes from two orthogonal X-ray projections. It adapts a frozen pretrained 3D latent diffusion model using a trainable control branch conditioned on geometrically back-projected features and multi-scale spatial–frequency features combining global and patch-local Fourier representations. A training-only auxiliary loss compares reconstructed and reference CT volumes through a frozen volumetric feature encoder. Evaluation on synthetic projections from LIDC-IDRI, with external testing on LCTSC, reports improved reconstruction quality over the compared baselines. Ablation studies examine the contributions of the conditioning and supervision components.

Critical strengths
  1. The framework combines geometric back-projection and multiscale spatial–frequency conditioning within a trainable control branch. They provide complementary information to the frozen generator. It makes sense and has gained perfomrnace improvements. The volumetric feature supervision offers a reasonable way to constrain decoded reconstructions without adding inference-time computation.
  2. The method improves both CT reconstruction and projection-consistency metrics on LIDC-IDRI and retains an advantage on the external LCTSC cohort. The factorial ablation provides evidence for the benefit of FEDE under the same frozen prior and BP conditioning.
Critical weaknesses
  1. The overall methodological contribution appears incremental. The core spatial–frequency encoding design closely follows DuFal, while the main extension is its integration into multiscale volumetric conditioning for an existing diffusion prior through ControlNet, together with auxiliary volumetric feature supervision.
  2. The contribution beyond DuFal needs clearer positioning and comparative evidence. Sections 2.2 and 3.2 acknowledge DuFal, but the relationship deserves a more precise discussion. DuFal already combines spatial encoding, global/local Fourier processing, frequency-domain cross-attention, and geometry-based feature sampling.
  3. Insufficient validation of anatomical detail fidelity. Preserving local structural detail and maintaining fidelity to the individual target volume are central motivations of the proposed conditioning and supervision mechanisms. However, the evaluation primarily reports aggregate PSNR, SSIM, and LPIPS, together with selected qualitative examples. Projection consistency is assessed only at the two conditioning views, where agreement does not resolve the underlying 3D ambiguity.
  4. Figure 1 would benefit from a legend explaining the module colors, symbols, dashed connections, and hatched boxes, together with explicit labels for predicted and reference CT volumes.
Questions for authors
  1. Were the LIDC-IDRI test subjects and LCTSC evaluation subjects included inthe pretraining data of the 3D MedDiffusion model? It can be a concern of data leaking.
  2. For each external baseline, was a published checkpoint used or was the model retrained? Please give some details.
  3. There should have more implementaion details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.