acceptodds
Under review as a conference paper at ICLR 2027

TomoTransformer: Towards a Foundation Model for CT Reconstruction

Abstract

Supervised deep learning has advanced sparse-view tomographic reconstruction. However, conventional models, which typically map filtered back-projection (FBP) images or sinograms to clean reconstructions, are brittle under distribution shifts. Because they require retraining whenever projection counts and angles, detector resolutions, or data distributions change, their deployment in real-world applications remains limited. To address this, we introduce TomoTransformer, a transformer-based architecture that treats each local filtered projection as an individual token and predicts missing views via self-attention. Crucially, TomoTransformer operates in a back-projection space that separates projections across spatial locations, making view interpolation geometrically well-posed and invariant to detector size. This design yields a single foundation model that can process any number of input projections, at arbitrary angular locations and detector dimensions, and query any number of target angles without retraining. Trained on a large-scale dataset spanning diverse medical CT anatomies and natural images, TomoTransformer generalizes effectively across anatomies, materials, and resolutions. Extensive evaluations on several benchmark sparse-view datasets show that TomoTransformer significantly outperforms concurrent multi-purpose models like ViewTrans and matches or exceeds strong protocol-specific baselines, while remaining fully agnostic to the number of input and target projections. Furthermore, the model demonstrates robust zero-shot generalization on real experimental nanoscale brain data collected from an X-ray synchrotron, showcasing its practical utility for real-world applications.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
computed tomography (CT), image reconstruction, transformers

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4Cs2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TomoTransformer, a framework for sparse-view CT reconstruction that accommodates varying projection counts, viewing angles, and image resolutions. Its main contribution is to predict missing views in a local filtered back-projection space, using a geometric tokenizer and an angle-conditioned Transformer. The model is trained on a mixture of medical CT and natural images. Experiments evaluate reconstruction quality under fixed and varying view settings against several baselines. Additional studies examine geometric attention, zero-shot denoising, and transfer to real nanoscale brain tomography.

Critical strengths
  1. A coherent geometric design for flexible reconstruction. The method combines local filtered back-projection tokens with angle-conditioned view prediction, allowing one architecture to accommodate varying view counts and image sizes. This directly addresses a practical limitation of protocol-specific reconstruction models.
  2. Promising transfer to real experimental data. The zero-shot results on nanoscale brain tomography demonstrate transfer beyond the medical CT and natural-image training domains. Together with the improvements over ViewTrans, these results provide meaningful evidence of the approach's potential.
Critical weaknesses
  1. The evidence for broad generalization remains limited. Variable-view evaluation mainly covers two view counts on LoDoPaB-CT, while training uses nearly uniform angular sampling with small perturbations. Moreover, the varied model trails Restormer by 1.15 dB on average in Table 1, with a larger gap on HNSCC. These results leave the claimed balance between flexibility and reconstruction quality insufficiently established.
  2. Generalization across acquisition settings is not sufficiently established. The quantitative comparison under variable-view settings mainly covers two input view counts on LoDoPaB-CT, while training uses nearly uniform angular sampling with small perturbations. Moreover, the varied model trails Restormer by 1.15 dB on average in Table 1. Broader evaluation is needed to establish the trade-off between acquisition flexibility and reconstruction quality.
Questions for authors
  1. Table 1 shows a substantial performance drop from the fixed to the varied model, particularly on HNSCC. What explains this difference, and how should it qualify the claim that a single model matches protocol-specific baselines?
  2. Can the authors provide results under substantially nonuniform angular sampling or on a fully held- out dataset? Such results would help distinguish generalization beyond the training distribution from flexibility within the sampled training settings.
  3. Can the authors isolate the contributions of the back-projection representation and geometric tokenizer through controlled ablations? A comparison with image-domain baselines using larger patches would also help establish whether the reported advantage is robust to spatial context.
  4. How were training and test data separated at the patient or volume level, and how many independent subjects are represented by the 32 test slices per dataset? Please provide uncertainty estimates for the main comparisons if available.
5 comments

We thank the reviewer for the careful review and the precise questions. We answer them in order; all changes are shown in blue in the revised PDF.

The revised manuscript introduces three key additions (see the general response): (i) ray-consistent stitching (RCS), a novel inference method that enforces consistency among all patches intersected by the same ray, improving reconstruction quality without requiring retraining; (ii) fine-tuning of the pre-trained model, either supervised, using a small set of ground-truth images, or self-supervised, using the measured views from a single scan without requiring ground truth; and (iii) a second real experimental dataset, consisting of a commercial processor chip from a publicly available 4-nm PXCT dataset. We now address each point raised in the review in turn.

Q1 (and W1–W2):

TomoTransformer is designed as a foundation model for tomographic reconstruction: a single model that adapts to different numbers of measured views, sampling patterns, and detector resolutions, while generalizing to unseen anatomies and scanners. In contrast, protocol-specific models are trained for a single acquisition protocol. TomoTransformer (varied) is the model that provides this flexibility. We apply it directly to the real experimental data in Section 5.5, and our newly proposed self-supervised test-time fine-tuning (Section 4.3) further improves its performance on these data.

In addition, our new inference method, ray-consistent stitching (RCS), improves TomoTransformer (fixed) without retraining, increasing its average PSNR in Table 1 from 39.66 to 40.07 dB. With RCS, TomoTransformer (fixed) becomes the best-performing method across all datasets in both PSNR and SSIM.

At the protocol of Table 1 (64 → 256 views), the gap between TomoTransformer (varied, RCS) and Restormer is now small: 0.81 dB on average (38.66 vs. 39.47 dB). Most of this difference comes from a single dataset, HNSCC; across the other seven datasets, the average gap is only 0.22 dB. This remaining gap can be closed by supervised fine-tuning on very little data (Section 4.3). Using only 400 training slices (50 per dataset), fine-tuned TomoTransformer (varied, RCS) achieves an average of 39.54 dB / 0.934, outperforming Restormer (39.47 dB / 0.927) and all other image-domain baselines (last rows of Tables 1 and 2).

We now further investigate the trade-off between performance and flexibility. To demonstrate the advantages of a single foundation model across different sparsity levels, we evaluated all models of Table 1, without retraining, under a new acquisition protocol with 128 → 256 (new Table 3 in the revised manuscript). When presented with twice as many measured views, the fixed-protocol models improve by at most 2.2 dB on average, and some even suffer a performance degradation. For example, CTformer loses 1.0 dB on average, while Restormer drops from 44.73 to 42.05 dB on HNSCC. In contrast, TomoTransformer (varied), even without fine-tuning, achieves the best results on all eight datasets in both PSNR and SSIM, with an average PSNR of 43.43 dB compared to 40.82 dB for Restormer. These results demonstrate the advantage of TomoTransformer's flexibility: unlike protocol-specific models, it can adapt to changes in the acquisition protocol without retraining.

Q2. Non-uniform angular sampling and fully held-out data:

Fully held-out data: We evaluate generalization beyond the training distribution at two levels.

  • Held-out datasets (new Appendix F.5, Table 11): We trained Restormer, our strongest baseline, and TomoTransformer (fixed) on a single dataset, either LoDoPaB-CT or KiTS23, using the same training protocol and settings as in Table 1 (an experiment also requested by another reviewer), and evaluated both models on all eight test datasets. On the respective training domains, the two models achieve comparable performance. However, on the seven datasets not seen during training, TomoTransformer generalizes substantially better, outperforming Restormer by up to 2.44 dB. While both models perform similarly in-domain when trained on a single dataset, TomoTransformer exhibits markedly stronger out-of-distribution generalization. This is particularly important in practical applications, where the scanned anatomy or scanner may differ from those encountered during training.

  • Held-out modality and samples (Section 5.5): The two real PXCT datasets are substantially different from all training data in terms of imaging physics (phase contrast), spatial scale (nanometer-sized pixels), and sample content (brain tissue and metal interconnects in a commercial processor chip). Without any fine-tuning, TomoTransformer improves upon FBP by 4.4 dB on the brain and 4.7 dB on the chip. Combining our new self-supervised test-time fine-tuning (Section 4.3) with RCS further improves the reconstructions, reaching 36.29 dB on the brain and 28.35 dB on the chip (Table 14). These experiments demonstrate generalization not only to entirely unseen datasets, but also to real experimental data with substantially different imaging physics, spatial scales, and sample content.

Non-uniform sampling (third paragraph of Appendix F.8 and Figure 11): We now evaluate the real chip scan at its acquired angles, which are non-uniformly spaced, with the angular density increasing sharply around 44° and 133° to as many as 29 views per degree, compared with a median of 8 views per degree. We preserved the original acquisition angles exactly and used 86 measured views as input. The resulting angular gaps range from 0.54° to 3.06°, rather than the uniform spacing of 2.1°. TomoTransformer achieves 28.3–28.5 dB, compared with 24.0–24.2 dB for weighted FBP using only the 86 input views. Moreover, the 617 views of the real brain scan are also not uniformly spaced; the baselines trained on uniform angles hallucinate texture and leave blocking artifacts there, while TomoTransformer, which conditions on each actual angle, gives the closest reconstruction to the reference (Figure 12 of the revised manuscript). These two experiments on real data demonstrate that TomoTransformer can handle realistic, non-uniform angular sampling patterns.

Q3. Ablations of the back-projection representation and the geometric tokenizer; image-domain baselines with larger patches:

  • Geometric tokenizer (new Appendix F.3, Table 9): We replaced the geometric tokenizer and detokenizer with linear maps of the full 32×3232 \times 32 back-projection patch and kept everything else the same. The geometric tokenizer is better on all eight datasets in both metrics, by 0.78 dB on average, with 1 M fewer parameters.
  • Encoder-decoder design, MAE vs. BERT-style (new Appendix F.4, Table 10): TomoTransformer uses a masked-autoencoder design. We compare it with a BERT-style variant in which all 88 transformer blocks process the full sequence of 256256 tokens from the start. Our experiments show that the two backbones perform almost identically. We retain the MAE-style design because its encoder attends only to measured tokens, reducing computation.
  • Patch size (Appendix F.2, Table 8): We trained TomoTransformer and DRUNet under the fixed 128→256128 \rightarrow 256 protocol with P∈{16,32,64,128}P \in \{16, 32, 64, 128\}, changing only the patch size. For TomoTransformer, P=16P=16 and P=32P=32 show comparable performance (44.20 vs. 44.17 dB average PSNR), while P=64P=64 and P=128P=128 fall to 43.84 and 42.93 dB. The larger patches require more measured views to sample their back-projections without aliasing, so with 128 measured views their missing views are harder to predict. DRUNet shows the opposite trend: its average PSNR rises from 41.97 dB at P=16P=16 to 44.34 dB at P=128P=128, as the larger image patches provide more spatial context. DRUNet with P=128P=128 is comparable to TomoTransformer with P=16P=16 on average.

Q4. Train/test split, number of subjects, and uncertainty:

Each of the 32 test slices per CT dataset is sampled independently from a randomly selected held-out test volume, with each dataset containing between 6 and 80 test volumes (now clarified in Section 5.1). The LoDoPaB-CT, FFHQ, and ImageNet test sets, in contrast, consist of collections of individual slices or images.

Uncertainty analysis (Appendix F.6, Table 12): We added a paired comparison against our strongest baseline, Restormer, and compute 95% confidence intervals using bootstrap resampling over test volumes. TomoTransformer (fixed, RCS) outperforms Restormer by an average of +0.60 dB (95% CI: [0.52, 0.68]) and achieves higher PSNR on 217 of the 256 test slices. The improvement is statistically significant on seven of the eight datasets. The exception is HNSCC, whose test set contains only six volumes: the confidence interval includes zero (+0.24 dB, 95% CI: [−0.15, 0.66]), indicating no statistically significant difference between the two methods on this dataset.

We thank the reviewer again; we believe these additions substantially strengthen the paper.

Reviewer_7iUGOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a method for training a foundation model for CT reconstruction, based on a transformer that uses a sequence of local sinogram patches as input, and outputs missing sinogram patches to increase the number of angles. Results are compared for a heterogeneous dataset of medical CT, showing improvement over comparison methods.

Critical strengths
  • The approach to use local sinogram patches is interesting, and the approach seems valid.
  • Results seem promising, with increased metrics for the specific cases the paper investigates.
Critical weaknesses
  • It is unclear how the method would handle cases with moderate to severe noise. I can imagine that the tokenizer could have trouble with such cases, and that the entire method could fail. The amount of noise in Figure 5 is very limited, making that experiment not very informative.
  • It would be useful to include a comparison with comparison methods trained specifically for a single dataset, instead of the heterogeneous dataset used now. In practice, one knows which region is scanned with CT, so a region-specific deep learning method is easy to achieve. In any case it would be useful to assess whether the improvement over other methods still holds for dataset-specific training.
  • I don't really see any clear visible improvements (or even significant differences) in Figure 6.
  • It is not clear why the zoom regions in Figure 3 and 4 were selected, as the don't show interesting regions or behaviour.
Questions for authors
  • How does the method handle cases with moderate to severe noise?
  • How does the method compare with other methods for single-dataset training?
  • Why are there no significant improvements visible in Figure 6?
3 comments

We thank the reviewer for the positive assessment and the constructive suggestions. We answer the points in order; all changes are shown in blue in the revised PDF.

The revised manuscript introduces three key additions (see the general response): (i) ray-consistent stitching (RCS), a novel inference method that enforces consistency among all patches intersected by the same ray, improving reconstruction quality without requiring retraining; (ii) fine-tuning of the pre-trained model, either supervised, using a small set of ground-truth images, or self-supervised, using the measured views from a single scan without requiring ground truth; and (iii) a second real experimental dataset, consisting of a commercial processor chip from a publicly available 4-nm PXCT dataset. We now address each point raised in the review in turn.

W1/Q1. Moderate to severe noise:

We agree that the noise levels of the original denoising experiment were mild. We repeated the experiment under much stronger noise (new Appendix F.7, Table 13, Figure 8), with the same model, trained only on noiseless projections, and no retraining. The last column is the better of the two re-masking schemes:

Noise (SNR on the projections)FBP of the noisy viewsTomoTransformer
Gaussian, 45 dB28.96 / 0.55936.93 / 0.887
Gaussian, 40 dB24.45 / 0.33935.08 / 0.852
Gaussian, 35 dB19.62 / 0.16432.71 / 0.786
Poisson, I0=105I_0 = 10^5 (46 dB)30.03 / 0.62037.54 / 0.900
Poisson, I0=3×104I_0 = 3\times10^4 (41 dB)25.44 / 0.39935.81 / 0.869
Poisson, I0=104I_0 = 10^4 (36 dB)20.87 / 0.21733.95 / 0.829

The method does not break down under stronger noise; in fact, its improvement over FBP increases with the noise level, from 8 dB to 13 dB. The reconstructions in Figure 8 show no spurious structures that are absent from the reference. This robustness is also supported by the tokenizer's linearity: it samples the back-projection along a chord and applies a linear mapping. Consequently, noise propagates into the transformer as additive noise on the tokens, rather than disrupting the tokenization process.

W2/Q2. Single-dataset training:

We agree that this is an important practical setting. To investigate it, we trained Restormer, our strongest baseline, and TomoTransformer (fixed) on a single dataset, either LoDoPaB-CT or KiTS23, using the same training protocol and settings as in Table 1. We then evaluated both models on all eight test datasets (new Appendix F.5, Table 11). On the respective training domains, the two models achieve comparable performance. However, on the seven datasets not seen during training, TomoTransformer generalizes substantially better, outperforming Restormer by up to 2.44 dB. While both models perform similarly in-domain when trained on a single dataset, TomoTransformer exhibits markedly stronger out-of-distribution generalization. This is particularly important in practical applications, where the scanned anatomy or scanner may differ from those encountered during training.

W3/Q3. No visible improvement in the real-data figure (submitted Figure 6):

We have revised this figure, which is now Figure 7:

  • Processor chip (new real experimental sample): The revised Figure 7 now includes a second real experimental sample, a commercial processor chip, where the improvement is particularly evident. The FBP reconstruction from 86 views exhibits strong streak artifacts across the metal interconnects, which TomoTransformer effectively removes (24.2 → 27.9 dB for the shown slice, further improving to 30.3 dB with self-supervised fine-tuning and RCS). The zoomed-in regions highlight how RCS further suppresses blocking artifacts in the chip reconstruction.

  • Nanoscale brain: Self-supervised fine-tuning (Section 4.3) combined with RCS further improves the reconstruction (36.29 dB / 0.849, Table 14). We also repositioned the zoomed-in region to focus on a more structured area, where the improvements are more visible.

W4. Choice of zoom regions (Figures 4 and 5):

We agree. The original zoom regions were placed on homogeneous soft tissue. In the revised Figures 4 and 5, we moved them to structured regions where the methods differ more clearly.

Thank you again for the supportive review.

Thank you for the response. The added noise results are useful, especially the high-noise cases.

For Q2, I meant more a standard UNet postprocessing approach, trained on a specific low-angle+noise to high-quality image, for one specific angle count and noise setting. In my experience, this can (and should) give very reasonable results, even with relatively little training data (many papers show this as well).

Looking at Figure 12, I started to have serious concerns about the application of the comparison methods. In that image, severe patching artifacts are visible for all methods except the proposed method, probably making the metrics worse. These artifacts are likely easy to mitigate by using overlapping patches and a windowed average (e.g. Hanning or gaussian).

Because of these remaining concerns, I keep my recommendation at 3 (although the patching issue should really be addressed before acceptance in my view).

We thank the reviewer for the follow-up and are pleased that the high-noise results were useful.

Regarding the standard post-processing network for a fixed setting, we believe the new single-dataset experiment (Appendix F.5, Table 11) implements the requested standard approach, with the exception that we do not add noise. We train Restormer, a post-processing network, on a single dataset (LoDoPaB-CT or, in a separate experiment, KiTS23), using a fixed view count and sparsity pattern. The network maps the 64-view FBP reconstruction to the corresponding high-quality 256-view FBP. We chose Restormer over U-Net because it was the strongest image-domain post-processing baseline in Table 1 (39.47 dB average, compared with 36.86 dB for U-Net and 38.88 dB for DRUNet). TomoTransformer (fixed) is trained on the same single dataset under the same protocol. As expected, both models perform very well in-distribution. When trained on LoDoPaB-CT, Restormer and TomoTransformer achieve 38.48 dB and 38.39 dB, respectively, on LoDoPaB-CT, improving by more than 9 dB over FBP (29.24 dB). The difference emerges on out-of-distribution data where TomoTransformer outperforms Restormer on all seven unseen datasets.

Regarding the patching artifacts in Figure 12, we thank the reviewer for this suggestion. We have added a new Figure 13 to the revised manuscript, extending the comparison in Figure 12 by using overlapping patches for all image-domain baselines. Specifically, every pixel is covered by 16 overlapping patches, whose predictions are combined using a Hann-window-weighted average. As anticipated by the reviewer, this eliminates the blocking artifacts and improves the baselines' PSNR by 0.2–0.4 dB, but at 16$\times$ the inference cost. However, TomoTransformer still outperforms all baselines without requiring overlapping patches. This experiment shows the better generalization of TomoTransformer compared to the baselines, where ray-consistent stitching (RCS, Section 4.2) provides consistent reconstruction across the image without the need for overlap-based averaging. Across the 24 slices, TomoTransformer with RCS achieves 34.97 dB, compared with at most 34.14 dB for the best overlapping-patch baseline (Restormer). For the representative slice shown in Figure 13, the corresponding scores are 33.7 dB and 33.1 dB.

We hope this addresses your concerns. Please let us know if we misunderstood any of your points.

Reviewer_5NAXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TomoTransformer is a transformer-based model for sparse-view CT reconstruction that addresses the dependence of existing methods on fixed projection counts, angular configurations, and detector resolutions. The key idea is to formulate missing-view prediction in a disentangled back-projection space. The proposed method converts local filtered back-projection patches into tokens using a geometric tokenizer and predicts missing views through an encoder-decoder Transformer with positional embeddings and geometric attention. The predicted patches are reconstructed using a geometric detokenizer. TomoTransformer is trained on a dataset of medical CT and natural images and evaluated on fixed and varied sparse-view reconstruction, zero-shot denoising, and real experimental nanoscale brain tomography. Experimental results show that TomoTransformer achieves the highest average PSNR in fixed sparse-view reconstruction, outperforms ViewTrans in varied-number sparse-view reconstruction, and demonstrates zero-shot denoising and generalization to real nanoscale brain data.

Critical strengths

The formulation of missing-view prediction in a disentangled back-projection space is interesting, and the overall reconstruction pipeline is reasonable and easy to understand. The experiments cover multiple medical CT anatomies and natural images, providing useful evidence for the proposed framework.

Critical weaknesses

Limited validation of the generalization claim. The experimental comparison focuses primarily on general-purpose image restoration models, with limited coverage of CT-specific reconstruction methods. Moreover, TomoTransformer (varied) achieves lower average PSNR than several baselines in Table 1, but the trade-off between reconstruction performance and flexibility is not fully analyzed. The variable-view comparison with ViewTrans is evaluated at only two sparsity levels (128→256 and 64→256 views) on LoDoPaB-CT, while zero-shot evaluation on data from other sources is limited to a single nanoscale brain dataset. This leads to concerns about the generalization claim of the paper.

Lack of ablation study on the key components. The transformer backbone largely follows a standard encoder-decoder design. However, the ablation studies primarily focus on geometric attention, without sufficiently evaluating the disentangled back-projection space, geometric tokenizer, and patch-based design. Furthermore, despite emphasizing computational efficiency and scalability, the paper lacks quantitative comparisons of inference time, computational cost, and memory consumption.

Questions for authors

Questions For Authors:

  1. Could the authors better justify their baseline selection and discuss relevant CT-specific reconstruction methods that are missing, such as SIPID [1], CIN [2], MDST [3], and TD-STrans [4] (cited in related work, but not compared)?
  2. TomoTransformer (varied) achieves lower average PSNR than several baselines in Table 1. Could the authors discuss the trade-off between reconstruction performance and flexibility further?
  3. The variable-view comparisons with ViewTrans cover only two input-view configurations on LoDoPaB-CT, while real-data zero-shot evaluation uses a single nanoscale brain dataset. Could the authors provide further evidence across different input-view counts, angular sampling patterns, and out-of-distribution datasets?
  4. Could the authors provide ablation studies to isolate the contributions of the disentangled back-projection space, geometric tokenizer, and patch-based design, including different patch sizes?
  5. Could the authors provide quantitative comparisons of inference time, FLOPs, and peak GPU memory usage against the baselines to support the claimed efficiency and scalability?
  6. Could the authors evaluate whether image-domain baselines trained with the same variable-view augmentation exhibit similar generalization capabilities, to better distinguish the architectural contributions from the effects of diverse training configurations?

[1] H. Yuan, J. Jia, Z. Zhu, SIPID: A deep learning framework for sinogram interpolation and image denoising in low-dose CT reconstruction, ISBI), 2018, pp. 1521–1524.

[2] F. Guo, B. Yang, H. Feng, W. Zheng, L. Yin, Z. Yin, et al., An efficient sinogram domain fully convolutional interpolation network for sparse-view computed tomography reconstruction, Appl. Sci. 13 (20) (2023) 11264.

[3] Y. Li, X. Sun, S. Wang, X. Li, Y. Qin, J. Pan, et al., MDST: Multi-domain sparse-view CT reconstruction based on convolution and swin transformer, Phys. Med. Biol. 68 (9) (2023) 95019.

[4] Y. Li, X. Sun, S. Wang, L. Guo, Y. Qin, J. Pan, et al., TD-STrans: Tri-domain sparse-view CT reconstruction based on sparse transformer, Comput. Methods Programs Biomed. 260 (2025) 108575.

4 comments

We thank the reviewer for the detailed review and the concrete suggestions. We answer the questions in order; all changes are shown in blue in the revised PDF.

The revised manuscript introduces three key additions (see the general response above): (i) ray-consistent stitching (RCS), a novel inference method that enforces consistency among all patches intersected by the same ray, improving reconstruction quality without requiring retraining; (ii) fine-tuning of the pre-trained model, either supervised, using a small set of ground-truth images, or self-supervised, using the measured views from a single scan without requiring ground truth; and (iii) a second real experimental dataset, consisting of a commercial processor chip from a publicly available 4-nm PXCT dataset. We now address each point raised in the review in turn.

Q1. Baseline selection and CT-specific methods:

We selected strong image-domain post-processing networks (U-Net/FBPConvNet, DRUNet, NAFNet, Restormer, and CTformer) because they are among the most widely used baselines for sparse-view CT. Like TomoTransformer, these methods can be applied patch-wise, allowing them to be trained on the same heterogeneous datasets at their respective native resolutions. ViewTrans serves as our flexible sinogram-domain baseline and is trained using the same variable-view schedule as TomoTransformer.

Following the reviewer's suggestion, we have added CIN [2] as a CT-specific sinogram-domain baseline (Tables 1–2). Trained on the same data under the same protocol, CIN achieves an average PSNR of 38.55 dB in Table 1, below both Restormer (39.47 dB) and TomoTransformer (40.07 dB with RCS).

The remaining three methods do not permit a directly comparable evaluation under our experimental setup. SIPID [1] has no publicly available implementation and combines a sinogram-interpolation network, similar in approach to CIN, with an image-domain denoiser, a class of methods already represented by our image-domain baselines. MDST [3] and TD-STrans [4] are dual- and tri-domain architectures, respectively, that incorporate the forward operator and FBP directly into the network. They process entire images rather than patches and are therefore tied to a fixed image size and angular sampling grid. Consequently, they cannot readily be trained across the native resolutions of our eight datasets using a common protocol. Furthermore, neither method has publicly available code. We now discuss all four methods in the related work section.

Q2. Trade-off between performance and flexibility:

TomoTransformer is designed as a foundation model for tomographic reconstruction: a single model that adapts to different numbers of measured views, sampling patterns, and detector resolutions, while generalizing to unseen anatomies and scanners. In contrast, protocol-specific models are trained for a single acquisition protocol. TomoTransformer (varied) is the model that provides this flexibility. We apply it directly to the real experimental data in Section 5.5, and our newly proposed self-supervised test-time fine-tuning (Section 4.3) further improves its performance on these data.

In addition, our new inference method, ray-consistent stitching (RCS), improves TomoTransformer (fixed) without retraining, increasing its average PSNR in Table 1 from 39.66 to 40.07 dB. With RCS, TomoTransformer (fixed) becomes the best-performing method across all datasets in both PSNR and SSIM.

At the protocol of Table 1 (64 → 256 views), the gap between TomoTransformer (varied, RCS) and Restormer is now small: 0.81 dB on average (38.66 vs. 39.47 dB). Most of this difference comes from a single dataset, HNSCC; across the other seven datasets, the average gap is only 0.22 dB. This remaining gap can be closed by supervised fine-tuning on very little data (Section 4.3). Using only 400 training slices (50 per dataset), fine-tuned TomoTransformer (varied, RCS) achieves an average of 39.54 dB / 0.934, outperforming Restormer (39.47 dB / 0.927) and all other image-domain baselines (last rows of Tables 1 and 2).

We now further investigate the trade-off between performance and flexibility. To demonstrate the advantages of a single foundation model across different sparsity levels, we evaluated all models from Table 1, without retraining, under a new acquisition protocol with 128 → 256 (new Table 3 in the revised manuscript). When presented with twice as many measured views, the fixed-protocol models improve by at most 2.2 dB on average, and some even suffer a performance degradation. For example, CTformer loses 1.0 dB on average, while Restormer drops from 44.73 to 42.05 dB on HNSCC. In contrast, TomoTransformer (varied), even without fine-tuning, achieves the best results on all eight datasets in both PSNR and SSIM, with an average PSNR of 43.43 dB compared to 40.82 dB for Restormer. These results demonstrate the advantage of TomoTransformer's flexibility: unlike protocol-specific models, it can adapt to changes in the acquisition protocol without retraining.

Q3. More view counts, angular sampling patterns, and out-of-distribution data:

View counts: In addition to the 64 → 256 and 128 → 256 view settings (Tables 1–4), we added an experiment with 86 → 256 views on the new real chip dataset (Figure 7, Table 14). The results demonstrate that TomoTransformer maintains strong performance across different numbers of measured views, including on real experimental data.

Out-of-distribution data: As mentioned earlier, we added a second real experimental dataset (Section 5.5): a pillar of a commercial processor chip from the publicly available 4-nm PXCT dataset of Aidukas et al. (Nature, 2024). Its high-contrast metal interconnects and sharp edges are substantially different from anything encountered during training. Using only 86 measured views, the zero-shot model improves upon FBP by 4.7 dB (22.00 → 26.73 dB), while self-supervised fine-tuning combined with RCS further improves the PSNR to 28.35 dB (Table 14). This experiment provides further evidence of the model's ability to generalize to real experimental data from a previously unseen domain.

Non-uniform angular sampling (new paragraph and Figure 11 in Appendix F.8): The real chip scan itself uses non-uniform angular sampling, with the angular density increasing sharply around 44° and 133° to as many as 29 views per degree, compared with a median of 8 views per degree. We preserved the original acquisition angles exactly and used 86 measured views as input. The resulting angular gaps range from 0.54° to 3.06°, rather than the uniform spacing of 2.1°. Against the weighted FBP reconstruction using all 256 views, TomoTransformer achieves 28.3–28.5 dB, compared with 24.0–24.2 dB for weighted FBP using only the 86 input views. This experiment demonstrates that TomoTransformer can handle realistic, non-uniform angular sampling patterns.

Q4. Ablations of the back-projection space, geometric tokenizer and patch-based design:

  • Geometric tokenizer (new Appendix F.3, Table 9): We replaced the geometric tokenizer and detokenizer with linear maps of the full 32×3232 \times 32 back-projection patch and kept everything else the same. The geometric tokenizer is better on all eight datasets in both metrics, by 0.78 dB on average at 128 → 256 (0.31–1.17 dB per dataset), with 1 M fewer parameters; the gap is the same at 64 → 256. A single-view back-projection patch is fully determined by its 1-D profile across the stripes. The geometric tokenizer builds this structure in, so every decoded patch is a valid single-view back-projection, whereas the full-patch model must learn it from data.
  • Patch size (Appendix F.2, Table 8): We trained TomoTransformer and DRUNet under the fixed 128→256128 \rightarrow 256 protocol with P∈{16,32,64,128}P \in \{16, 32, 64, 128\}, changing only the patch size. For TomoTransformer, P=16P=16 and P=32P=32 show comparable performance (44.20 vs. 44.17 dB average PSNR), while P=64P=64 and P=128P=128 fall to 43.84 and 42.93 dB. The larger patches require more measured views to sample their back-projections without aliasing, so with 128 measured views their missing views are harder to predict. DRUNet shows the opposite trend: its average PSNR rises from 41.97 dB at P=16P=16 to 44.34 dB at P=128P=128, as the larger image patches provide more spatial context. DRUNet with P=128P=128 is comparable to TomoTransformer with P=16P=16 on average.
  • Encoder-decoder backbone (Appendix F.4, Table 10): The MAE-style and BERT-style backbones perform almost identically, showing that performance does not depend on the encoder-decoder split. We retain the MAE-style design because its encoder attends only to measured tokens, so it is cheaper than processing the full sequence in every block.
  • Geometric attention is already ablated in Tables 5–6.
  • Back-projection space vs. raw sinogram: Operating in the back-projection space is a key design choice of TomoTransformer. It allows the network to attend to the measured views that are relevant to each image patch, and it enables the geometric tokenizer to encode the structure of a single-view back-projection. Without this choice, the network must learn the structure of a single-view back-projection from data, similar to ViewTrans. We already showed that TomoTransformer outperforms ViewTrans by 4.6 dB at both 64 → 256 and 128 → 256 (Table 4). Although this comparison is not a single-component ablation, the representation and patch-based design are intrinsically linked: each sinogram view couples all pixels in the image, whereas the back-projection space makes a patch-based approach possible.

Q5. Inference time, FLOPs, and memory (new Appendix F.1, Table 7):

We benchmarked all models on a single 362×362362 \times 362 slice under the 64 → 256 view protocol using an NVIDIA A100 GPU. TomoTransformer requires 666 GFLOPs, comparable to CTformer (669 GFLOPs) and approximately half the computational cost of CIN (1,211 GFLOPs). However, it has the longest inference time (67 ms per slice, compared with 10–58 ms for the other models of Table 1) and a memory footprint comparable to Restormer (1.5 vs. 1.4 GiB). We do not claim that TomoTransformer is faster than the baselines. Our scalability claim instead concerns the image size. Since TomoTransformer processes only P×PP \times P patches at a time, its peak memory consumption is independent of the overall image dimensions, while its computational cost scales linearly with the number of image pixels. In contrast, methods that process the entire sinogram or incorporate the forward operator directly into the network can become computationally and memory intensive as the image size increases. TomoTransformer's patch-based design provides an important scalability advantage for high-resolution tomographic reconstruction.

Q6. Image-domain baselines trained with variable-view augmentation:

We agree that training image-domain baselines with variable-view augmentation would help disentangle the contribution of the architecture from that of the training configuration. Nevertheless, our existing experiments already provide two relevant comparisons. First, in Tables 1 and 2, TomoTransformer and all baselines are trained under the same configurations: for the fixed-view protocol, all models use the same training data and number of measured views. Under these controlled conditions, TomoTransformer (fixed, RCS) significantly outperforms the best baseline, Restormer, by 0.60 dB on average (95% CI [0.52, 0.68]; Appendix F.6, Table 12).

Second, we added an investigation of the generalization of Restormer and TomoTransformer (fixed) when trained on a single dataset under a single acquisition configuration (Appendix F.5, Table 11), as also requested by another reviewer. While the two models achieve comparable performance on their respective training domains, TomoTransformer generalizes substantially better to unseen datasets, outperforming Restormer by up to 2.44 dB. This comparison provides further evidence that the architecture itself contributes to the improved cross-domain generalization.

Furthermore, variable-view augmentation is not directly transferable to image-domain models. Such models map an FBP reconstruction to a clean image and therefore cannot explicitly access or query additional measured views. The acquisition angles are available only implicitly, through the streak patterns in the FBP image. In contrast, TomoTransformer explicitly processes the measured views and their geometric information. Consequently, implementing a comparable variable-view training strategy for image-domain models, analogous to our experiment with ViewTrans in Section 5.3, is not straightforward.

We hope these address your questions. Please let us know if we misunderstood any of your points.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank you + many updates

We thank the reviewers for their careful reading and constructive comments. We are glad that the reviewers found TomoTransformer an interesting formulation of missing-view prediction in the local back-projection space (5NAX & 7iUG), with a useful geometric design for flexible reconstruction supported by promising results, in particular the transfer to real experimental data (4Cs2). We have revised the paper; all changes are shown in blue in the updated PDF. Below we summarize the main changes, and we answer each reviewer's remaining points in the individual responses.

1. New methods and data

  • Ray-consistent stitching (RCS): We propose a novel inference method specifically designed for TomoTransformer (Section 4.2). Each token predicted by TomoTransformer represents the local back-projection of a single filtered projection. Consequently, all predicted patches intersected by the same ray should have identical values. Our proposed ray-consistent stitching (RCS) enforces this constraint by averaging the predicted back-projected patches along each ray for missing views. It requires no retraining and introduces negligible computational overhead. As shown in Tables 1 and 2, RCS improves the average PSNR of TomoTransformer (fixed) from 39.66 to 40.07 dB, making it the best-performing method across all datasets in both PSNR and SSIM.

  • Self-supervised test-time fine-tuning: We introduced TomoTransformer as a foundation model for tomographic reconstruction. Despite its strong zero-shot generalization to real experimental data, a domain gap may remain between the training and test distributions. To address this, we propose self-supervised test-time fine-tuning using the measured projections from a single scan (Section 4.3). No ground-truth images are required: we randomly mask a subset of the measured views and fine-tune the model to predict them. On real experimental data, this approach consistently outperforms the zero-shot model, with a particularly substantial improvement of +1.41 dB on the new chip dataset (Figure 7, Table 14).

  • Supervised fine-tuning: When a small number of ground-truth images are available, the pre-trained TomoTransformer can also be fine-tuned using its original training loss (Section 4.3). With only 400 training slices, TomoTransformer (varied, RCS) outperforms all protocol-specific baselines on average, as shown in the last rows of Tables 1 and 2.

  • A second real experimental dataset: In addition to the nanoscale brain dataset, we now evaluate TomoTransformer on a pillar of a commercial processor chip from the publicly available 4-nm PXCT dataset of Aidukas et al. (Nature, 2024) (Section 5.5). The chip's high-contrast metal interconnects and sharp edges are substantially different from anything encountered during training. In the zero-shot setting, TomoTransformer improves upon filtered back-projection (FBP) by 4.7 dB (22.00 → 26.73 dB), while test-time fine-tuning combined with RCS further increases the PSNR to 28.35 dB. This additional experiment demonstrates the model's ability to generalize to real experimental data from a substantially different, previously unseen domain.

2. Additional experiments, ablations and analysis

  • CT-specific sinogram-domain baseline: We added CIN (Guo et al., 2023) as a CT-specific sinogram-domain baseline, trained under the same protocol as our other models (Tables 1 and 2). CIN achieves an average PSNR of 38.55 dB, compared to 40.07 dB for TomoTransformer (fixed, RCS).

  • Additional experiments, ablation studies, and analysis: We evaluated all models from Table 1 with an increased number of measured views (128 → 256; Table 3) and compared the computational costs of TomoTransformer and the baselines (Appendix F.1). We also ran ablation studies on the patch size for both TomoTransformer and the image-domain baseline DRUNet (Appendix F.2, Table 8), the geometric tokenizer (Appendix F.3), and the encoder–decoder backbone (Appendix F.4).

  • Single-dataset training: We trained Restormer and TomoTransformer (fixed) on a single dataset, either LoDoPaB-CT or KiTS23, and evaluated both models on all eight test sets (Appendix F.5, Table 11). On in-domain datasets, the two models perform comparably. However, on out-of-domain datasets, TomoTransformer demonstrates substantially stronger generalization, outperforming Restormer by up to 2.44 dB.

  • Robustness and statistical analysis: We added a statistical significance analysis of the main comparison (Appendix F.6), evaluated robustness under stronger noise conditions (Appendix F.7), and investigated performance with the non-uniform acquisition angles used in the real chip scan (Appendix F.8, Figure 11).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.