- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents TomoTransformer, a framework for sparse-view CT reconstruction that accommodates varying projection counts, viewing angles, and image resolutions. Its main contribution is to predict missing views in a local filtered back-projection space, using a geometric tokenizer and an angle-conditioned Transformer. The model is trained on a mixture of medical CT and natural images. Experiments evaluate reconstruction quality under fixed and varying view settings against several baselines. Additional studies examine geometric attention, zero-shot denoising, and transfer to real nanoscale brain tomography.
Critical strengths
- A coherent geometric design for flexible reconstruction. The method combines local filtered back-projection tokens with angle-conditioned view prediction, allowing one architecture to accommodate varying view counts and image sizes. This directly addresses a practical limitation of protocol-specific reconstruction models.
- Promising transfer to real experimental data. The zero-shot results on nanoscale brain tomography demonstrate transfer beyond the medical CT and natural-image training domains. Together with the improvements over ViewTrans, these results provide meaningful evidence of the approach's potential.
Critical weaknesses
- The evidence for broad generalization remains limited. Variable-view evaluation mainly covers two view counts on LoDoPaB-CT, while training uses nearly uniform angular sampling with small perturbations. Moreover, the varied model trails Restormer by 1.15 dB on average in Table 1, with a larger gap on HNSCC. These results leave the claimed balance between flexibility and reconstruction quality insufficiently established.
- Generalization across acquisition settings is not sufficiently established. The quantitative comparison under variable-view settings mainly covers two input view counts on LoDoPaB-CT, while training uses nearly uniform angular sampling with small perturbations. Moreover, the varied model trails Restormer by 1.15 dB on average in Table 1. Broader evaluation is needed to establish the trade-off between acquisition flexibility and reconstruction quality.
Questions for authors
- Table 1 shows a substantial performance drop from the fixed to the varied model, particularly on HNSCC. What explains this difference, and how should it qualify the claim that a single model matches protocol-specific baselines?
- Can the authors provide results under substantially nonuniform angular sampling or on a fully held- out dataset? Such results would help distinguish generalization beyond the training distribution from flexibility within the sampled training settings.
- Can the authors isolate the contributions of the back-projection representation and geometric tokenizer through controlled ablations? A comparison with image-domain baselines using larger patches would also help establish whether the reported advantage is robust to spatial context.
- How were training and test data separated at the patient or volume level, and how many independent subjects are represented by the 32 test slices per dataset? Please provide uncertainty estimates for the main comparisons if available.
5 comments
Q2. Non-uniform angular sampling and fully held-out data:
Fully held-out data: We evaluate generalization beyond the training distribution at two levels.
-
Held-out datasets (new Appendix F.5, Table 11): We trained Restormer, our strongest baseline, and TomoTransformer (fixed) on a single dataset, either LoDoPaB-CT or KiTS23, using the same training protocol and settings as in Table 1 (an experiment also requested by another reviewer), and evaluated both models on all eight test datasets. On the respective training domains, the two models achieve comparable performance. However, on the seven datasets not seen during training, TomoTransformer generalizes substantially better, outperforming Restormer by up to 2.44 dB. While both models perform similarly in-domain when trained on a single dataset, TomoTransformer exhibits markedly stronger out-of-distribution generalization. This is particularly important in practical applications, where the scanned anatomy or scanner may differ from those encountered during training.
-
Held-out modality and samples (Section 5.5): The two real PXCT datasets are substantially different from all training data in terms of imaging physics (phase contrast), spatial scale (nanometer-sized pixels), and sample content (brain tissue and metal interconnects in a commercial processor chip). Without any fine-tuning, TomoTransformer improves upon FBP by 4.4 dB on the brain and 4.7 dB on the chip. Combining our new self-supervised test-time fine-tuning (Section 4.3) with RCS further improves the reconstructions, reaching 36.29 dB on the brain and 28.35 dB on the chip (Table 14). These experiments demonstrate generalization not only to entirely unseen datasets, but also to real experimental data with substantially different imaging physics, spatial scales, and sample content.
Non-uniform sampling (third paragraph of Appendix F.8 and Figure 11): We now evaluate the real chip scan at its acquired angles, which are non-uniformly spaced, with the angular density increasing sharply around 44° and 133° to as many as 29 views per degree, compared with a median of 8 views per degree. We preserved the original acquisition angles exactly and used 86 measured views as input. The resulting angular gaps range from 0.54° to 3.06°, rather than the uniform spacing of 2.1°. TomoTransformer achieves 28.3–28.5 dB, compared with 24.0–24.2 dB for weighted FBP using only the 86 input views. Moreover, the 617 views of the real brain scan are also not uniformly spaced; the baselines trained on uniform angles hallucinate texture and leave blocking artifacts there, while TomoTransformer, which conditions on each actual angle, gives the closest reconstruction to the reference (Figure 12 of the revised manuscript). These two experiments on real data demonstrate that TomoTransformer can handle realistic, non-uniform angular sampling patterns.
Q3. Ablations of the back-projection representation and the geometric tokenizer; image-domain baselines with larger patches:
- Geometric tokenizer (new Appendix F.3, Table 9): We replaced the geometric tokenizer and detokenizer with linear maps of the full back-projection patch and kept everything else the same. The geometric tokenizer is better on all eight datasets in both metrics, by 0.78 dB on average, with 1 M fewer parameters.
- Encoder-decoder design, MAE vs. BERT-style (new Appendix F.4, Table 10): TomoTransformer uses a masked-autoencoder design. We compare it with a BERT-style variant in which all transformer blocks process the full sequence of tokens from the start. Our experiments show that the two backbones perform almost identically. We retain the MAE-style design because its encoder attends only to measured tokens, reducing computation.
- Patch size (Appendix F.2, Table 8): We trained TomoTransformer and DRUNet under the fixed protocol with , changing only the patch size. For TomoTransformer, and show comparable performance (44.20 vs. 44.17 dB average PSNR), while and fall to 43.84 and 42.93 dB. The larger patches require more measured views to sample their back-projections without aliasing, so with 128 measured views their missing views are harder to predict. DRUNet shows the opposite trend: its average PSNR rises from 41.97 dB at to 44.34 dB at , as the larger image patches provide more spatial context. DRUNet with is comparable to TomoTransformer with on average.
Q4. Train/test split, number of subjects, and uncertainty:
Each of the 32 test slices per CT dataset is sampled independently from a randomly selected held-out test volume, with each dataset containing between 6 and 80 test volumes (now clarified in Section 5.1). The LoDoPaB-CT, FFHQ, and ImageNet test sets, in contrast, consist of collections of individual slices or images.
Uncertainty analysis (Appendix F.6, Table 12): We added a paired comparison against our strongest baseline, Restormer, and compute 95% confidence intervals using bootstrap resampling over test volumes. TomoTransformer (fixed, RCS) outperforms Restormer by an average of +0.60 dB (95% CI: [0.52, 0.68]) and achieves higher PSNR on 217 of the 256 test slices. The improvement is statistically significant on seven of the eight datasets. The exception is HNSCC, whose test set contains only six volumes: the confidence interval includes zero (+0.24 dB, 95% CI: [−0.15, 0.66]), indicating no statistically significant difference between the two methods on this dataset.
We thank the reviewer again; we believe these additions substantially strengthen the paper.
We thank the reviewer for the careful review and the precise questions. We answer them in order; all changes are shown in blue in the revised PDF.
The revised manuscript introduces three key additions (see the general response): (i) ray-consistent stitching (RCS), a novel inference method that enforces consistency among all patches intersected by the same ray, improving reconstruction quality without requiring retraining; (ii) fine-tuning of the pre-trained model, either supervised, using a small set of ground-truth images, or self-supervised, using the measured views from a single scan without requiring ground truth; and (iii) a second real experimental dataset, consisting of a commercial processor chip from a publicly available 4-nm PXCT dataset. We now address each point raised in the review in turn.
Q1 (and W1–W2):
TomoTransformer is designed as a foundation model for tomographic reconstruction: a single model that adapts to different numbers of measured views, sampling patterns, and detector resolutions, while generalizing to unseen anatomies and scanners. In contrast, protocol-specific models are trained for a single acquisition protocol. TomoTransformer (varied) is the model that provides this flexibility. We apply it directly to the real experimental data in Section 5.5, and our newly proposed self-supervised test-time fine-tuning (Section 4.3) further improves its performance on these data.
In addition, our new inference method, ray-consistent stitching (RCS), improves TomoTransformer (fixed) without retraining, increasing its average PSNR in Table 1 from 39.66 to 40.07 dB. With RCS, TomoTransformer (fixed) becomes the best-performing method across all datasets in both PSNR and SSIM.
At the protocol of Table 1 (64 → 256 views), the gap between TomoTransformer (varied, RCS) and Restormer is now small: 0.81 dB on average (38.66 vs. 39.47 dB). Most of this difference comes from a single dataset, HNSCC; across the other seven datasets, the average gap is only 0.22 dB. This remaining gap can be closed by supervised fine-tuning on very little data (Section 4.3). Using only 400 training slices (50 per dataset), fine-tuned TomoTransformer (varied, RCS) achieves an average of 39.54 dB / 0.934, outperforming Restormer (39.47 dB / 0.927) and all other image-domain baselines (last rows of Tables 1 and 2).
We now further investigate the trade-off between performance and flexibility. To demonstrate the advantages of a single foundation model across different sparsity levels, we evaluated all models of Table 1, without retraining, under a new acquisition protocol with 128 → 256 (new Table 3 in the revised manuscript). When presented with twice as many measured views, the fixed-protocol models improve by at most 2.2 dB on average, and some even suffer a performance degradation. For example, CTformer loses 1.0 dB on average, while Restormer drops from 44.73 to 42.05 dB on HNSCC. In contrast, TomoTransformer (varied), even without fine-tuning, achieves the best results on all eight datasets in both PSNR and SSIM, with an average PSNR of 43.43 dB compared to 40.82 dB for Restormer. These results demonstrate the advantage of TomoTransformer's flexibility: unlike protocol-specific models, it can adapt to changes in the acquisition protocol without retraining.