acceptodds
Withdrawn from ICLR 2027

AnyCT-Ω: Scaling Cross-View Representations for Heterogeneous Sparse-View CT Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

22 views

Abstract

Sparse-view computed tomography (CT) reconstruction aims to recover a volumetric attenuation field from a small number of X-ray projections. Existing learning-based methods are often optimized for a single dataset or anatomical domain, which limits their robustness under changes in anatomy, preprocessing, and acquisition statistics. We present AnyCT-, a unified framework for heterogeneous sparse-view CT reconstruction that adapts transferable cross-view representations from a pretrained VGGT- backbone to projection-based medical imaging. The model uses known CT geometry to align multi-view features with arbitrary three-dimensional query points and employs a content-routed residual mixture-of-experts head to adapt dense features across heterogeneous inputs. We further construct AnyCT-70K, a standardized corpus containing 73,610 processed samples from seven public CT sources covering thoracic, abdominal, pelvic, and musculoskeletal anatomy. Under a common joint-training protocol, AnyCT- outperforms the strongest baseline on the mixed AnyCT-70K benchmark by 1.50, 1.72, and 1.64 dB PSNR at 6, 8, and 10 views, respectively. The corresponding SSIM improvements are 3.76, 3.63, and 3.06 points. Additional experiments show positive transfer when increasing the number of training sources and demonstrate that the proposed model reaches competitive reconstruction quality with 30,000 optimization steps. These results suggest that geometry-guided adaptation of transferable cross-view representations is a promising direction for heterogeneous sparse-view CT reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Geometry-guided Point Query, Attenuation Prediction, DPT Dense Feature Decoder, MoE, VGGT-Omega

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_V3yZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents AnyCT-Ω, a framework for heterogeneous sparse-view CT reconstruction that adapts pretrained VGGT-Ω representations to CT attenuation-field prediction. Experiments under 6-, 8-, and 10-view settings demonstrate improvements over existing methods in heterogeneous joint training.

Critical strengths
  1. The paper addresses an important limitation of existing sparse-view CT reconstruction methods, which are typically developed for individual datasets or anatomical regions.
  2. AnyCT-Ω consistently outperforms the evaluated baselines on the mixed AnyCT-70K benchmark.
Critical weaknesses
  1. The proposed framework largely integrates existing techniques so that the novelty is limited.
  2. Although AnyCT-Ω achieves strong results under mixed-data training, Table 1 shows that the individually trained DeepSparse baseline generally outperforms AnyCT-Ω across individual datasets.
Questions for authors
  1. What is the essential technical contribution beyond combining pretrained VGGT-Ω features with existing geometry-guided point-wise reconstruction?
  2. Given that DeepSparse generally performs better on individual datasets in Table 1, can the authors clarify the practical advantages of the unified model and investigate whether the performance gaps can be reduced without source-specific fine-tuning?
  3. How does AnyCT-Ω perform under different projection geometries, realistic noise levels, and scanner-specific acquisition conditions?
Reviewer_rnjNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a method for heterogeneous sparse-view CT reconstruction, based on using a VGGT-\Omega backbone, a mixture of experts, and known CT geometry information. Results show modest improvement on seven public medical CT sources.

Critical strengths
  • The idea of extending VGGT-\Omega for a CT context is useful.
Critical weaknesses
  • The method consists of several different parts that all make very specific choices, but the paper does not give any motivation about why these specific parts were chosen and why specific choices were made. This makes it difficult to assess which parts are most important to make the method work, how sensitive the whole system is to specific choices, and whether there might not be better choices. For example, it is unclear why layers 4, 11, 17, and 23 were chosen (why these? why not more? why not fewer?). Another example is the choice for using Z^23 for the MoE, and not a different layer. Hyperparameter choices are given but not motivated (e.g. \lambda_g = 0.1, why? What happens if we change this?). The paper should make it much clearer what the motivation behind specific choices is, and why other choices might not be better (potentially supported by experimental evidence).
  • The overall explanation of the method is lacking a clear overview (again without a high-level motivation). The method uses VGGT-\omega as a backbone, but the paper does not explain this method. A clear explanation of this method, and how the proposed method extends it to the CT context, will make the entire paper clearer and easier to follow.
  • The improvements in metric is fairly modest. This is in principle not problematic, but combined with the many unmotivated hyperparameter choices (see above) this makes it difficult to assess whether the proposed method could actually improve reconstruction quality in practice. An additional weakness here is the presentation of the results, as figure 2 does not clearly show any differences between the methods.
Questions for authors
  • How were architectural choices and hyperparameter choices made? How sensitive is the accuracy to these specific choices?
  • What specific changes are in the proposed method that extend VGGT-\omega to enable application to CT?
  • How does your method compare with other existing baseline methods, including classical methods?
Reviewer_hiasOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AnyCT-Ω\Omega, a heterogeneous sparse-view CT reconstruction framework, and a mixture of 7 datasets, AnyCT-70K, used to train the proposed model. After a joint-training stage on AnyCT-70K, the proposed model shows robust performance on each target task without source-specific fine-tuning.

Critical strengths

The idea of leveraging VGGT-Ω\Omega, a 3D geometric reconstruction model capable of learning cross-view relationships and spatial structure, for X-ray-to-CT reconstruction is intuitive, given the shared need to reason across multiple views, despite the differences in image formation. The proposed adaptation is also relatively straightforward and simple.

The paper addresses a valid and important problem: the effectiveness of joint training on a mixture of different data sources in multi-view CT reconstruction. his research direction is important for the domain and could potentially create a new benchmark with a useful dataset. Some experiments, such as data-scaling, are insightful, and the proposed method exhibits strong performance in joint-training settings.

Critical weaknesses

While the reviewer found the motivation and problem valid, the paper lacks many critical experiments, which reduce the paper’s rigor and weaken many claims. Please carefully address these issues:

Insufficient architectural ablations:

  • The paper lacks ablation studies on other proposed components and design choices: ablation on other backbones and backbone sizes (given “scaling” in the title); cross-view attention (“Global attention” vs “Register attention”, which is never discussed).
  • The benefit of VGGT-Ω\Omega over simpler encoders remains unclear. A comparison with a U-Net, with and without cross-view attention, would help justify its use.
  • The authors claim that the natural-image pretrained backbone can be used for CT reconstruction. To validate this point, the authors should provide ablation studies on initialization strategies for visual backbones (natural image pretrained, random initialization, or medical image pretrained).

- Limited evidence for the MoE design: The selection and number of residual experts appear heuristic. Ablations on expert count/type, local versus global routing, and a parameter-matched non-MoE head are needed

Concerns about baseline comparison fairness:

  • Table 1 shows that AnyCT-Ω\Omega significantly underperforms DeepSparse. It is unclear whether AnyCT-Ω\Omega can surpass DeepSparse with extra source-specific training.
  • Table 2 shows that other models underperform the proposed AnyCT-Ω\Omega when using the same training dataset (AnyCT-70K). However, the authors do not provide the details of the training settings for these models. It also raises the question of how the authors train these models and whether the comparison is fair. Please provide the detailed training settings. If the authors reused the training settings of AnyCT-Ω\Omega, the reported results may not be their optimal performance.

Concerns about the data-scaling experiments:

  • Contribution 3 claims that AnyCT-Ω\Omega benefits from more diverse training sources, but in Table 3 adding sources also adds data (e.g., LUNA16 alone has 738 training samples vs ~56K for the four-source mixture), so it is unclear whether the gain comes from diversity or data size. A sample-count-matched control, balanced vs proportional sampling, and leave-one-source-out would separate these factors.

Qualitative Results:

  • The qualitative results in Figure 2 do not show the improvement of AnyCT-Ω\Omega over DeepSparse. The panels are also very small and carry no method or dataset labels, so the figure is hard to read even when zoomed in. A better visualization and discussion are needed (like highlight or enlarge RoI)
Questions for authors

Please refer to the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.