acceptodds
Withdrawn from ICLR 2027

CT2X: An Empirical Study of Physics-Grounded World Models for Radiographic Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

CT-grounded virtual views can appear plausible despite incorrect three-dimensional alignment, leaving both their geometric reliability and the source of downstream gains unclear. We introduce CT2X, an empirical framework that makes this information flow explicit: patient-specific CT is a persistent 3D world, a source radiograph updates an explicit rigid pose, a known imaging action analytically propagates that pose, and a task renderer produces the target view. CT2X holds the estimated state fixed to compare direct state use with a virtual-landmark round trip, and matches renderer budgets to compare additional computation with an additional real radiograph. We combine controlled experiments on an 80-case CT cohort with evaluations on four public real-image benchmarks, using learned state correction, intensity-based registration, and learned 2D–3D correspondence under dataset-specific evaluation settings. On 224 view pairs from seven held-out patients, state correction reduces mean future translation error from 47.42 to 7.89 mm; direct use of the estimated pose and recovery from virtual landmarks yield numerically equivalent results across three seeds. For this geometric task, the gain is explained by state correction. In a six-object DeepFluoro comparison, adding a second real view reduces one failed object's mean target registration error from 25.91 to 3.05 mm under a matched maximum budget of 3,000 renderer evaluations; additional single-view optimization leaves that failure unresolved. Gains vary across objects and evaluation settings, and action perturbations cause large target errors even from reference source poses. Evaluations on public datasets use reference-derived actions and assess retrospective geometric reliability. Overall, the results show that virtual navigation depends on accurate state estimation and action fidelity; additional computation cannot always substitute for a new observation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Radiographic navigation, 2D–3D registration, Medical imaging

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UfeXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CT2X studies the geometric reliability of CT-based virtual X-ray views. It compares direct pose use with pose recovery from virtual landmarks and, under the same rendering budget, adds single-view optimization with an extra real radiograph. The experiments use a controlled CT cohort and four public datasets, with actions derived from the reference annotations in the public-data experiments.

Critical strengths

The paper clearly distinguishes acquired images from virtual outputs and reports unsuccessful cases rather than only successful registrations. The use of official XVR/RayEmb implementations and patient/object-level statistics is a strength.

Critical weaknesses
  • Limited novelty. CT rendering and pose refinement are already used in DiffPose [3] and XVR [1], while Cui et al. [4] and Cho et al. [5] study multi-view constraints and registration uncertainty, respectively. The main attribution result is also largely expected: exact landmarks are projected from an estimated pose and then used to recover the same pose. Under Eq. (7), the same rigid action preserves 3D landmark mTRE, as Table 6 notes. It is unclear what new understanding these checks provide beyond existing registration work.

  • Table 1B includes only six objects, and most of the mean improvement comes from one case. The 95% interval for the mean change includes zero (-11.49 to 0.94 mm), and the number of objects below 2 mm drops from 5/6 to 4/6. Since the budget comparison uses a fixed optimizer, it does not show whether the unresolved single-view error comes from insufficient information or the optimization procedure. The learned rejection model also misses all seven failures, but its input features, training procedure, and decision threshold are not described in Section 4.4 or Appendix D. Without these details or a comparison with existing registration-quality methods, its failure is difficult to interpret.

[1] Gopalakrishnan et al. Rapid patient-specific neural networks for X-ray to volume registration. Nature, 2026. [2] Shrestha et al. RayEmb: Arbitrary Landmark Detection in X-Ray Images Using Ray Embedding Subspace. ACCV, 2024. [3] Gopalakrishnan et al. Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering. CVPR, 2024. [4] Cui et al. Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints. ICRA, 2025. [5] Cho et al. Uncertainty Quantification in Image-based 2D/3D Registration and Its Relationship with Accuracy. International Journal of Computer Assisted Radiology and Surgery, 2025.

Questions for authors
  • Can the authors repeat the direct-versus-virtual comparison with a downstream method that uses rendered image pixels rather than exact projected landmarks?
  • Does the second-view benefit persist across different view pairs and alternative single-view restarts or optimizers under matched rendering budgets?
  • How is the learned rejection model trained and thresholded, and how does it compare with an existing registration-quality estimator?
  • What rotation center and composition order are used for action noise, and how do these perturbations relate to actual acquisition errors?
Reviewer_K4k2OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper evaluates a number of recent techniques for CT-based 2d-3d registration. A "world-model" evaluation is presented. In this framework, a number of retrospective computational studies is done to quantify evaluations on geometric accuracy, action fidelity and acquisition budget.

Critical strengths
  • The scope of the presented "world-model" evaluation can be valuable to the field, by defining and investigating better ways to empirically validate these methods.
Critical weaknesses
  • Lack of new methods. As said above, the paper can be valuable, but does not present any new methods. I don't think it fits the venue.
  • It is cumbersome to read, since the prose feels heavily influenced by LLM's (lots of words that are not saying much). The scheme's (Figs. 1 & 2) are very dense.
  • Very limited novel insights are presented in this paper. It is quite uninspiring.
Questions for authors
  • can you broaden the scope of this work? E.g., in the direction of (as you wrote yourself): "clinical acquisition savings and clinician benefit"
Reviewer_VWZzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyzes the task of medical rigid 2D to 3D registration in a world-model framing. The specific task is registering a preop CT to one or more images captured using a C-arm fluroscopy image. The authors perform analyses of attribution and reliability of existing methods and focus on four research questions, including whether a pose estimate or landmark registration affects performance (RQ1), evaluating single-view registration accuracy of existing methods (RQ2), whether more optimization iterations match the benefit of additional observations (RQ3), and the effect of random errors added to the C-arm motion on registration success and accuracy (RQ4).

Critical strengths

The authors follow good conventions for determining an analysis plan and success criteria prior to analyzing the data. They also candidly report failures in registration.

Critical weaknesses

The paper appears to be an evaluation of existing registration tools on small public benchmarks and does not present a new application or method. It is unclear whether there are new broadly-applicable insights or benchmarks. The world model framing also does not seem to meaningfully add to the analysis. In this setting, no dynamics are learned. There's no uncertainty about the belief of the hidden state because only a point estimate is used and there's no planning or choosing between actions.

RQ1 claims to be investigating attribution, (whether an improvement in performance comes from a better pose estimate or from a virtual view), but in practice appears to only demonstrate that C_geom is a correct implementation of the PnP algorithm using projected 2D landmarks.

Questions for authors

What could the RQ1 experiment show other than a correct PnP algorithm implementation? It appears that the CT landmarks are projected using the estimated pose without any noise.

What insights do the world model lens of analysis bring to the evaluations? How is this beneficial beyond a standard 2D/3D registration evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.