- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when coordinate input is helpful for learned projection-to-volume models. The main idea is that coordinates work like a population prior, so they help a lot when the measurements are ambiguous (for example single view, where the depth along the ray is unknown), but they become not useful or even harmful when the measurements are enough. The authors separate the effect of position from other spatial components like calibrated lifting, attention and decoder context, and also check which direction of position is more useful and in which conditions it helps. They validate this mainly on 3D lung segmentation from simulated X-ray projections, including one held-out cohort with locked protocol, and on some synthetic inverse problems. They propose a simple gate that changes the coordinate strength by acquisition geometry, which works for very sparse and broad views but not for intermediate ones.
Critical strengths
- The main OFF/FULL reversal is supported better than usual for this kind of empirical study. The LCTSC protocol is fixed before training, the labels are manual contours, the scan box is not cropped by label, and the sign holds in all three seeds with paired-patient intervals clearly away from zero. This is important for my assessment, because it shows the core observation is not only an artifact of the reused development cohorts or the automated labels.
Critical weaknesses
-
It is not clear for me that the reversal is a property of positional input under broad coverage, rather than a property of training one model on mixed coverage. All clinical OFF/FULL comparisons train a single model on 1 to 12 views together, and for the sparse samples the coordinate is the main localization cue. The scalar gate result actually points to this direction. With the same mixed training, removing position only for broad inputs recovers OFF-level broad accuracy, and FULL also fits the twelve-view training cases worse than OFF (0.838 vs 0.859, Table 15). Without OFF/FULL models trained on broad coverage only, the claim that coordinate benefit reverses as coverage increases is not yet separated from this training setup.
-
The paper has no prior-only reference, for example a coordinate-only model without image input or simply the training mean mask. I think this matters a lot, because on LCTSC the FULL model changes very little with more views (0.672 at one view to 0.740 at twelve, while OFF goes from 0.364 to 0.803). This suggests FULL mostly outputs a population shape and makes little use of additional measurements. If this is the case, both the sparse gain and the broad penalty can be explained by a simple prior-versus-evidence tradeoff, and the depth, synthetic and context analyses add limited understanding beyond it. The proposed gate is also closer to a negative result. With the cutoff 0.25, the formula in Appendix D gives a zero gate exactly at a 60 degree pair, so the intermediate-span failure is expected by construction rather than a new finding.
Questions for authors
-
How is the patient box, and therefore c0 and the detector scale, defined for LIDC-IDRI and TotalSegmentator? Appendix H says LCTSC uses the full scan box without a label-defined crop, so I want to check whether the development cohorts use a label-derived box. This affects how I read the development-only results (gate, depth vs plane, context).
-
Section 5.1 says the full and no-coordinate arms were not repeated in the directional replication. Were they run in the original single-seed directional experiment? If so, please report them, because without OFF it is hard to tell whether depth helps or detector-plane coordinates hurt.
-
Could you train OFF and FULL on broad coverage only (for example four spread or twelve views) and report whether the broad penalty still remains?
-
Could you report one-view and twelve-view Dice of a coordinate-only model without image input, or of the training mean mask, at least on LIDC-IDRI and LCTSC?
-
The related work misses sparse-view CBCT methods that combine calibrated back-projection queries with coordinate encodings, such as DIF-Net (MICCAI 2023) and C2RV (CVPR 2024). Discussing them would help to position the study.
5 comments
Response to Reviewer duBN
Q: The paper has no prior-only reference, for example a coordinate-only model without image input or simply the training mean mask. On LCTSC the FULL model changes very little with more views (0.672 at one view to 0.740 at twelve, while OFF goes from 0.364 to 0.803). This suggests FULL mostly outputs a population shape. Could you report one-view and twelve-view Dice of a coordinate-only model or of the training mean mask?
A: We computed the mean-mask reference and trained a coordinate-only model. The training mean mask scores 55.37% Dice on the LCTSC test patients (95% interval 51.28–59.51%) and 72.59% on the LIDC-IDRI development patients (69.43–75.49%); it is the same at every view count.
-
In the original runs, FULL is above the mean mask at one view (67.19% on LCTSC, 80.84% on LIDC-IDRI) and at twelve views (73.98% and 91.36%). The matched coordinate-only comparison below further supports a benefit from image conditioning at the long budget, although these comparisons do not quantify how much FULL relies on patient-specific projection evidence.
-
One-view OFF is below the mean mask on both cohorts (36.37% and 49.74%). Without coordinates, one view therefore yields less than average anatomy, which is consistent with your reading. The comparison does not establish what fraction of the gain is a shape prior.
-
On LCTSC, FULL gains 6.8 points from one to twelve views while OFF gains 44.0, which fits your prior-versus-evidence reading. On LIDC-IDRI FULL gains 10.5 points.
Section 5.1 of the revised paper now reports both image-free references. Figure 1 plots this contrast on the LCTSC test patients, and Figure 4 shows the mean mask and a coordinate-only model next to the OFF and FULL predictions for two of them. The coordinate-only model receives no image and scores 57.46% on LCTSC and
75.12% on LIDC-IDRI at every view count. In the matched long-budget runs (Appendix I.2, Table 17), FULL exceeds it at one view (68.16% and 80.37%) and at twelve views (72.53% and 91.42%), while one-view OFF is below it (38.90% and 53.47%). At the short budget, LIDC-IDRI one-view FULL (68.99%) is below both references, so the result depends on the training budget. The coordinate-only model bypasses the image branch entirely. The encoder and lifting operations are skipped; their parameters remain stored but frozen. We do not feed zero-valued projections through the encoder. Patient-centered coordinates pass through the coordinate MLP directly to the 3D decoder, and only these two components are trained. Training follows the same patient order, mixed-view schedule, loss and optimizer as the corresponding local FULL model. Both use three matched seeds, with checkpoints at 2,250 and 11,250 updates. The coordinate-only prediction is independent of view count, but its physical coordinates still provide patient-box size information. Appendix I.2 states this.
Q: The proposed gate is closer to a negative result. With the cutoff 0.25, the formula in Appendix D gives a zero gate exactly at a 60 degree pair, so the intermediate-span failure is expected by construction rather than a new finding.
A: We agree, and the revised paper presents the gate as a test with a negative outcome between the endpoints. The contribution is retitled “A test of geometry-based gating”. Appendix D states that the gate reaches zero at 60° and treats layouts identically even though their measured content differs. Section 5.3 now says that the 60° gate is zero, so the rule removes the coordinate input in a layout where FULL (0.853) outperforms OFF (0.760), and that SCALAR (0.761) empirically scores close to OFF. Appendix D adds that this closeness is an empirical result and not a consequence of the rule alone, because SCALAR and OFF are trained separately. Most of the gate material has moved to Appendix D; the main text keeps the definition (Section 3, Equation 4) and one results paragraph (Section 5.3).
Response to Reviewer duBN
Q: How is the patient box, and therefore c0 and the detector scale, defined for LIDC-IDRI and TotalSegmentator? Appendix H says LCTSC uses the full scan box without a label-defined crop, so I want to check whether the development cohorts use a label-derived box.
A: We audited the patient boxes by replaying the stored geometry (Appendix I.7). For LIDC-IDRI (215 cases) and LCTSC (60 cases, from the native CT headers), the patient box is the full CT volume, not a crop derived from the organ masks, and c0 is the centre of that box. The maximum corner discrepancy in the replay is
0.00002 mm for LIDC-IDRI and 0 mm for LCTSC. TotalSegmentator also uses the full CT-volume box, not a box derived from segmentation labels. Its bounds are calculated from the CT dimensions and spatial affine. Labels are used for organ-volume eligibility checks when selecting scans, but not to define the box. The audit of this cohort covers 248 processed records, of which six were replayed from native data. For all three cohorts, we project the eight CT-volume corners onto the detector and choose one pixel spacing large enough to contain them across the specified angles, with a 10% margin. The same spacing is used in both detector directions and for every view of that patient. LIDC-IDRI uses the original 12 angles, spaced 15° apart, and this spacing is retained when the bank is expanded to 24 views. LCTSC and TotalSegmentator use all 24 angles, spaced 7.5° apart. The detector scale therefore depends on the CT volume and imaging geometry, not the segmentation labels. Section 3 now states the box rule, and Appendix I.7 states both the box and the detector-scale rules.
Q: Section 5.1 says the full and no-coordinate arms were not repeated in the directional replication. Were they run in the original single-seed directional experiment? If so, please report them, because without OFF it is hard to tell whether depth helps or detector-plane coordinates hurt.
A: We have now run both arms. Section 5.2 and Appendix I.4 (Table 19) add a matched OFF model and a model with all three acquisition-frame coordinates to the directional comparison on LIDC-IDRI, with three seeds. At source acquisitions, depth improves over OFF by 22.93 points [21.62, 24.14], detector-plane coordinates by 13.25 [11.93, 14.49], and both together by 23.42. Depth therefore helps, and detector-plane coordinates also help, by less; they do not hurt relative to OFF. The anchors have not been repeated on TotalSegmentator. To answer your question directly: yes, the original seed-0 run included both arms. On the 37 LIDC-IDRI development patients after 11,250 updates, using heart-excluded lung Dice, OFF scores 0.5984 (source), 0.5849 (held rotations) and 0.5917 (offset), and the model with full acquisition coordinates scores 0.8320, 0.8297 and 0.8256. Here FULL uses acquisition-frame coordinates, not the world-coordinate input of the main FULL model, and these are original single-seed results, not three-seed averages. Appendix I.4 now reports them.
Q: The related work misses sparse-view CBCT methods that combine calibrated back-projection queries with coordinate encodings, such as DIF-Net (MICCAI 2023) and C2RV (CVPR 2024).
A: Thank you for these references. We have added DIF-Net and C2RV to Section 2 (Related Work) and cite them in the first paragraph of the introduction. Both are close to our pipeline: they project each 3D query point onto the views and regress the output from the image features sampled there. C2RV additionally interpolates multi-scale 3D feature volumes and fuses the views with cross-attention. One clarification from our reading: in both papers the query point is used only for projection and interpolation, and its coordinate is not given to the network as an input feature (DIF-Net, Section 2.2; C2RV, Section 3, Eqs. 1, 2, 7 and 9). In our terms both correspond to the OFF setting, and both are evaluated at 6 to 10 views. We also tested the coordinate input in our own adaptation of DIF-Net (Section 5.4, Table 5). There it improves Dice at every view count, including twelve views, so the broad-view penalty of our pipeline does not appear in that architecture. The cited method that does add a positional encoding of the coordinate, PerX2CT, reports a gain from it at two views, which agrees with our sparse-view result. We will gladly correct this reading if we have missed a coordinate input in either method.
Response to Reviewer duBN
Q: It is not clear that the reversal is a property of positional input under broad coverage, rather than a property of training one model on mixed coverage. All clinical OFF/FULL comparisons train a single model on 1 to 12 views together. Could you train OFF and FULL on broad coverage only (for example four spread or twelve views) and report whether the broad penalty still remains?
A: We ran this experiment, and it supports your reading. We trained OFF and FULL on twelve views only, on both cohorts, with the same optimizer settings and update budget as the mixed-view models. FULL−OFF at twelve views, 11,250 updates, three-seed means on the evaluation patients, with patient-bootstrap 95% intervals:
LCTSC, mixed-view training: −11.28 Dice points [−14.44, −8.35]; the difference on the training patients is +0.51.
LCTSC, twelve-view-only training: −0.10 [−0.44, +0.22].
LIDC-IDRI, mixed-view training: −1.43 [−2.26, −0.83]; the difference on the training patients is −0.10.
LIDC-IDRI, twelve-view-only training: −0.02 [−0.07, +0.03].
The dense-view penalty is therefore largely tied to mixed-view training. We have changed the claim accordingly: a model trained across view counts with a coordinate input loses dense-view accuracy; we do not claim that coordinates are harmful under broad coverage as such.
One limitation remains. At equal update counts, twelve-view-only training sees five times as many twelve-view examples and 2.22 times as many projections, so this control changes exposure as well as the training distribution. It does not isolate mixed-view interference. Section 5.1, the abstract and the conclusion of the revised paper now report this control and state the narrower claim. We therefore also matched twelve-view exposure instead of updates, by comparing mixed-view models at 11,250 updates with twelve-view-only models at 2,250. The regime difference remains on LCTSC (+11.45 points [8.46, 14.71]) but not on LIDC-IDRI (−0.57 [−1.30, 0.25]). Exposure alone therefore does not explain the LCTSC contrast, and the cause remains unresolved. Table 2 in Section 5.1 and Appendix I.1 give these results. A single-seed DIF-Net adaptation trained on mixed views shows no twelve-view penalty (Table 5), so the effect is specific to our pipeline.