- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies test-time adaptation for image reconstruction, including sparse-view CT, using only measurements from the current scan. The proposed Routed Data Consistency (RDC) method freezes the pretrained network and optimizes its input or intermediate features to improve agreement between the reconstruction and the measurements. By exploiting the network’s learned image prior, RDC can also reduce errors in components not directly constrained by the measurements, thereby improving reconstruction quality.
Critical strengths
- The paper addresses a relevant problem in sparse-view reconstruction, where limited measurements and a lack of target-domain ground truth pose practical challenges. It investigates how to improve image components that are not directly constrained by the measurements.
- The paper is well organized, with detailed explanations and reasonably comprehensive experimental analysis.
- The method is broadly applicable. RDC retains the pretrained network and data-consistency loss, changing only the optimization variable without requiring model retraining. Optimizing features or inputs rather than all network weights also reduces adaptation costs and facilitates reuse across different tasks.
Critical weaknesses
- Limited novelty. Prior work has explored optimizing inputs or intermediate features through frozen pretrained networks, as well as using data consistency for test-time adaptation [1,2].
- The “label-free” claim requires qualification. Although per-scan adaptation does not use ground truth, the learning rate is selected using labeled data, and parts of the variable-selection procedure rely on a precalibrated error ratio.
- Experimental support for some key claims remains limited. Clinical evaluation includes only 18 scans, experiments with other operators primarily involve simulated 2D tasks, and the held-out-view selection strategy is mainly validated on a single backbone. These results are promising but do not establish robustness across different scanners, noise conditions, forward-model inaccuracies, and network architectures.
[1] Daras, Giannis, et al. "Intermediate layer optimization for inverse problems using deep generative models." arXiv preprint arXiv:2102.07364 (2021). [2] Darestani, Mohammad Zalbagi, Jiayu Liu, and Reinhard Heckel. "Test-time training can close the natural distribution shift performance gap in deep learning based compressed sensing." International conference on machine learning. PMLR, 2022.
Questions for authors
Is the optimal number of adaptation steps consistent across datasets and individual scans? Could an adaptive stopping criterion improve performance?
3 comments
Response 2/3: W3 — Clinical validation, noise, architectures and selection
W3. Experimental support
We address the items in order.
1. Clinical data. We agree that 18 cases are few. During the rebuttal period, we processed all remaining cases acquired at the same centre with the same scanner model. The pre-existing inclusion rule adds 43 cases, with preprocessing, hyperparameters and the 40-step protocol unchanged.
On the additional cases, RDC gains 1.11 dB over the frozen model and improves on it in all 43; weight-space TTT gains 0.71 dB. RDC exceeds weight-space TTT in 40/43 cases (, two-sided exact Wilcoxon). The corresponding figures for the original 18 are +1.06 dB, +0.64 dB and 16/18 wins ().
Splitting each cohort into two halves by agreement between measured and reference projections gives a positive RDC–weight margin in both halves: +0.43 and +0.35 dB on the additional cases, and +0.40 and +0.45 dB on the original cases. This checks whether the result depends on having the best reference images. The reference remains a deformed planning CT, so evaluation is restricted to the clinical CBCT field of view.
We also run weight-space TTT for 120 steps on all 61 cases. RDC at 40 steps remains higher by 0.15 dB on average, with 36/61 wins. On the original 18 cases, running both methods for 120 steps gives RDC a mean lead of 0.28 dB, with 13/18 wins. RDC at 120 steps has not been run on the additional 43 cases. These are specified-budget comparisons, not evidence of convergence.
The inclusion protocol, reference limitations and individual additional cases are reported in App. J.2, Tables R2–R3; the longer-budget control is Table R8.
2. Noise. The paper includes three operators with simulated measurement noise and clinical projections carrying real noise. Our original CT noise injection omitted TIGRE's projection normalisation, making its noise standard deviation 4–5 times stronger than intended. We corrected it using TIGRE's CTnoise.add (, Gaussian ), retaining the original run as a stronger-noise condition.
Under corrected noise, RDC leads on all six scans at 40 steps, with a mean margin of 0.64 dB. At 400 steps, after selecting RDC's longer-budget rate on scans 0–2, the weights lead by 0.09 dB overall and 0.20 dB on scans 3–5, excluded from selection (App. T, Tables 26, R5–R6).
MRI with 5% k-space noise is reported in Table 32. Super-resolution with 5% and 1% noise is reported in Tables 33 and 34, respectively. Under sufficiently strong super-resolution noise, every tested variable falls below the frozen model. These failures delimit the benefit of further measurement fitting.
3. Forward-model error. With geometric miscalibration of 0.5° and 1.0° per view, RDC reaches 27.95 and 27.58 dB; weight-space TTT under its published protocol reaches 27.29 and 27.17 dB (Table 5).
On walnut micro-CT with the calibrated operator, scatter, beam hardening and calibration error together change the comparison margin by 0.07 dB (Table 31). With an idealised circular-orbit operator, the corresponding forward-model control shows a 1.75 dB difference (App. J.1). Clinical projections additionally require a per-view gain term, applied identically to all variants (Table 8).
4. Architectures. The experiments cover eight reconstruction networks: two cone-beam backbones, three U-Nets, EDSR, the transformer-based SwinIR and NAFNet. For SwinIR and the inpainting U-Net, the variable choice was fixed and recorded before the runs (App. S).
5. Withheld-view selection. We agree that its validation scope should be explicit. It has been tested on one architecture, covering two sets of published weights (lung and dental priors) and two view counts. In two of the three settings, weight-space TTT is better. The criterion chooses correctly on all 18 scan–setting pairs, whereas the fitted residual succeeds on only 6/18 (Table 22). The separate relative-length check has been applied to a second architecture, including real walnut scans (App. U).
6. Scanners. Real projections come from a laboratory micro-CT and a clinical linear-accelerator CBCT system. RDC improves over frozen on both. The 61 clinical cases still share one centre, scanner model and anatomical site; increasing their number does not establish cross-centre generalisation. This limitation is explicit in App. Z.
Response 3/3: Q1 — Peak steps and adaptive stopping
Q1. Consistency of the best number of steps, and the role of adaptive stopping
Consistency across settings. The best number of steps is not universal. It depends on the noise level, network, optimised variable and learning rate. Within the six-scan cone-beam conditions, the individual peaks lie in the following ranges:
| Cone-beam CT condition | Individual peak steps | Interpretation |
|---|---|---|
| Noise-free | 210–330 | Broad plateau with little subsequent loss |
| Noise model reported by DeepSparse | 110–170 | RDC learning rate 0.02 |
| Original stronger-noise condition | 40–60 | Noise standard deviation 4–5 times stronger |
These are ranges of individual-scan peak steps. They should be distinguished from the peak of the mean trajectory. Without noise, the mean curve peaks at step 270; the average individual peak step is approximately 253. With corrected noise and RDC rate 0.02, the mean curve peaks at step 150; the average individual peak step is approximately 148.
The broader experiments show that a single stopping step does not describe all networks. Table 23 reports mean individual peak steps over 400-step runs. Its two-dimensional-network entries range from 49 to 400 steps, depending on the variable: for example, 49 for SwinIR weights, 173 for its input, 226 for the EDSR input, and 339–400 for the MRI variables. On real walnut projections, peaks at approximately 40–200 steps depend on the learning rate (App. J.1).
Forty steps is therefore a fixed compute budget, not a universal optimum or a convergence claim. It precedes most observed peaks and already obtains 87% of the gain reached at 200 steps in noise-free cone-beam CT. In a setting with an early peak, such as noisy MRI, the input variable peaks at step 38 on average and loses 0.04 dB by the fixed 40-step output.
Adaptive stopping. We used the existing 400-step cone-beam trajectories to compare two stopping procedures with the fixed 40-step result.
| Relative to fixed 40 steps | Noise-free | Reported noise | Original stronger noise |
|---|---|---|---|
| Stop each scan at its own peak | +0.42 dB | +0.31 dB | +0.04 dB |
| Discrepancy principle | Not applicable | +0.18 dB | −0.12 dB |
| Mean discrepancy-principle stop | — | Step 378 | Step 32 |
Stopping each scan at its own peak requires ground truth and provides an upper bound for any stopping rule within the recorded trajectory. It is used for diagnosis, not deployment.
The discrepancy principle is label-free given the measurement-noise variance: it stops when the mean squared measurement residual reaches that variance. Under the reported noise model, it improves PSNR by 0.18 dB but runs to step 378 on average, approximately nine times the fixed budget. Under the stronger noise, it stops at step 32 and gives a result 0.12 dB below the fixed 40-step output.
Thus, for these trajectories, the benefit under weaker noise comes from allowing substantially more computation. Under stronger noise, the fixed 40 steps is already near the peak, and this discrepancy-principle implementation does not improve on it. These results do not rule out other stopping criteria; they quantify the two comparisons tested. The analysis is included in App. T, Table R7.
We hope these responses address the concerns, and we are happy to clarify any further point.
Response 1/3: W1–W2 — Novelty and qualification of the label-free claim
We thank the reviewer for the positive assessment and for the specific comments. We respond point by point.
W1. Novelty relative to ILO and Darestani et al. (2022)
On ILO. Like ILO, RDC differentiates the loss through frozen network layers; the two differ in the question studied. The measurement loss cannot observe the operator's null space, and in our deterministic sparse-view setting (n/m=32), 81% of the unadapted frozen model's error lies in that space. We study how the choice of optimisation variable affects whether, and how much, minimising the measurement loss reduces this unobserved error component.
ILO's theory concerns the global optimum under Gaussian random measurements. Its method inverts an unconditional generator, uses hand-set constraint radii, and does not compare against weight-space adaptation. RDC starts from a measurement-conditioned reconstruction, optimises one activation or the input through the frozen reconstruction network, and analyses the resulting update for a fixed CT operator.
The analysis separates update alignment from relative length and identifies distinct failure modes. For example, an oversized variable on the second backbone increases null-space error; the relative-length check detects overshoot and guides narrowing of the variable (Table 14, App. U). These distinctions in setting, analysis, comparison and configuration constitute our contribution beyond using an intermediate representation as an optimisation variable.
On Darestani et al. (2022). This method is our main baseline: the same measurement-consistency objective applied to all network weights. We evaluated it under two protocols:
Its results are 0.03 to 0.56 dB higher under the matched protocol. The main table and convergence experiments compare against this stronger version (App. B). Our contribution is to explain why changing the optimised variable changes the null-space error removed, and when adapting the weights is the better choice (Tables 16, 25). The analysis does not guarantee that activation adaptation is always better.
W2. Qualifying the “label-free” claim
We agree that the label requirements should be more prominent. The one-time labelled learning-rate sweep was stated in the introduction and conclusion, but not in the abstract. The revised abstract now states both the labelled sweep and the error calibration used by the relative-length check.
Labels enter in two places:
1. Learning rate. For the primary cone-beam backbone at the 40-step budget, the rate is swept once on 10 labelled lung scans and confirmed on another 20. It is then reused unchanged on pelvis, abdomen and the other settings of that backbone, without target-domain labels for those transfers. Other backbones have their own one-time rate sweeps, described in their experimental sections.
2. Error ratio. The value s=0.81, the fraction of the frozen model's squared error lying in the null space, is calibrated on the backbone's own pretraining distribution: 10 AbdomenAtlas scans. It is then transferred to the target data. Across the primary-backbone conditions discussed in App. R, the measured share is 78% to 81%, and the resulting change in r is below 5%. This calibration is an offline label requirement, even though the check is applied without the target scan's ground truth.
The learning-rate sweep also shows that the advantage is not confined to one narrowly selected rate. On the same 10 scans, RDC gains 2.01, 2.48, 2.78, 2.88 and 2.66 dB at rates 0.005, 0.01, 0.02, 0.04 and 0.08. Weight-space TTT gains 2.00 dB at its best rate under the published protocol, approximately 0.06 dB more under the matched protocol (Table 28).
RDC therefore leads over the eightfold range from 0.01 to 0.08; at the lowest rate, the published-protocol results are approximately level. These results support robustness to the selected rate, while the revision explicitly acknowledges the labelled calibration involved.