acceptodds
Under review as a conference paper at ICLR 2027

Data Consistency Reaches the Null Space

10 views

Abstract

Sparse-view reconstruction is hard because most of the error lives where the measurements cannot see it, the null space of the forward operator. A measurement-consistency loss minimised over the image cannot move that component. We ask whether the same loss reaches the null space when it is minimised through a network instead, and what limits it there. Routed data consistency (RDC) frees one internal activation of a released reconstructor and fits the measurements through the frozen decoder behind it, with every weight fixed. How much it removes from the null space is bounded by one alignment, the cosine cos θ between its null-space update and the null-space error. For a fixed operator that alignment is set by the pretrained network and its target, not by the scan, and it caps the removable fraction at 1 − sin θ. With the relative length r of the update it forms an identity that ranks candidate variables from their decompositions alone, and a label-free test on withheld views defers to the weights where the frozen mapping is wrong. On eight-view cone-beam CT, forty steps finish 1.72 dB above the range oracle, the bound that no fit of the image can exceed, and surpass a budget-matched supervised finetune given forty-eight labelled volumes. On 18 clinical scans RDC leads weight-space adaptation in 16. Across six networks on four operators the identity predicts the better variable in 110 of 115 comparisons.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, inverse problems, null space, data consistency, image reconstruction, pretrained reconstructors, tangent kernel, label-free adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cMFrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies test-time adaptation for image reconstruction, including sparse-view CT, using only measurements from the current scan. The proposed Routed Data Consistency (RDC) method freezes the pretrained network and optimizes its input or intermediate features to improve agreement between the reconstruction and the measurements. By exploiting the network’s learned image prior, RDC can also reduce errors in components not directly constrained by the measurements, thereby improving reconstruction quality.

Critical strengths
  1. The paper addresses a relevant problem in sparse-view reconstruction, where limited measurements and a lack of target-domain ground truth pose practical challenges. It investigates how to improve image components that are not directly constrained by the measurements.
  2. The paper is well organized, with detailed explanations and reasonably comprehensive experimental analysis.
  3. The method is broadly applicable. RDC retains the pretrained network and data-consistency loss, changing only the optimization variable without requiring model retraining. Optimizing features or inputs rather than all network weights also reduces adaptation costs and facilitates reuse across different tasks.
Critical weaknesses
  1. Limited novelty. Prior work has explored optimizing inputs or intermediate features through frozen pretrained networks, as well as using data consistency for test-time adaptation [1,2].
  2. The “label-free” claim requires qualification. Although per-scan adaptation does not use ground truth, the learning rate is selected using labeled data, and parts of the variable-selection procedure rely on a precalibrated error ratio.
  3. Experimental support for some key claims remains limited. Clinical evaluation includes only 18 scans, experiments with other operators primarily involve simulated 2D tasks, and the held-out-view selection strategy is mainly validated on a single backbone. These results are promising but do not establish robustness across different scanners, noise conditions, forward-model inaccuracies, and network architectures.

[1] Daras, Giannis, et al. "Intermediate layer optimization for inverse problems using deep generative models." arXiv preprint arXiv:2102.07364 (2021). [2] Darestani, Mohammad Zalbagi, Jiayu Liu, and Reinhard Heckel. "Test-time training can close the natural distribution shift performance gap in deep learning based compressed sensing." International conference on machine learning. PMLR, 2022.

Questions for authors

Is the optimal number of adaptation steps consistent across datasets and individual scans? Could an adaptive stopping criterion improve performance?

3 comments
AuthorsOpenReview ↗

Response 1/3: W1–W2 — Novelty and qualification of the label-free claim

We thank the reviewer for the positive assessment and for the specific comments. We respond point by point.

W1. Novelty relative to ILO and Darestani et al. (2022)

On ILO. Like ILO, RDC differentiates the loss through frozen network layers; the two differ in the question studied. The measurement loss cannot observe the operator's null space, and in our deterministic sparse-view setting (n/m=32n/m=32), 81% of the unadapted frozen model's error lies in that space. We study how the choice of optimisation variable affects whether, and how much, minimising the measurement loss reduces this unobserved error component.

ILO's theory concerns the global optimum under Gaussian random measurements. Its method inverts an unconditional generator, uses hand-set constraint radii, and does not compare against weight-space adaptation. RDC starts from a measurement-conditioned reconstruction, optimises one activation or the input through the frozen reconstruction network, and analyses the resulting update for a fixed CT operator.

The analysis separates update alignment from relative length and identifies distinct failure modes. For example, an oversized variable on the second backbone increases null-space error; the relative-length check detects overshoot and guides narrowing of the variable (Table 14, App. U). These distinctions in setting, analysis, comparison and configuration constitute our contribution beyond using an intermediate representation as an optimisation variable.

On Darestani et al. (2022). This method is our main baseline: the same measurement-consistency objective applied to all network weights. We evaluated it under two protocols:

  • Its published protocol, with two views held out for early stopping.
  • A matched protocol, with all measurements in the loss, the final iterate reported, and only the optimised variable changed.

Its results are 0.03 to 0.56 dB higher under the matched protocol. The main table and convergence experiments compare against this stronger version (App. B). Our contribution is to explain why changing the optimised variable changes the null-space error removed, and when adapting the weights is the better choice (Tables 16, 25). The analysis does not guarantee that activation adaptation is always better.

W2. Qualifying the “label-free” claim

We agree that the label requirements should be more prominent. The one-time labelled learning-rate sweep was stated in the introduction and conclusion, but not in the abstract. The revised abstract now states both the labelled sweep and the error calibration used by the relative-length check.

Labels enter in two places:

1. Learning rate. For the primary cone-beam backbone at the 40-step budget, the rate is swept once on 10 labelled lung scans and confirmed on another 20. It is then reused unchanged on pelvis, abdomen and the other settings of that backbone, without target-domain labels for those transfers. Other backbones have their own one-time rate sweeps, described in their experimental sections.

2. Error ratio. The value s=0.81s=0.81, the fraction of the frozen model's squared error lying in the null space, is calibrated on the backbone's own pretraining distribution: 10 AbdomenAtlas scans. It is then transferred to the target data. Across the primary-backbone conditions discussed in App. R, the measured share is 78% to 81%, and the resulting change in rr is below 5%. This calibration is an offline label requirement, even though the check is applied without the target scan's ground truth.

The learning-rate sweep also shows that the advantage is not confined to one narrowly selected rate. On the same 10 scans, RDC gains 2.01, 2.48, 2.78, 2.88 and 2.66 dB at rates 0.005, 0.01, 0.02, 0.04 and 0.08. Weight-space TTT gains 2.00 dB at its best rate under the published protocol, approximately 0.06 dB more under the matched protocol (Table 28).

RDC therefore leads over the eightfold range from 0.01 to 0.08; at the lowest rate, the published-protocol results are approximately level. These results support robustness to the selected rate, while the revision explicitly acknowledges the labelled calibration involved.

AuthorsOpenReview ↗

Response 2/3: W3 — Clinical validation, noise, architectures and selection

W3. Experimental support

We address the items in order.

1. Clinical data. We agree that 18 cases are few. During the rebuttal period, we processed all remaining cases acquired at the same centre with the same scanner model. The pre-existing inclusion rule adds 43 cases, with preprocessing, hyperparameters and the 40-step protocol unchanged.

On the additional cases, RDC gains 1.11 dB over the frozen model and improves on it in all 43; weight-space TTT gains 0.71 dB. RDC exceeds weight-space TTT in 40/43 cases (p=5×10−10p=5\times10^{-10}, two-sided exact Wilcoxon). The corresponding figures for the original 18 are +1.06 dB, +0.64 dB and 16/18 wins (p=0.0002p=0.0002).

Splitting each cohort into two halves by agreement between measured and reference projections gives a positive RDC–weight margin in both halves: +0.43 and +0.35 dB on the additional cases, and +0.40 and +0.45 dB on the original cases. This checks whether the result depends on having the best reference images. The reference remains a deformed planning CT, so evaluation is restricted to the clinical CBCT field of view.

We also run weight-space TTT for 120 steps on all 61 cases. RDC at 40 steps remains higher by 0.15 dB on average, with 36/61 wins. On the original 18 cases, running both methods for 120 steps gives RDC a mean lead of 0.28 dB, with 13/18 wins. RDC at 120 steps has not been run on the additional 43 cases. These are specified-budget comparisons, not evidence of convergence.

The inclusion protocol, reference limitations and individual additional cases are reported in App. J.2, Tables R2–R3; the longer-budget control is Table R8.

2. Noise. The paper includes three operators with simulated measurement noise and clinical projections carrying real noise. Our original CT noise injection omitted TIGRE's projection normalisation, making its noise standard deviation 4–5 times stronger than intended. We corrected it using TIGRE's CTnoise.add (I0=105I_0=10^5, Gaussian σ=10\sigma=10), retaining the original run as a stronger-noise condition.

Under corrected noise, RDC leads on all six scans at 40 steps, with a mean margin of 0.64 dB. At 400 steps, after selecting RDC's longer-budget rate on scans 0–2, the weights lead by 0.09 dB overall and 0.20 dB on scans 3–5, excluded from selection (App. T, Tables 26, R5–R6).

MRI with 5% k-space noise is reported in Table 32. Super-resolution with 5% and 1% noise is reported in Tables 33 and 34, respectively. Under sufficiently strong super-resolution noise, every tested variable falls below the frozen model. These failures delimit the benefit of further measurement fitting.

3. Forward-model error. With geometric miscalibration of 0.5° and 1.0° per view, RDC reaches 27.95 and 27.58 dB; weight-space TTT under its published protocol reaches 27.29 and 27.17 dB (Table 5).

On walnut micro-CT with the calibrated operator, scatter, beam hardening and calibration error together change the comparison margin by 0.07 dB (Table 31). With an idealised circular-orbit operator, the corresponding forward-model control shows a 1.75 dB difference (App. J.1). Clinical projections additionally require a per-view gain term, applied identically to all variants (Table 8).

4. Architectures. The experiments cover eight reconstruction networks: two cone-beam backbones, three U-Nets, EDSR, the transformer-based SwinIR and NAFNet. For SwinIR and the inpainting U-Net, the variable choice was fixed and recorded before the runs (App. S).

5. Withheld-view selection. We agree that its validation scope should be explicit. It has been tested on one architecture, covering two sets of published weights (lung and dental priors) and two view counts. In two of the three settings, weight-space TTT is better. The criterion chooses correctly on all 18 scan–setting pairs, whereas the fitted residual succeeds on only 6/18 (Table 22). The separate relative-length check has been applied to a second architecture, including real walnut scans (App. U).

6. Scanners. Real projections come from a laboratory micro-CT and a clinical linear-accelerator CBCT system. RDC improves over frozen on both. The 61 clinical cases still share one centre, scanner model and anatomical site; increasing their number does not establish cross-centre generalisation. This limitation is explicit in App. Z.

AuthorsOpenReview ↗

Response 3/3: Q1 — Peak steps and adaptive stopping

Q1. Consistency of the best number of steps, and the role of adaptive stopping

Consistency across settings. The best number of steps is not universal. It depends on the noise level, network, optimised variable and learning rate. Within the six-scan cone-beam conditions, the individual peaks lie in the following ranges:

Cone-beam CT conditionIndividual peak stepsInterpretation
Noise-free210–330Broad plateau with little subsequent loss
Noise model reported by DeepSparse110–170RDC learning rate 0.02
Original stronger-noise condition40–60Noise standard deviation 4–5 times stronger

These are ranges of individual-scan peak steps. They should be distinguished from the peak of the mean trajectory. Without noise, the mean curve peaks at step 270; the average individual peak step is approximately 253. With corrected noise and RDC rate 0.02, the mean curve peaks at step 150; the average individual peak step is approximately 148.

The broader experiments show that a single stopping step does not describe all networks. Table 23 reports mean individual peak steps over 400-step runs. Its two-dimensional-network entries range from 49 to 400 steps, depending on the variable: for example, 49 for SwinIR weights, 173 for its input, 226 for the EDSR input, and 339–400 for the MRI variables. On real walnut projections, peaks at approximately 40–200 steps depend on the learning rate (App. J.1).

Forty steps is therefore a fixed compute budget, not a universal optimum or a convergence claim. It precedes most observed peaks and already obtains 87% of the gain reached at 200 steps in noise-free cone-beam CT. In a setting with an early peak, such as noisy MRI, the input variable peaks at step 38 on average and loses 0.04 dB by the fixed 40-step output.

Adaptive stopping. We used the existing 400-step cone-beam trajectories to compare two stopping procedures with the fixed 40-step result.

Relative to fixed 40 stepsNoise-freeReported noiseOriginal stronger noise
Stop each scan at its own peak+0.42 dB+0.31 dB+0.04 dB
Discrepancy principleNot applicable+0.18 dB−0.12 dB
Mean discrepancy-principle stop—Step 378Step 32

Stopping each scan at its own peak requires ground truth and provides an upper bound for any stopping rule within the recorded trajectory. It is used for diagnosis, not deployment.

The discrepancy principle is label-free given the measurement-noise variance: it stops when the mean squared measurement residual reaches that variance. Under the reported noise model, it improves PSNR by 0.18 dB but runs to step 378 on average, approximately nine times the fixed budget. Under the stronger noise, it stops at step 32 and gives a result 0.12 dB below the fixed 40-step output.

Thus, for these trajectories, the benefit under weaker noise comes from allowing substantially more computation. Under stronger noise, the fixed 40 steps is already near the peak, and this discrepancy-principle implementation does not improve on it. These results do not rule out other stopping criteria; they quantify the two comparisons tested. The analysis is included in App. T, Table R7.

We hope these responses address the concerns, and we are happy to clarify any further point.

Reviewer_R5XZOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper shows that in inverse problem solving, for example CT reconstruction, a pretrained network can be used to further improve the reconstruction quality in form of test-time adaptation. The authors analyze the minimium error bound and deduce the necessary conditions on updates that reduces null space error. The way of achieving that is by routed data consistency (RDC), which keep rest of the network frozen and optimize data consistency on certain intermediate features. Results show that RDC improves image quality metrics and reduces error in null space.

Critical strengths

This is a interesting topic. The inverse problem community would benefit from an approach that reduces error in the null space.

The paper covers a variety of ranges on inverse problem modalities, on every single one it shows improvement using their RDC.

Critical weaknesses

The paper is difficult to read. All texts read abstract and highly AI-style, and it doesn't read like a standard scientific paper. I would suspect at least the writing is actually done by AI, different than the authors claimed. There are certain ways of phrasing read rhetorical and hollow. There are also many repeated claim about the theory, but the connection from the theory to the method is not clearly established.

The theory part and the actual method feel like quite apart for me. The paper first illustrates the minimum error (Lemma 1), which can serve as a starting point.

Proposition 1 states that fitting measurement via gradient descent can alert the null space part (which is also intuitive), not the null space error is guaranteed be reduced. The following experiments and design uses adam, which deviates bit and need further exploration.

Section 4.2 further deduces the necessary conditions on reduction of null space error (move from initial reconstruction towards ground truth).

After those analysis, the rolled out method is just optmization certain intermediate feature over data consistency, which is not guaranteed by any analysis that it will reduce the null space error. The paper spends quite some space coming to the necessary conditions on null space error reduction, yet the followed algorithm has little to do with the conditions and cannot garanutee the conditions to be fulfilled.

I feel like there is a overstatement in the analysis as derivation of RDC, where the method itself is largely emprical.

Questions for authors

There are in total three variants in Table 1, the results show that certain varaiants can be better than the others in specific cases. Do authors have an explainations why alerting different features would cause those differences?

Why optimize it for too many steps would cause PSNR drop? How would the authors choose the steps in practice?

The cosine alignment is just a measure that can be used to see the direction, only when ground truth is present. How would the author determine if the improvement or null space error reduction is true in practice where ground truth is not avaible?

What would happen if the measurements are not noiseless anymore?

How do the authors compute the null and range operators effectively and efficiently? It mentions onjugate gradients somewhere, which formula does it exactly solve?

For CT image quality, sometime classical MBIR with TV achieves better metrics than deep learning solutions, just becasue they have more smooth yet blur details, but this would give better overal fit. Does this method also do similar, solving a smooth solution?

How do the authors garanutee better qulity metrics mean lower null space errors?

What is the computational cost on the selection procedure?

minor: the notations can be better unified, for example some places use range while the other use R\mathcal{R} (no defined)

10 comments
AuthorsOpenReview ↗

Response 1/5: W1–W2 — Presentation and the link between theory and method

We thank the reviewer for the detailed comments and questions, and for the assessment that the field of inverse problems would benefit from methods that can reduce null-space error. We respond point by point.

W1. Presentation

We agree that the presentation needs to be clearer and more direct. As described in the AI-use statement, the authors wrote the text in English and used large language models to polish grammar, wording and sentence structure. We take responsibility for the resulting presentation. In the revised version, the authors have rewritten the affected passages, replaced rhetorical phrasing by direct statements, defined the terminology, added a notation table (Table R1), and gathered the link between theory and method in App. R.

W2. Link between theory and method

We agree with both of the reviewer's assessments: Proposition 1 shows only that the null-space component can be changed, and the proposed method carries no guarantee that the null-space error decreases. We explain why no unconditional guarantee exists, what our analysis provides, and where each choice in the method comes from.

1. Without a prior, no unconditional measurement-only guarantee. Images satisfying Ax=yAx=y differ only in their null-space components. For a fixed pretrained network, a method using the same measurements returns the same null-space update dnd_n. If dn≠0d_n\ne0, there is another ground truth x′x' consistent with the measurements, with null-space component PNx^0−dnP_N\hat{x}_0-d_n. Relative to x′x', the null-space error is ∥dn∥\|d_n\| before the update and ∥2dn∥\|2d_n\| afterwards. Thus a nonzero update cannot improve every image consistent with the same measurements.

Improvement beyond the range oracle therefore depends on a prior. This is consistent with the non-identifiability result of Tachella et al. (2023); it does not preclude guarantees under additional assumptions on the signal class. The argument has been added to App. R as Remark 1.

2. Our analysis gives a measurable identity. Eq. (2) states ρ2=sin⁡2θ+(r−cos⁡θ)2\rho^2=\sin^2\theta+(r-\cos\theta)^2, where ρ\rho is the ratio of null-space error after adaptation to that before, cos⁡θ\cos\theta is the alignment between the update and the direction correcting the initial error, and rr is the relative update length.

The direction sets the limit of error reduction (ρ≥∣sin⁡θ∣\rho\ge|\sin\theta|, Proposition 2), and the length determines whether that limit is attained. Exceeding r=cos⁡θr=\cos\theta means passing the optimal length for that direction; it does not by itself mean that the error exceeds its initial value. For a nonzero update, improvement requires cos⁡θ>0\cos\theta>0 and 0<r<2cos⁡θ0<r<2\cos\theta.

The identity is not a performance guarantee. It reduces the outcome to two measurable quantities. Across all 174 decompositions, the measured error satisfies the alignment bound and agrees with the identity to within 8×10−58\times10^{-5} (Fig. 9 and the discussion accompanying Table 15). Failure cases are included: Table 14 reports increased null-space error for the oversized variable on the second backbone.

3. Where the method choices come from. We retain the data-consistency objective and change the optimised variable. The choices fall into two classes.

Choices supported directly by the analysis:

  • Route optimisation through the frozen network rather than optimise the image by gradient descent. Image-space gradient descent preserves the initial null-space component and cannot exceed the range oracle (Lemma 1; Tables 2, 12).
  • Use an activation as the variable. Proposition 1 identifies when the frozen mapping after that variable permits a null-space update (Sec. 4.3).

Choices determined empirically, guided by the quantities in the analysis:

  • Restrict the variable to no more entries than image voxels and check rr. The identity motivates controlling update length; the specific size restriction and threshold are determined by measurements (Tables 14, 17; App. U).
  • Prefer the activation followed by the most frozen layers. Increasing alignment with frozen depth is a measured regularity, not a derived theorem (Tables 13, 24).
  • Choose between activation and weights using withheld-view residuals. Corollary 3 ranks variables by ρ\rho, which requires ground truth; withheld-view residual is an empirical surrogate (Tables 16, 22).

Statements such as “The answer gives the method” did not distinguish these categories and claimed more than the analysis supports. The revised abstract, introduction and App. R now make this distinction explicit.

AuthorsOpenReview ↗

Response 2/5: W3 and Q1 — Optimiser assumptions and differences between variables

W3. The analysis uses gradient descent while the experiments use Adam

We agree that the optimiser assumptions should be explicit. There are three distinct points.

1. The final-update identity is independent of the optimiser. Eq. (2) is a geometric identity for the displacement between the initial and final reconstruction. Its alignment bound, Proposition 2, therefore also holds independently of the optimiser. The decompositions in Fig. 9 were measured on Adam runs; applying the identity to these updates does not require the trajectories to follow gradient descent.

2. Proposition 1 describes a local update mechanism. Let JJ be the Jacobian from the optimised variable to the image. For gradient descent, the first-order image update is δx=−ηJJ⊤g\delta x=-\eta JJ^\top g, where g=A⊤(Ax−y)g=A^\top(Ax-y) lies in R(A⊤)\mathcal{R}(A^\top). Proposition 1 asks whether JJ⊤JJ^\top maps this subspace into itself.

For a diagonally preconditioned current-gradient step, the corresponding kernel is JDtJ⊤JD_tJ^\top. Adam additionally uses first-moment accumulation: its local update has the form δx≈−ηJDtmt\delta x\approx-\eta JD_t m_t, where mtm_t combines current and historical variable-space gradients. Therefore, treating general Adam as only a diagonal rescaling of the current gradient is insufficient.

Under a fixed-Jacobian local approximation, the historical gradients are also generated through J⊤J^\top, and subspace preservation can be studied using the preconditioned mapping. Along the actual nonlinear trajectory, however, the Jacobian changes. We therefore distinguish the gradient-descent mechanism in Proposition 1 from the optimiser-independent identity used to analyse the measured Adam updates.

3. The image-space invariance statement needs a restricted scope. The argument preceding Lemma 1 applies when updates remain in the span of evaluated gradients, as for ordinary gradient descent and unpreconditioned conjugate gradients. It does not generally apply to Adam on the image, since coordinatewise preconditioning can introduce a null-space component. The revised manuscript states this restriction explicitly. The abstract refers specifically to image-space gradient descent.

Q1. Why the three variants perform differently

The difference is characterised by alignment cos⁡θ\cos\theta, the cosine between the null-space update and the direction correcting the initial null-space error (Eq. (2); App. G).

On the abdomen data, for the network input, the feature volume at the encoder output, and the network weights, respectively, cos⁡θ\cos\theta is 0.64, 0.54 and 0.50, and the null-space error removed is 23%, 16% and 14%. The input is followed by the entire frozen network, whereas the feature volume is followed only by the decoder. Across the networks tested, alignment generally increases with the depth of frozen layers after the variable (Table 24).

In the four-view setting, the corresponding alignments are 0.66, 0.60 and 0.66, with 25%, 20% and 25% of null-space error removed. This explains why the feature volume falls below weight-space TTT in that setting while the input is level with it.

RDC$^+$ additionally frees a second tensor, making its total number of free entries exceed the number of image voxels. It does not satisfy the size restriction used by the configuration procedure and is therefore not included in that procedure's candidate ranking.

AuthorsOpenReview ↗

Response 3/5: Q2–Q3 — Stopping and assessment without ground truth

Q2. Why PSNR drops with more steps, and how to choose the number of steps in practice

1. Noise-free behaviour. PSNR changes little after reaching its plateau. Across six scans, the mean RDC trajectory peaks at step 270 at 28.55 dB and reaches 28.52 dB at step 400. Individual peak steps range from 210 to 330; their average is approximately 253. These are different statistics, and the average individual peak step should not be interpreted as the peak of the mean trajectory (App. T, Table R7).

2. Noise or forward-model mismatch. Further optimisation can begin fitting measurement components caused by noise or model error. In the decomposition, this can appear as the update length exceeding the optimum for its alignment. For example, after 400 steps under the original stronger-noise condition, r=0.75r=0.75 and cos⁡θ=0.49\cos\theta=0.49 (Table 27). This exceeds the optimal relative length, although the remaining null-space error is still below its initial value.

3. Dependence on noise strength and learning rate. Under the noise model reported by DeepSparse, using RDC's learning rate 0.02 selected for the 40-step budget, the mean trajectory peaks at step 150 at 28.26 dB and loses approximately 0.14 dB by step 400. Averaging each scan's own peak instead gives a mean peak step of approximately 148 and a mean peak-to-final loss of 0.15 dB.

When the learning rate is selected for the 400-step budget on scans 0–2, rate 0.005 removes the decline: the drop from the peak to step 400 is at most 0.01 dB on each of the six scans. Thus the decline is not an unavoidable property of activation optimisation (App. T, Table R5).

4. Practical budget. The main comparisons use a fixed 40 steps and report the final iterate; labels were not used to select a stopping step for each scan. This is a compute budget rather than a claim that all settings converge by step 40. Forty steps is close to the earliest observed peak: on noisy MRI, the input variable peaks at step 38 on average, and the loss from passing the peak is 0.04 dB.

5. Label-free stopping. We evaluated the discrepancy principle, which stops when the residual reaches the noise variance. Under the original stronger noise, it stops at step 32 on average and gives 25.37 dB, against 25.49 dB at the fixed 40 steps. Under the reported noise model, its condition is met only at step 378 on average, giving 28.14 dB; the mean of the individual peak PSNRs is 28.27 dB. The stopping analysis is reported in App. T, Table R7.

Q3. How to judge whether an improvement is real without ground truth

We agree that cos⁡θ\cos\theta cannot be measured on an individual scan without ground truth. The non-identifiability argument in App. R, Remark 1, also means that measurements alone cannot certify improvement for every compatible ground truth. Two empirical findings support practical use.

1. Within-setting stability. Alignment varies relatively little across scans within a setting: its measured standard deviation is at most 0.08. The relative alignment ranking of the two variables agrees on 58 of 62 paired scans (Table 15). This is an empirical regularity, not a claim that alignment is identical for every scan.

2. Label-free deployment quantities. The procedure uses a check on relative update length rr to detect excessive updates, and residuals on withheld views to choose between activation and weight adaptation. The former uses an error-ratio calibration; the latter keeps a subset of projection angles out of the fit and compares residuals on those angles.

The withheld-view criterion chooses correctly on all 18 scan–setting pairs tested, including the two settings in which weight-space TTT is better. Ranking by the fitted residual succeeds on only 6/18 (Table 22).

This criterion has so far been validated on two sets of published weights for one cone-beam CT architecture and two view counts. It is an empirically validated selection rule, not a certificate that an individual reconstruction has lower unobserved error.

Reviewer_oLjuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This work investigates the test-time adaptation problem in medical inverse reconstruction. It optimizes an input or intermediate activation of a pretrained network while keeping its weights frozen. The proposed RDC minimizes a data-consistency loss through the frozen network. This work performs various experiments, including sparse-view CBCT, MRI acceleration, image SR, deblurring, etc. The empirical results show that the proposed method can improve reconstruction quality and enable faster adaptation.

Critical strengths
  1. This work addresses a meaningful problem, i.e., adapting a released reconstructor using a scan's measurements without updating its weights is interesting.
  2. The experiments are comprehensive, including multiple architectures and operators, real clinical projections, downstream tasks, and convergence studies. The authors also provide negative results and settings where weight-space adaptation performs better.
  3. The null-space decomposition provides a useful perspective on why reducing measurement error can improve unobserved image components through a pretrained network.
Critical weaknesses
  1. The methodological novelty over existing input/latent optimization methods is not sufficiently clear. Intermediate-layer optimization has been explored in ILO [1], and input adaptation for sparse-view CT has been explored in PINER [2]. This work should clarify what RDC contributes beyond applying these existing ideas to a pretrained conditional reconstruction network.
  2. The advantage over weight-space adaptation is sensitive to the optimization budget and noise level. For example, in Table 26, the PSNR gain in noiseless CT decreases to approximately 0.11 dB after 400 steps. Under Poisson noise, the gain is only 0.16 dB at 40 steps and becomes negative with more iterations. These results show an limited advantage in certain settings.
  3. The real-world data validation is relatively limited. The clinical evaluation includes 18 pelvic scans from a single centre, uses deformed planning CT images as references, and evaluates PSNR only within the field of view. The other real-data experiment includes five walnut micro-CT scans, while the MRI, super-resolution, and deblurring experiments use simulated data. Broader validation across clinical acquisition settings would strengthen this work.
  4. The paper is difficult to follow in several places. Frequent use of shorthand such as "the weights" and "the variable", together with generic terms such as "reconstructor", makes the technical discussion less clear.

[1] Daras, G., Dean, J., Jalal, A., & Dimakis, A. G. (2021). Intermediate layer optimization for inverse problems using deep generative models. arXiv preprint arXiv:2102.07364.

[2] Song, B., Shen, L., & Xing, L. (2023, January). Piner: Prior-informed implicit neural representation learning for test-time adaptation in sparse-view ct reconstruction. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 1928-1937). IEEE.

Questions for authors

See strengths and weaknesses, please.

4 comments
AuthorsOpenReview ↗

Response 1/4: W1 — Novelty relative to ILO and PINER

We thank the reviewer for the careful review and for recognising the significance of the problem, the breadth of the experiments and the null-space perspective. We respond to each point below.

W1. Novelty relative to ILO and PINER

Like CSGM, ILO and related methods, RDC differentiates the loss through frozen network layers. Our contribution is to answer a question that ILO's theory does not address; the analysis and experimental conclusions are directed at that question.

The question. The measurement loss cannot observe the null space of the operator. In our setting (n/m=32n/m = 32), 81% of the frozen model's error lies in the null space. Our central question is how the choice of optimisation variable affects whether, and how much, minimising the measurement loss reduces the error component that the loss cannot observe. ILO's Theorem 1 concerns the global optimum under Gaussian random measurements. We instead analyse updates for a fixed CT operator and measure the resulting null-space error directly on 174 decompositions (Fig. 9, Tables 12–16).

Two observations that differ from ILO in our setting.

  1. Choice of layer. ILO proceeds layer by layer towards the output. In our setting we observe the opposite trend: the more frozen layers follow the free variable, the higher the alignment (Table 24).

  2. Overshoot. ILO discusses the excessive flexibility of unconstrained intermediate representations and the difficulty of transferring constraint radii across tasks. We quantify a corresponding failure mode: on the second backbone, when the free variable has four times as many entries as the image, the update overshoots (r−cos⁡θ=+0.56r-\cos\theta = +0.56) and the null-space error increases by 7.8% (Table 14). The label-free relative-length check detects this; narrowing the variable returns its error reduction to the alignment bound (App. U). In our setting, this check plays the role of the hand-set radius.

Comparison with weight-space adaptation. Neither ILO nor PINER compares against adapting the weights. We use activation-space versus weight-space adaptation as the main experimental comparison to study how the choice of optimisation variable affects null-space error.

ILOPINERRDC (ours)
Pretrained networkUnconditional generatorBlack-box reconstruction networkReconstruction network
What is optimisedIntermediate-layer representation with an ℓ1\ell_1-ball constraintWeights of an implicit representationA single activation, or the input
Gradient passes through the reconstruction networkYesNoYes
TheoryGlobal optimum under random measurementsNoneNull-space identity for a fixed CT operator
Compared with weight adaptationNoNoYes

Our setting uses a released reconstruction network rather than requiring a separately pretrained unconditional 3D generator.

On PINER. We agree that the original description did not distinguish the two procedures clearly enough. PINER neither back-propagates through the reconstruction network nor optimises the network input under the measurement loss. It first selects an intermediate result from the fitting trajectory of an implicit representation and uses it as the input of the black-box network; it then fits a second implicit representation under data consistency. RDC back-propagates the measurement residual through the entire frozen network. The revised related-work discussion and App. G distinguish these procedures and state explicitly that RDC requires gradient access to the network, whereas PINER only needs to call it.

AuthorsOpenReview ↗

Response 2/4: W2 — Noise correction and matched-compute comparison

W2. Dependence of the advantage on the step budget and the noise level

We thank the reviewer for the close reading of Table 26. The noisy results in Table 26 of the original submission used noise 4–5 times stronger than intended; this was our error. After correction, RDC's lead at 40 steps is 0.64 dB rather than 0.16 dB. Our claim concerns a fixed per-scan compute budget, and the revised abstract and introduction make this explicit.

1. Noise: a correction. In the original submission, the noise generation for the rows labelled I0=105I_0 = 10^5 omitted TIGRE's normalisation by the maximum projection value. The noise standard deviation was therefore 4–5 times that of the noise model reported by DeepSparse (the SSIM of the frozen model dropped from 0.823 to 0.295). We re-ran the experiment with TIGRE's own CTnoise.add at the reported parameters (I0=105I_0 = 10^5, Gaussian σ=10\sigma = 10), with 6 scans, 400 steps and all other settings unchanged.

The table gives RDC's lead over weight-space TTT in dB; RDC uses learning rate 0.02 in all rows.

Steps40100200400
Noise-free+0.73+0.48+0.27+0.11
Corrected noise+0.64+0.36+0.08−0.17
Original submission (stronger noise)+0.16−0.57−1.22−1.90
  • At the reported noise level, noise changes the lead by about 0.1 dB up to 100 steps.
  • At every recorded budget from 10 to 100 steps, RDC leads on all 6 scans; 40 steps is not a specially chosen point.
  • Under corrected noise and with RDC's learning rate fixed at 0.02, RDC leads at step 230; the first recorded crossover is at step 240. This is a crossover between methods, not RDC's peak.
  • When RDC's learning rate is re-selected for the 400-step budget (candidates 0.02, 0.005 and 0.002; 0.005 is selected on scans 0–2), the decline is removed: on all 6 scans the drop from the peak to step 400 is at most 0.01 dB. At 400 steps, weight-space TTT is then 0.20 dB higher on scans 3–5, which took no part in the selection (28.49 vs 28.29), and 0.09 dB higher on all 6 scans.

The learning rate of weight-space TTT was likewise re-checked under corrected noise (10−510^{-5}, 3×10−53\times10^{-5} and 10−410^{-4}, 400 steps each; 10−410^{-4} on all 6 scans, 10−510^{-5} on scans 0–2). At 40 steps, 3×10−53\times10^{-5} is the best rate on every scan tested. At 400 steps, 10−410^{-4} is 0.18 and 0.32 dB higher on 2 scans and 0.05 to 0.26 dB lower on 3; on the remaining scan it starts to oscillate at about step 170, falls below the frozen model after step 329 and ends 3.05 dB below it. Weight-space TTT therefore uses 3×10−53\times10^{-5} throughout this response (App. T, Table R5).

2. Comparison at matched compute. Weight-space TTT needs 141 steps (noise-free) or 121 steps (corrected noise) to reach the quality RDC has at 40 steps: 3.5 and 3.0 times as many steps. Per-step optimisation costs are similar: on an RTX 4090, RDC takes 6.9 s and weight-space TTT 7.1 s (medians over 24 steps on each of two cards). Wall-time ratios are therefore close to step-count ratios. The factor of 5.7 in the original App. T overstates the difference; the revision corrects it using measured optimisation times (App. B, App. T).

  • Noise-free: RDC leads at the compared budgets through 400 steps. Weight-space TTT at 400 steps remains below RDC's peak (28.45 vs 28.55 dB). At 400 steps the two have the same alignment and remove the same null-space error; the remaining difference lies in the range component (Table 27).
  • Corrected noise, RDC learning rate 0.02: RDC's mean PSNR peaks at step 150 (28.26 dB). Weight-space TTT first exceeds RDC's contemporaneous mean at the recorded step 240 and needs about 290 steps to reach RDC's peak quality.

These results support an advantage at smaller budgets, while showing that weight-space adaptation can become better at longer budgets.

AuthorsOpenReview ↗

Response 3/4: W2 — Other operators, clinical budgets and practical properties

W2 continued. Other operators and clinical budgets

3. Longer-budget results on other operators. To account for early stopping, the following comparison lets weight-space TTT stop at its own peak on every image, an upper bound for any early-stopping rule. The input variable is taken at its final value after 400 steps. The table lists all nine noise-free conditions on MRI and super-resolution.

ConditionLead of the input variable (dB)Images on which it leads
SwinIR, Urban100+1.8552/52
EDSR, Urban100+0.8390/92
EDSR, IXI+0.8032/32
MRI, T2 4×+0.7023/24
SwinIR, BSD100+0.5752/52
EDSR, BSD100+0.5591/92
EDSR, BSD100 (inverted contrast)+0.4792/92
MRI, T2 8×+0.1217/24
MRI, T1 (contrast shift)−1.670/24

RDC leads in eight of these nine conditions. Across the full set of 18 conditions run to 400 steps, the activation variable is better in 10 and weight-space TTT in 8 (Table 25). These results limit the scope of the advantage; our analysis examines which variable is better in each setting.

Clinical comparison at a longer budget. We additionally run weight-space TTT for 120 steps on all 61 clinical cases, with the same learning rate, measurements and per-view gain. Its mean gain over the frozen model rises to 0.94 dB on the original 18 and 0.95 dB on the new 43. RDC at 40 steps remains higher in mean PSNR by 0.13 and 0.16 dB, respectively: +0.15 dB pooled, with 36/61 paired wins. Thus the average advantage persists when the weights receive three times as many steps, but the per-case advantage is smaller than at equal budgets.

On the original 18 cases, RDC at 120 steps gains 1.22 dB and leads the 120-step weights by 0.28 dB, on 13/18 cases. RDC at 120 steps has not been run on the new 43; these are fixed-budget comparisons, not a convergence claim (App. J.2, Table R8).

4. Additional practical properties.

  • Robustness to the step size. RDC does not diverge in a 16-fold range of learning rates, and its worst rate is 0.87 dB below its best; weight-space TTT diverges at the first step at 10 times its best rate (Table 28).
  • A label-free fallback. The withheld-view test chooses correctly between activation and weight adaptation for all 18 scan–setting pairs tested, including the settings in which weight-space TTT is better (Table 22).
  • Real noise. On clinical projections, at the 40-step budget RDC gains 1.06 dB and weight-space TTT 0.64 dB; RDC is better in 16/18 cases (Table 9). On the 43 additional cases, the gains are 1.11 dB and 0.71 dB, with RDC better in 40/43 (App. J.2, Tables R2–R3).
  • Downstream task. At the 40-step budget, pelvic bone segmentation Dice improves by 0.069, against 0.016 for weight-space TTT (Table 10; weight-space TTT follows its published protocol in this table).
AuthorsOpenReview ↗

Response 4/4: W3–W4 — Clinical validation and presentation

W3. Scope of the validation on real data

We agree that the validation on real data in the original submission is limited. During the rebuttal period we extended the clinical evaluation, and we state the remaining limitations below.

1. The clinical cases grow from 18 to 61. COBRA2026 contains 77 cases acquired at this centre with the same scanner model (Halcyon). The original submission downloaded the 22 with the lowest case numbers and kept 18 under the inclusion rule. We have now processed all remaining 55; the same rule keeps 43 and excludes 12.

The inclusion rule (at each of the eight views, measured projections correlate above 0.8 with projections of the reference), preprocessing script and hyperparameters are unchanged and were fixed before the new results were obtained. All results on the additional cases are reported.

CasesOriginal 1843 additional
Frozen model21.56 dB21.41 dB
RDC gain+1.06 dB+1.11 dB
RDC above frozen17/1843/43
Weight-space TTT gain+0.64 dB+0.71 dB
Weight-space TTT above frozen13/1838/43
RDC minus weight-space TTT+0.42 dB+0.39 dB
RDC better than weight-space TTT16/1840/43
Paired RDC–TTT p-value0.00025×10−105\times10^{-10}

All numbers use the same 40-step protocol for both methods. PSNR is computed inside the CBCT field of view; p-values are from two-sided exact Wilcoxon signed-rank tests. On the 43 additional cases, RDC improves on the frozen model in every case (p=2×10−13p = 2\times10^{-13}), with a minimum gain of +0.12 dB. Weight-space TTT falls below the frozen model on 5 cases, by up to 0.42 dB.

The additional cases reproduce the findings on the original 18. The revised manuscript includes the cohort summary and every additional case (App. J.2, Tables R2–R3). The longer-budget control is reported separately in Table R8: RDC at 40 steps remains higher in mean PSNR than weight-space TTT at 120 steps by 0.15 dB across all 61 cases, with 36/61 wins.

2. Reference image and metric. The reference is a planning CT deformed to the anatomy of the day. It is valid only inside the clinical CBCT field of view, which is why PSNR is evaluated there. Registration error, scatter and beam hardening limit the reference's accuracy.

To check whether the conclusion depends on reference quality, we split each cohort into two halves by agreement between measured and reference projections. Among the 43 additional cases, RDC leads weight-space TTT by 0.43 dB in the better-agreeing half and 0.35 dB in the other; for the original 18, the figures are 0.40 and 0.45 dB (App. J.2). The advantage therefore appears in both reference-quality groups.

3. Remaining limitations. All clinical cases come from one centre, one scanner model and one anatomical site (the pelvis). Most other centres in the dataset use an offset detector, a geometry the backbone cannot directly accommodate, so we could not add a second centre. The walnut micro-CT data comprise five scans, and the MRI, super-resolution and deblurring experiments use simulated measurements. Validation across clinical centres and scanner models remains to be done. The revised limitations state these boundaries explicitly (App. Z).

W4. Clarity of presentation

We agree that the terminology should be clearer. The revised manuscript makes the following changes:

  • Defines the technical terms and adds a notation and terminology table (Table R1).
  • Replaces ambiguous shorthand with “weight-space TTT (Darestani et al., 2022)” and “the free variable $v”; defines “reconstructor” as the published feed-forward reconstruction network.
  • Uses separate symbols for network weights and the angle θ\theta, and consistent notation for range and null spaces.
  • Defines terms introduced in the paper, including reversal, error identity and the variable selected by the configuration procedure.

We hope these responses address the concerns, and we are happy to clarify any further point.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Update to the revision

We thank the reviewers for their careful reading and constructive comments. We have revised the manuscript and responded to each reviewer. (Updated 19 Nov. Item 8 is new, and items 5–7 were added on 18 Nov.) The main updates are listed below.

  1. Expanded clinical evaluation. We processed all remaining cases from the same clinical centre and scanner with the existing inclusion rule and unchanged hyperparameters, adding 43 cases for a total of 61. On the new cases, RDC improves over the frozen model on all 43 and exceeds weight-space adaptation on 40 at the matched 40-step budget (App. J.2, Table R2; individual cases in Table R3 of the supplementary material).
  2. Longer-budget clinical comparisons. With weight-space adaptation run for 120 steps on all 61 cases, RDC at 40 steps is higher by 0.15 dB on average and on 36/61 cases. On the original 18 cases, with both methods at 120 steps, RDC leads by 0.28 dB on average and on 13/18 cases (App. J.2, Table R8).
  3. Noise and timing. Our original CT noise omitted TIGRE's projection normalisation and is 4–5 times stronger in standard deviation than the noise model reported by DeepSparse. The revision adds experiments with that model and keeps the original results as a stronger-noise condition. Timing is now reported as measured optimisation time per step, 6.9 s for RDC and 7.1 s for weight-space adaptation (Apps. B and T).
  4. Theoretical scope and presentation. Section 4 states what the analysis guarantees and which configuration rules are set empirically, and Table R12 (App. R) maps each statement of the analysis to the component of the method it motivates. Table R11 compares RDC with ILO, PINER and weight-space TTT. The abstract states the labelled learning-rate sweep and the error calibration, the main text has been edited throughout for readability, and the appendix adds a notation table, a TV reconstruction baseline, selection costs and a stopping-rule analysis.
  5. Jittered-phase rows of Table 1. The gantry phase in these rows was seeded with Python's per-process string hash, so each run drew its own phase. The paired column now compares each run's gain over its own frozen reconstruction, and RDC(F) leads TTT on 161 of the 177 scans of Table 1 (previously 160), as in Fig. 3. The other entries of Table 1 are unchanged (App. B).
  6. Learning rate and downstream tasks. Step sizes were swept on LUNA16 test scans 0–9, which are part of the lung rows. RDC uses 0.02, below the 0.04 its sweep favours; a fourfold larger rate lowers PSNR by 0.11 dB and a fourfold smaller one by 0.77 dB. The rate selected for weight-space adaptation also improves on its default on the held-out scans 10–29 (+0.53 dB, 20/20) (P4; App. B). In the downstream tasks, RDC recovers 27% of the lost pelvic Dice against 6–7% for the weight-space baselines, and reaches a nodule AUC of 0.961 against 0.956 for TTT† (Table R9).
  7. Supplementary PDF. Peripheral material has moved to a supplementary PDF (supp.pdf), namely the full Appendix Q, Appendix M, background paragraphs from Apps. A and F, the distilled adapter of App. B, and the per-case clinical listings (Tables 9 and R3). Each location keeps a short summary, and moved tables and equations keep their numbers.
  8. The identity without ground truth, and deviations. On views withheld from the loss, the error identity of Sec. 4.2 can be evaluated without labels, with the same two conditions for a lower null-space error. In the 36 runs of Table 22 the label-free quantities are within 0.004 of the same quantities computed with ground truth, and the withheld residual picks the better of RDC and weight-space adaptation on 18 of 18 scan–setting pairs, including the two settings where the weights are better (App. R, Table R13). Under the backbone's noise model it picks the better variant on 6 of 6 scans (Table R16). On the 2D operators the same test picks the better variant in 13 of 15 conditions over 1,516 images (Table R14). Table R15 gives every row of Table 1 as mean ± one standard deviation over scans, and Sec. 5.5 names the three kinds of shift under which weight-space adaptation ends higher.

The responses to each reviewer give the supporting evidence. We welcome further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.