acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Support Fields for Task-Preserving Medical Image Reconstruction

Abstract

Medical image reconstruction is commonly optimised for whole-image fidelity, although errors of similar magnitude can have different diagnostic consequences depending on their location. Under aggressive radial MR undersampling and sparse-view CT, lesion structures may remain degraded despite high global image quality. We propose CSF-Recon, a diagnosis-oriented framework that uses a counterfactual support field (CSF) to coordinate spatially targeted reconstruction and diagnostic evidence aggregation. Our analysis bounds diagnostic-margin degradation by reconstruction error weighted by spatial decision sensitivity, motivating preferential restoration of diagnostically relevant regions. Counterfactual alignment trains a reference teacher on fully sampled images to discourage reliance on decision-changing evidence outside annotated lesions. Its lesion-restricted class responses supervise a student field that gates microstructure restoration, local refinement, and local diagnostic pooling while retaining global context. The student predicts the field from a physics-driven coarse reconstruction, enabling spatial guidance directly from incomplete measurements. The teacher and lesion annotations are required only during training. On fastMRI+ and LIDC-IDRI at 16-fold acceleration, CSF-Recon achieves the highest mean accuracy and AUC among evaluated methods without maximising whole-image PSNR or SSIM. These results support spatially selective restoration as a practical approach to preserving diagnostic information under incomplete measurements.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical Image Reconstruction, Inverse Problems, Counterfactual Learning, Diagnostic Imaging

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KpLWOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CSF-Recon, a framework that preserves diagnostic information for medical image reconstruction. The core idea is that they want to put more detailed reconstruction on lesion regions than elsewhere.

The method first trains a teacher on fully sampled images using an counterfactual alignment method, then use the grad-cam created by teacher to guide the student training for reconstruction.

Critical strengths
  1. This paper is well motivated and easy to follow.
  2. It addresses a meaningful problem: good overall reconstruction quality does not necessary mean preservation of diagnostically important details. Using the spatial guidance for reconstruction and diagnosis gives the framework a clear and coherent design,
  3. The results on MRI and CT datasets are promising.
Critical weaknesses
  1. I think the main weakness is the denefit of counterfactual alignment over simpler spatial guidance is not clearly shown. In Table 3, a direct lesion mask supervision already achieves a AUC of 0.813+-0.019, compared with 0.821+-0.018 for aligned Grad-CAM, it seems to be within confidence interval for clinically meaningful difference?
  2. Considering the close gap between the direct GT, i suggest adding one direct baseline using lesion supervision (i mean the mask) to train the teacher and to use the probability map to guide the student training (replace the alignment Grad-CAM) to see the performance comparison.
  3. Also, the restoration and refinement stages lack clear measurement consistency. Improved classification performance can not establish that the restored details faithfully reflect the anatomy.
Questions for authors
  1. How much does the counterfactual alignment add beyond a direct lesion map supervision?
  2. could you include an anatomy-guided or task oriented reconstruction baseline with comparable supervision and refinement capacity to better isolate the proposed contribution?
  3. How do you evaluate the restored details are consistent with the acquired measurement?
5 comments
AuthorsOpenReview ↗

Thank you for your careful review and valuable suggestions, which have helped us improve the paper! We have revised the manuscript in response to your feedback. Below, we address each weakness (W) and question (Q) and describe the revisions. Please let us know whether these revisions address your concerns.

Specifically, we implemented the directly lesion-supervised teacher you suggested and added paired support-supervision comparisons, direct measurement-domain evaluation, and a final data-consistency correction experiment. Changes to the main text are highlighted in purple in the revised manuscript. Tables labelled R are numbered within this response; other section and table numbers refer to the revised manuscript.

W1 & Q1: Added value of alignment over direct GT-mask supervision

A1: We added support-supervision comparisons in Appendix C.11, summarized in Section 3.3.

We repeated the GT-versus-aligned comparison in three supplementary runs per modality using one newly generated random data split per modality, held fixed across three independent training seeds. Within each paired comparison, both methods use the same split and seed, with the downstream pipeline held fixed.

Table R1. GT-mask versus aligned Grad-CAM supervision, mean ±\pm standard deviation over three training seeds (Appendix C.11, manuscript Table 12).

ModalitySupport targetAccuracyAUC
MRGT mask0.688±0.0400.688\pm0.0400.733±0.0050.733\pm0.005
MRAligned Grad-CAM0.738±0.0200.738\pm0.0200.808±0.0100.808\pm0.010
CTGT mask0.673±0.0110.673\pm0.0110.783±0.0120.783\pm0.012
CTAligned Grad-CAM0.707±0.0050.707\pm0.0050.805±0.0150.805\pm0.015

The paired analysis in Table R2 (manuscript Table 13) uses 5,000 cluster-bootstrap resamples over MR volumes or CT patients while retaining slice/patch-level metrics.

Table R2. Paired cluster-bootstrap comparison of aligned and GT-mask supervision (Appendix C.11, manuscript Table 13).

ModalityMetricAligned minus GT95% CIBootstrap pp
MRAccuracy+0.0499[-0.0217, 0.1246]0.1902
MRAUC+0.0752[0.0054, 0.1567]≈0.050\approx0.050
CTAccuracy+0.0338[0.0096, 0.0596]0.0098
CTAUC+0.0215[0.000076, 0.043248]0.0514

The paired analysis supports a CT accuracy gain of 0.0338 (95% CI [0.0096, 0.0596], p=0.0098p=0.0098), while MR and CT AUC comparisons are near the conventional significance boundary (p≈0.050p\approx0.050 and 0.05140.0514). Percentile confidence intervals and centred-bootstrap tests are not exact inverses, so their conclusions may differ near the significance threshold. This addresses uncertainty using paired predictions rather than treating seed-level standard deviations as confidence intervals.

The distinction is between learning lesion membership and learning class-dependent spatial priorities within the lesion. Direct mask supervision learns annotated extent. Counterfactual alignment encourages prediction-changing evidence to lie within that extent, producing a class-conditioned attribution target for reconstruction.

The complementary directly lesion-supervised teacher control is detailed below in W2 & Q2.

W2 & Q2: An anatomy-guided baseline with comparable supervision and refinement capacity

A2: Following your suggestion, we trained a teacher with a lesion-probability head supervised by BCE and Dice losses, alongside a cross-entropy-trained diagnostic classifier, without counterfactual optimization or alignment. Its predicted probability maps replace aligned Grad-CAM as student support targets. Crucially, this control retains the same student reconstruction architecture, six-step refiner and Local+Global diagnostic architecture, data partitions and evaluation protocol. At inference, the student predicts its own support field; neither teacher nor lesion annotations are provided.

Table R3. Directly lesion-supervised versus aligned teacher guidance with matched downstream architectures (manuscript Table 14).

ModalityTeacher guidanceROI-PSNRAccuracyAUC
MRDirect lesion supervision27.13±0.0927.13\pm0.090.692±0.0380.692\pm0.0380.805±0.0520.805\pm0.052
MRAligned Grad-CAM (CSF-Recon)27.20±0.1727.20\pm0.170.763±0.0100.763\pm0.0100.825±0.0890.825\pm0.089
CTDirect lesion supervision30.59±0.1730.59\pm0.170.702±0.0320.702\pm0.0320.803±0.0010.803\pm0.001
CTAligned Grad-CAM (CSF-Recon)30.63±0.3430.63\pm0.340.734±0.0100.734\pm0.0100.821±0.0180.821\pm0.018

With similar local fidelity, aligned guidance improves mean accuracy by 7.1 percentage points for MR and 3.2 for CT, and AUC by 0.020 and 0.018, respectively. The differences cannot be attributed to giving CSF-Recon an additional refiner or a different downstream head architecture.

For complementary baselines, Appendix C.12, Table 15 adds annotation-matched ROI-weighted U-Net and MoDL, and Table 1 includes task-oriented joint U-Net. CSF-Recon retains higher mean accuracy and AUC than the annotation-weighted controls. These broader comparisons differ in architecture; the directly supervised teacher above specifically answers your request for comparable supervision and matched refinement capacity.

W3 & Q3: Are the restored details consistent with acquired measurements?

A3: We agree that higher classification accuracy alone does not establish anatomical fidelity. We therefore added direct measurement-domain evaluation and an optional final soft data-consistency correction in Appendix C.13, with corresponding clarification in Section 2.4 and the Discussion.

We measure the relative residual rmeas=∥AΩzeval−yΩ∥2/∥yΩ∥2r_{\mathrm{meas}}=\lVert\mathcal{A}_{\Omega}\mathbf{z}_{\mathrm{eval}}-\mathbf{y}_{\Omega}\rVert_2/\lVert\mathbf{y}_{\Omega}\rVert_2 at the acquired measurements. CT uses the projector matched to sinogram generation. MR uses the measured multicoil samples and sensitivity maps, with coarse-reconstruction phase used to initialize the complex image.

Table R4. Relative acquired-measurement residuals (manuscript Table 16).

OutputMR residualCT residual
Coarse reconstruction0.060447±0.0000000.060447\pm0.0000000.007785±0.0000000.007785\pm0.000000
Final CSF-Recon0.093254±0.0007090.093254\pm0.0007090.010725±0.0000960.010725\pm0.000096
Final + soft DC0.050886±0.0000350.050886\pm0.0000350.007269±0.0000370.007269\pm0.000037

Table R4 shows that the original final output is less measurement-consistent than the coarse reconstruction. The added correction is inference-only, with no retraining and no reference images, annotations or diagnostic labels as inputs. It minimizes a normalized image-proximity plus measurement-residual objective using conjugate gradient and the validation-selected weight λDC=10\lambda_{\mathrm{DC}}=10. For MR, both magnitude and phase can change during the solve. The reported residual is recomputed after output clipping.

This reduces measurement residual by 45.4% for MR and 32.2% for CT, below the respective coarse levels. We re-encode the corrected image with all network weights, the diagnostic head and the original support field fixed:

Table R5. Diagnostic performance before and after final soft DC (manuscript Table 17).

MethodMR accuracyMR AUCCT accuracyCT AUC
CSF-Recon0.763±0.0100.763\pm0.0100.825±0.0890.825\pm0.0890.734±0.0100.734\pm0.0100.821±0.0180.821\pm0.018
CSF-Recon + DC0.775±0.0340.775\pm0.0340.826±0.0330.826\pm0.0330.731±0.0170.731\pm0.0170.819±0.0190.819\pm0.019

Table R5 shows similar mean diagnostic performance after correction, with the original uncorrected model retained for the main results. These experiments demonstrate compatibility with improved acquired-data agreement; under undersampling, neither measurement agreement nor diagnostic accuracy alone guarantees the anatomical correctness of every restored detail. The revised Discussion makes this distinction explicit.

Dear Reviewer,

Thank you again for your thoughtful comments and valuable suggestions. We have revised the paper and added clarifications and experiments in response to your feedback. When you have a chance, we would greatly appreciate hearing whether our response addresses your concerns. Please let us know if you have any remaining questions or would like further clarification. We would be happy to address them during the discussion.

Thank you for your time and consideration.

Best regards,

The Authors

Reviewer_HztdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CSF-Recon for reconstruction that preserves information needed for a diagnostic task. A counterfactually aligned teacher provides lesion-restricted spatial supervision, and a student predicts a support field from a coarse reconstruction. This field guides high-frequency restoration, local refinement, and diagnostic pooling. Experiments on fastMRI+ and LIDC-IDRI at 16-fold undersampling report improved diagnostic accuracy and AUC. Component ablations, alternative support targets, and frozen diagnostic probes provide additional evidence for the approach.

Critical strengths
  1. Using the same spatial guidance for reconstruction and diagnostic pooling is well motivated. The paper explicitly examines the mismatch between image fidelity and diagnostic performance, and evaluates the approach in both MR and CT rather than relying on PSNR alone.

  2. The supplementary controls strengthen the evidence. In particular, the common frozen-probe comparison in Table 10 supports a contribution from the reconstructed images beyond the specialized diagnostic head. The support-source ablations and independent sensitivity analysis also make the role of spatial guidance more concrete.

Critical weaknesses
  1. The benefit of counterfactual alignment over direct lesion supervision is modest in the quantitative support-source comparison. On CT, the GT-mask target reaches 0.813 AUC and 0.727 accuracy, versus 0.821 and 0.734 for aligned Grad-CAM (Table 3). The comparison with plain Grad-CAM is useful, but paired uncertainty against the stronger GT-mask control is not reported. This matters because counterfactual teacher training adds substantial complexity to the method.

  2. The final reconstruction is not explicitly constrained to agree with the measurements. The paper acknowledges this, but a small number of refinement steps does not establish consistency. The evaluated tasks are also restricted to selected lesion categories and CT nodule patches at one acceleration setting. The diagnostic results support this setting, while preservation of measured information and structures outside the predicted support remains less clear.

Questions for authors
  1. Is the improvement over GT-mask supervision consistent across seeds and cases? Please provide a paired comparison, and clarify whether the same conclusion holds for the MR task.

  2. How does measurement-consistency error change from the coarse reconstruction to the final output, particularly for CT where the forward operator is directly available? Does a final consistency step retain the diagnostic advantage?

5 comments
AuthorsOpenReview ↗

Thank you for your careful review and valuable suggestions, which have helped us improve the paper!We have revised the manuscript in response to your feedback. Below, we address each weakness (W) and question (Q) and describe the revisions. Please let us know whether these revisions address your concerns.

Specifically, we added support-supervision comparisons with paired statistical analysis, together with direct measurement-domain evaluation and a final data-consistency correction experiment, to address your two main concerns. Changes to the main text are highlighted in purple in the revised manuscript. Tables labelled R are numbered within this response; other section and table numbers refer to the revised manuscript.

W1 & Q1: Added value over GT masks, consistency across repeated runs, and the MR task

A1: We agree that the original CT-only comparison and overlapping seed-level standard deviations do not establish the additional benefit of alignment. We added three supplementary runs for each of MR and CT using one newly generated random data split per modality, held fixed across three independent training seeds. Within each paired comparison, both methods use the same split and seed; only the support-supervision source changes, while the student, six-step refiner and Local+Global diagnostic architecture remain matched. These runs are reported separately from the original CT experiment in Appendix C.11, Table 12.

Table R1. GT-mask versus aligned Grad-CAM supervision with three random training seeds on one newly generated random data split per modality (manuscript Table 12).

ModalitySupport targetAccuracyAUC
MRGT mask0.688±0.0400.688\pm0.0400.733±0.0050.733\pm0.005
MRAligned Grad-CAM0.738±0.0200.738\pm0.0200.808±0.0100.808\pm0.010
CTGT mask0.673±0.0110.673\pm0.0110.783±0.0120.783\pm0.012
CTAligned Grad-CAM0.707±0.0050.707\pm0.0050.805±0.0150.805\pm0.015

The accuracy and AUC improvements are consistent across training seeds. To account for dependence between slices or patches, we performed 5,000 paired cluster-bootstrap resamples over entire MR volumes or CT patients, retaining all samples belonging to each selected case. Metrics remain slice/patch-level. The same cases are sampled for both methods and all seeds; metrics are computed per seed and then averaged. The test sets contain 25 MR volumes and 90 CT patients.

Table R2 reports differences computed from the underlying predictions, rather than rounded table means:

Table R2. Paired cluster-bootstrap comparisons of aligned and GT-mask supervision (manuscript Table 13).

ModalityMetricAligned minus GT95% CIBootstrap pp
MRAccuracy+0.0499[-0.0217, 0.1246]0.1902
MRAUC+0.0752[0.0054, 0.1567]≈0.050\approx0.050
CTAccuracy+0.0338[0.0096, 0.0596]0.0098
CTAUC+0.0215[0.000076, 0.043248]0.0514

The clearest paired evidence is for CT accuracy. For MR, aligned guidance also achieves higher mean accuracy and AUC than GT supervision, with gains of 0.0499 and 0.0752, respectively; its AUC comparison lies near the conventional significance boundary (p≈0.050p\approx0.050). CT AUC is also near that boundary (p=0.0514p=0.0514). Percentile confidence intervals and centred-bootstrap tests are not exact inverses, so their conclusions may differ near the significance threshold. These are aggregate improvements.

We additionally tested a directly lesion-supervised teacher whose BCE/Dice-trained probability maps replace aligned Grad-CAM, with the downstream architecture unchanged. CSF-Recon achieves higher mean accuracy/AUC: 0.763/0.825 versus 0.692/0.805 for MR, and 0.734/0.821 versus 0.702/0.803 for CT (C.11, Table 14). Together, these controls support the usefulness of aligned guidance beyond lesion localization alone. Section 3.3 now summarizes these supplementary comparisons.

W2 & Q2: Measurement consistency and final DC correction

A2: We agree that coarse-stage data consistency and a small number of refinement steps do not guarantee consistency of the final output. We now evaluate the acquired-measurement residual directly in Appendix C.13:

rmeas=∥AΩzeval−yΩ∥2∥yΩ∥2.r_{\mathrm{meas}}=\frac{\lVert\mathcal{A}_{\Omega}\mathbf{z}_{\mathrm{eval}}-\mathbf{y}_{\Omega}\rVert_2}{\lVert\mathbf{y}_{\Omega}\rVert_2}.

The CT projector matches sinogram generation. For MR, evaluation uses the sampling mask and coil sensitivities; magnitude outputs are lifted to the complex domain using the coarse reconstruction's phase. We also evaluated an optional inference-only final soft DC correction, with no network retraining or reference-image, annotation or diagnostic-label input. The correction balances image proximity with measurement agreement using the validation-selected weight λDC=10\lambda_{\mathrm{DC}}=10. The MR complex solve can update both magnitude and phase. Reported measurement residuals are evaluated after output clipping, not taken from the numerical solver's convergence residual.

Table R3. Relative acquired-measurement residuals (manuscript Table 16).

OutputMR residualCT residual
Coarse reconstruction0.060447±0.0000000.060447\pm0.0000000.007785±0.0000000.007785\pm0.000000
Final CSF-Recon0.093254±0.0007090.093254\pm0.0007090.010725±0.0000960.010725\pm0.000096
Final + soft DC0.050886±0.0000350.050886\pm0.0000350.007269±0.0000370.007269\pm0.000037

As shown in Table R3, the original final output has a higher residual than the coarse image. Final soft DC reduces it by 45.4% for MR and 32.2% for CT, to below the respective coarse levels.

We then re-encode the corrected image while keeping all network weights, the diagnostic head and the original predicted support field fixed:

Table R4. Diagnostic performance before and after final soft DC (manuscript Table 17).

MethodMR accuracyMR AUCCT accuracyCT AUC
CSF-Recon0.763±0.0100.763\pm0.0100.825±0.0890.825\pm0.0890.734±0.0100.734\pm0.0100.821±0.0180.821\pm0.018
CSF-Recon + DC0.775±0.0340.775\pm0.0340.826±0.0330.826\pm0.0330.731±0.0170.731\pm0.0170.819±0.0190.819\pm0.019

Table R4 shows that mean diagnostic performance remains close to that of the original model. These results indicate that the observed diagnostic performance is compatible with improved measurement consistency. The main experiments still report the original model without this optional correction. Section 2.4 and the Discussion now explicitly distinguish learned refinement from final measurement consistency.

Dear Reviewer,

Thank you again for your thoughtful comments and valuable suggestions. We have revised the paper and added clarifications and experiments in response to your feedback. When you have a chance, we would greatly appreciate hearing whether our response addresses your concerns. Please let us know if you have any remaining questions or would like further clarification. We would be happy to address them during the discussion.

Thank you for your time and consideration.

Best regards,

The Authors

Reviewer_Ur3POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CSF-Recon, a method to preserve diagnostic information in reconstructions from undersampled MR and CT. The teacher, trained with counterfactual alignment on fully sampled images, provides lesion-focused supervision for a student. The student predicts a support field from the undersampled data and uses it to guide both where the image is restored and where diagnostic evidence is pooled. On fastMRI+ and LIDC-IDRI at 16x acceleration, the paper reports that CSF-Recon achieves the highest accuracy and AUC among the evaluated methods without maximising PSNR or SSIM.

Critical strengths
  1. The paper studies an important problem: whole-image fidelity does not tell us whether the evidence needed for diagnosis is preserved after undersampling.

  2. The paper includes several probe experiments to test its claims, including a common frozen probe across methods as well as fixed and matched probes. These experiments provide a more thorough evaluation than standard reconstruction metrics alone.

  3. The paper discusses its limitations openly and describes the architecture, training, and inference procedures in detail.

  4. The teacher and lesion annotations are required only during training. At inference time, the student operates from the measurements alone.

Critical weaknesses
  1. Most of the headline diagnostic gain appears to come from CSF-Recon's own diagnostic pathway rather than from the reconstructed images. In Table 10, the CT AUC improvement over ISTA-Net+ is only a small fraction of the margin reported in Table 1, and no significance test is reported. Under the matched probes in Table 9, the final reconstruction achieves the same CT AUC as the coarse reconstruction.

  2. Counterfactual alignment is emphasized in the contributions, but Table 3 shows that using the lesion annotation directly as the support field performs almost as well. The aligned Grad-CAM improves accuracy by only about 0.007 and AUC by 0.008, both within the standard deviations across seeds. It is unclear whether the counterfactual optimization, implicit differentiation, and teacher training provide a meaningful benefit over simply using the annotation. Moreover, the support-field source ablation is reported only for CT.

  3. Learned and RL-based adaptive sampling methods address a closely related resource allocation problem at the acquisition stage. The paper should discuss whether CSF-Recon competes with or complements these methods.

  4. All baselines are annotation-free, while CSF-Recon is trained with lesion annotations. Therefore, the comparison does not isolate whether the gain comes from the proposed method or from the additional supervision. Baselines using the same annotations, such as weighting the reconstruction loss by the lesion annotation, are missing.

Questions for authors
  1. Do the baselines' reconstruction errors actually concentrate in the lesion regions?

  2. What does the coarse reconstruction score with the full local and global diagnostic head?

  3. How does CSF-Recon compare with MoDL or U-Net trained with a reconstruction loss weighted by the lesion annotations?

  4. What happens to a lesion that the support field misses?

  5. How far is the final reconstruction from the measurements, compared with the coarse reconstruction?

  6. Why does CT diagnostic performance improve when adding the support field lowers lesion-region fidelity?

  7. Why does the refiner improve lesion-region fidelity without improving CT diagnostic performance?

5 comments
AuthorsOpenReview ↗

Thank you for your careful review and valuable suggestions, which have helped us improve the paper! We have revised the manuscript in response to your feedback. Below, we address each weakness (W) and question (Q) and describe the revisions. Please let us know whether these revisions address your concerns.

Specifically, we added annotation-matched reconstruction controls, MR/CT support-supervision comparisons, a coarse-image Local+Global probe, a comparison of coarse and final reconstructions with the original diagnostic pathway fixed, and measurement-consistency experiments. Changes to the main text are highlighted in purple in the revised manuscript. Tables labelled R are numbered within this response; other section and table numbers refer to the revised manuscript.

W1: Reconstruction contribution versus diagnostic-head contribution

A1: Thank you for highlighting the distinction between reconstruction and diagnostic-pathway contributions. We agree that the full-system improvement in Table 1 should not be attributed entirely to the reconstructed images.

The equal CT AUC of 0.811 in Table 9(b) specifically concerns xc+Global\mathbf{x}_c+\mathrm{Global} and x^T+Global\hat{\mathbf{x}}_T+\mathrm{Global}, evaluated using independently trained probes with the same architecture. We acknowledge that the reported mean AUC does not improve under this readout. This result, however, does not establish that the two reconstructions contain identical diagnostic information or perform identically under other readouts.

CSF-Recon is designed to coordinate where diagnostically relevant details are restored and how those details are aggregated, rather than to provide a task-independent enhancement that improves every downstream classifier. Global averaging may dilute spatially localized responses, providing a possible explanation for the absence of a Global-probe AUC gain.

New fixed original-pathway control. We additionally compared xc\mathbf{x}_c and x^T\hat{\mathbf{x}}_T while freezing the original CSF-Recon diagnostic encoder and L+G head. The support field is predicted once from xc\mathbf{x}_c and held fixed for both diagnostic inputs. Only the diagnostic image and its encoded features change; the coarse image is passed directly to diagnosis, without another reconstruction pass or any retraining.

Table R1. xc\mathbf{x}_c–x^T\hat{\mathbf{x}}_T comparison with the original diagnostic pathway fixed (Appendix C.9, manuscript Table 11).

(a) Accuracy and AUC, mean ±\pm standard deviation over three seeds. Balanced accuracy and F1 are also reported in the manuscript.

ModalityDiagnostic inputAccuracyAUC
MRxc\mathbf{x}_c0.737±0.0150.737\pm0.0150.793±0.0220.793\pm0.022
MRx^T\hat{\mathbf{x}}_T0.750±0.019\mathbf{0.750\pm0.019}0.807±0.015\mathbf{0.807\pm0.015}
CTxc\mathbf{x}_c0.711±0.0340.711\pm0.0340.822±0.0050.822\pm0.005
CTx^T\hat{\mathbf{x}}_T0.729±0.029\mathbf{0.729\pm0.029}0.832±0.005\mathbf{0.832\pm0.005}

(b) Paired case-cluster bootstrap; Δ\Delta is the metric for x^T\hat{\mathbf{x}}_T minus that for xc\mathbf{x}_c.

ModalityMetricΔ\Delta95% CIBootstrap pp
MRAccuracy+0.01282[-0.02353, 0.05129]0.5565
MRAUC+0.01449[-0.06403, 0.08535]0.7129
CTAccuracy+0.01784[0.00347, 0.03287]0.0212
CTAUC+0.01026[0.00207, 0.01906]0.0184

We performed 5,000 paired case-cluster bootstrap resamples over 25 MR volumes (104 slices) or 90 CT patients (299 patches), retaining all samples from each selected case. Both inputs and all seeds use the same sampled cases; slice/patch-level metrics are computed per seed and then averaged within each resample.

With the diagnostic pathway unchanged, x^T\hat{\mathbf{x}}_T improves CT accuracy by 1.78 percentage points and AUC by 0.01026, with both paired confidence intervals excluding zero. MR also has higher mean accuracy and AUC than xc\mathbf{x}_c, although its paired intervals include zero. The small number of independent MR volumes (n=25n=25) limits the precision and power of this case-level resampling analysis. The CT result directly supports an image-side contribution under the original L+G readout: it cannot be explained by changing the diagnostic head or predicted support field.

Complementary image-side gains are also observed with reference-trained frozen probes. In Table 9(a), replacing xc\mathbf{x}_c with x^T\hat{\mathbf{x}}_T increases mean CT AUC from 0.718 to 0.728 and accuracy from 0.656 to 0.672. Under the common frozen probe in Table 10, CSF-Recon achieves an AUC of 0.673, compared with 0.653 for ISTA-Net+. Neither comparison introduces CSF-Recon's specialized diagnostic head. These probe comparisons report mean performance differences.

W2: Benefit beyond direct GT-mask supervision, including MR

A2: We added three supplementary runs per modality using one newly generated random data split per modality, held fixed across three independent training seeds. Within each paired comparison, both methods use the same split and seed, and the student, refiner and diagnostic architecture are unchanged; only the support-supervision source changes.

Table R2. GT-mask versus aligned Grad-CAM supervision with three random training seeds on one newly generated random data split per modality (manuscript Table 12).

ModalitySupport targetAccuracyAUC
MRGT mask0.688±0.0400.688\pm0.0400.733±0.0050.733\pm0.005
MRAligned Grad-CAM0.738±0.0200.738\pm0.0200.808±0.0100.808\pm0.010
CTGT mask0.673±0.0110.673\pm0.0110.783±0.0120.783\pm0.012
CTAligned Grad-CAM0.707±0.0050.707\pm0.0050.805±0.0150.805\pm0.015

We also performed 5,000 paired cluster-bootstrap resamples over entire MR volumes or CT patients, retaining slice/patch-level metrics. CT accuracy improves by 0.0338, with 95% CI [0.0096, 0.0596] and p=0.0098p=0.0098; the MR and CT AUC comparisons lie near the conventional significance boundary (p≈0.050p\approx0.050 and 0.05140.0514). Percentile confidence intervals and centred-bootstrap tests are not exact inverses, so their conclusions may differ near the significance threshold. Full intervals and tests are reported in Appendix C.11, Tables 12-13, separately from the original CT runs. This supports the benefit of alignment in the evaluated settings. We also added a directly lesion-supervised teacher with matched downstream capacity (Table 14) and summarized these controls in Section 3.3.

W3: Relation to learned and adaptive acquisition

A3: We expanded Appendix B.2. LOUPE (Bahadir et al., 2019) jointly learns MR sampling and reconstruction; Pineda et al. (2020) select MR measurements sequentially; and Shen et al. (2022) personalize CT angles and dose. Acquisition can also be task-driven, as in TACKLE (Wu et al., 2024) and Du et al. (2024).

The distinction is therefore which measurements to acquire versus how acquired information is restored and aggregated, not simply fidelity versus diagnosis. Under a prescribed acquisition operator, CSF-Recon coordinates image-domain restoration and diagnostic pooling. The mechanisms are potentially complementary: learned measurements could feed a CSF-Recon model trained for that sampling distribution, with matched forward/noise models and adaptation of reconstruction-conditioned policies.

W4 & Q3: Baselines with the same lesion annotations

A4: We added ROI-weighted U-Net and MoDL in Appendix C.12, Table 15, referenced in Section 3.2. Both use the same training annotations, combining full-image L1 loss with a per-image normalized lesion L1 term weighted by 0.05. Annotations are not used at inference; diagnosis follows the baseline classifier protocol.

Table R3. Diagnostic results of annotation-matched reconstruction baselines (manuscript Table 15).

ModalityMethodAccuracyAUC
MRROI-weighted U-Net0.635±0.1360.635\pm0.1360.680±0.1780.680\pm0.178
MRROI-weighted MoDL0.702±0.0390.702\pm0.0390.753±0.0170.753\pm0.017
MRCSF-Recon0.763±0.0100.763\pm0.0100.825±0.0890.825\pm0.089
CTROI-weighted U-Net0.623±0.0170.623\pm0.0170.684±0.0110.684\pm0.011
CTROI-weighted MoDL0.641±0.0220.641\pm0.0220.670±0.0070.670\pm0.007
CTCSF-Recon0.734±0.0100.734\pm0.0100.821±0.0180.821\pm0.018

Annotation-weighted reconstruction alone does not reproduce CSF-Recon's mean diagnostic performance. These are annotation-matched complete-system comparisons; the separate teacher control in C.11 additionally holds downstream reconstruction, refinement and diagnostic architectures fixed.

Q1: Do the baselines' reconstruction errors actually concentrate in the lesion regions?

A5: Our current results do not establish that baseline reconstruction errors predominantly concentrate in lesion regions, and we neither assume nor claim such concentration. ROI metrics and qualitative examples assess lesion-region reconstruction quality but do not establish how the total error is distributed across the image. Our motivation concerns the diagnostic importance of errors rather than their spatial concentration: high whole-image quality need not imply preservation of diagnostically relevant local information. Errors in a small lesion can contribute little to image-averaged metrics yet affect diagnosis substantially, even when larger errors occur elsewhere.

Section 2.7 and Appendix D formalize this distinction by bounding diagnostic-margin degradation using reconstruction errors weighted by spatial decision sensitivity. CSF-Recon therefore uses the predicted support field to prioritize potentially decision-relevant detail, while the base reconstruction and global diagnostic branch retain anatomical context. The mechanism addresses spatially varying diagnostic importance; it does not rely on lesions containing the largest errors or the majority of the total reconstruction error.

Q2: Local+Global diagnosis from the coarse reconstruction

A6: To examine the diagnostic-pathway contribution directly, we added the requested coarse-image Local+Global (L+G) control:

Table R4. CT Global and Local+Global diagnostic probes (manuscript Table 9(b)).

CT input and readoutAccuracyAUCF1
Coarse, Global0.717±0.0470.717\pm0.0470.811±0.0100.811\pm0.0100.735±0.0740.735\pm0.074
Coarse, L+G (added)0.726±0.0180.726\pm0.0180.821±0.0180.821\pm0.0180.763±0.0140.763\pm0.014
Final, L+G0.747±0.0100.747\pm0.0100.828±0.0160.828\pm0.0160.775±0.0140.775\pm0.014

L+G pooling improves the coarse-image results, while the final reconstruction performs better with the same head architecture. These probes are trained independently; they do not share classifier weights. The fixed-weight comparison in our response to W1 (Table R1) separately isolates the image-side contribution under the original L+G pathway.

We therefore distinguish the complete-system gains in Table 1 from the smaller, readout-dependent image-side gains in the probe experiments. Because these evaluations use different diagnostic protocols, their margins cannot be directly decomposed into numerical contributions from reconstruction and the diagnostic head. The probe controls are detailed in Appendix C.9, and their complementary roles are discussed in Section 4. Together, these controls support complementary contributions from reconstruction and spatial diagnostic aggregation: the observed gains cannot be explained solely by CSF-Recon's specialized diagnostic head, nor should they be attributed entirely to reconstruction alone.

Q4: What happens when the support field misses a lesion?

A7: Thank you for raising this important failure case. A support-field miss does not directly mask out the lesion from the reconstructed image. As described in Section 2.4:

x^0=xbase+m⊙xmicro.\hat{\mathbf{x}}_0=\mathbf{x}_{\mathrm{base}}+\mathbf{m}\odot\mathbf{x}_{\mathrm{micro}}.

The support field gates only the added microstructure component, while the base branch reconstructs the full field of view. However, low support in a lesion region attenuates its high-frequency restoration and gated residual refinement. A missed lesion may therefore lose diagnostically relevant detail, particularly when its visibility depends on fine structures or boundaries.

The ungated global diagnostic branch can use evidence that remains outside the predicted support, but it cannot be assumed to recover information already lost during reconstruction. Support-field misses are therefore a potential failure mode, even though global context is retained (Sections 2.4-2.5 and Appendix C.10).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.