acceptodds
Withdrawn from ICLR 2027

Post-Training Clinical-Signal Auditing of Medical Diagnostic Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In diagnostic imaging, specific findings vary in appearance with disease severity or clinical state, and these relationships, accumulated as clinical knowledge, provide an important basis for diagnostic interpretation. Diagnostic performance is a central axis for evaluating medical AI, but from diagnostic performance alone it can be difficult to determine whether models with similar performance respond differently to these changes. Using clinical ordering information withheld from training and final model selection as a reference, we assess how closely frozen-model responses follow the order of these changes within the same diagnostic label, and propose this assessment as a retraining-free post-training audit method. Across backbone–task combinations on four medical image diagnosis tasks (Liver US, Breast US, Knee X-ray, and Fundus), this degree of agreement ranged from to , and its relationship with diagnostic performance varied by task. In the three tasks with a coarse spatial scale prespecified for the relevant clinical findings, higher agreement was associated with better preservation of this ordering after controlled perturbations, and the association remained positive after adjustment for relevant covariates (). Because the audit method requires only frozen-model forward outputs on the model side, it can be applied to publicly released diagnostic checkpoints without access to their original training pipelines or internal representations. These results provide a complementary post-training criterion for assessing how closely model responses agree with the direction of change indicated by clinical knowledge, beyond what conventional diagnostic performance alone readily captures.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
medical AI, medical imaging, trustworthy AI, post-training auditing, model evaluation, clinical signal

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PRSzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a post-training audit that measures how well a medical image classifier's output follows an ordered clinical factor within the same diagnostic label. The proposed metric 'F' uses partial Spearman correlation between model outputs on a clinically selected spatial-frequency band and an ordinal clinical factor withheld from training. The authors evaluate 96 checkpoints across four medical imaging tasks and 11 public Breast US checkpoints. They show that models with similar diagnostic performance can have different alignment scores and report associations with robustness under image perturbations.

Critical strengths
  1. The paper raises an interesting evaluation question: models with similar diagnostic performance may respond differently to clinically ordered factors. The proposed audit requires only frozen-model outputs and does not need retraining or access to internal representations, which makes it applicable to existing models.

  2. The experimental evaluation is fairly extensive, covering four imaging tasks and multiple architectures. I particularly appreciate the statistical controls, including permutation tests, seed stability, normalization sensitivity, and patient-disjoint cross-fitting. The association with clinical-order retention after image perturbations (Table 9) is also an interesting finding.

Critical weaknesses
  1. I am not convinced that higher clinical-signal alignment necessarily indicates a better or more clinically meaningful model. All models are trained for binary classification, so they are not explicitly expected to produce monotonically increasing probabilities across severity grades within the same diagnostic class. More importantly, agreement with an established clinical grading system is not necessarily equivalent to better diagnosis. A model may capture useful information that is not represented by the grading system. For example, Pierson et al. (Nature Medicine, 2021, An Algorithmic Approach to Reducing Unexplained Pain Disparities) showed that deep learning could capture pain-related information from knee radiographs that conventional radiographic grades did not fully explain. The paper needs to clarify why higher alignment should be preferred and what it tells us about model quality. I also have concerns about the clinical interpretation of the anchor bands in Section 3 and Table 1. The authors select spatial-frequency bands based on clinical literature, but do not directly verify that these bands capture the intended findings, such as joint-space narrowing, lesion morphology, or microaneurysms. A high correlation with KL grade or BI-RADS does not necessarily mean that the model is responding to those specific imaging findings rather than other image characteristics.

  2. The practical contribution beyond existing evaluation methods remains unclear. Table 6 shows that the relationship between the proposed alignment metric and corruption vulnerability almost disappears after controlling for anchor-band AUROC ((\rho=-0.035)). In Table 15, anchor-band AUROC performs as well as or better than the proposed metric for stress-testing prioritization. The association with clinical-order retention in Table 9 is interesting, but retention is also defined using the same clinical factor rather than an independent clinical outcome. I therefore find it difficult to identify a concrete situation where this metric would lead to a better model selection or evaluation decision. The experimental design also lacks comparisons with models explicitly trained to learn ordinal relationships. Methods such as CORAL (Cao et al., 2020) and CORN (Shi et al., 2023) seem particularly relevant. Although these methods use additional ordinal supervision and are not direct replacements for a post-training audit, they would provide useful reference models for checking whether the proposed metric captures the intended behavior. Currently, the paper mainly reports differences among binary classifiers without showing whether those differences reveal a meaningful limitation or something that can be improved.

Questions for authors

1. What is the intended use of higher clinical-signal alignment?

Does it indicate better clinical reasoning, greater robustness, or simply stronger agreement with an existing grading system? If a model has better diagnostic performance but lower alignment, how should we interpret that result?

2. Could the authors validate the clinical anchor bands using actual imaging findings?

For example, in Knee X-ray, evaluating the relationship with quantitative joint-space width rather than KL grade alone would help establish whether the coarse-band response reflects the intended finding. Similar finding-level or lesion-region analyses would be useful for the other tasks.

3. What does the proposed metric add beyond anchor-band AUROC?

Given Tables 6 and 15, could the authors provide an example where models have similar full-image and anchor-band AUROC but different alignment scores, and show that this difference corresponds to an independent and meaningful behavioral or clinical outcome?

4. Why were only binary cross-entropy classifiers evaluated?

Could the authors include multiclass or ordinal classification models, such as CORAL or CORN, as supervised reference models? It would be useful to see whether explicitly learning the clinical ordering increases the proposed metric and whether that improvement is associated with other desirable model properties.

5. Could the authors report class-specific alignment rather than only the pooled partial Spearman correlation?

The current implementation residualizes global ranks on the binary label, which is not equivalent to computing Spearman correlation separately within each class. Reporting (y=0) and (y=1) results, along with their uncertainty and grade distributions, would make the interpretation clearer.

6. How should the large model differences in Table 2 be interpreted?

For example, Liver US ResNet18 has a full-image AUROC of 96.2 but an anchor-band AUROC of only 32.3 and negative anchor alignment, while its full-image alignment is positive. Could this reflect sensitivity to the band-restricted input rather than clinically undesirable behavior? The Liver US evaluation also contains only 17 independent patients, so uncertainty around this example is particularly important.

7. Could the authors clarify the numerical stability and model selection in the supplementary analysis?

Some stored softmax outputs appear to be saturated, which may affect rank-based statistics. Comparing probabilities with logit margins would be helpful. The supplementary caches also appear to include ResNet50 results outside the reported eight-backbone panel; clarification of the architecture inclusion criteria and treatment of undefined alignment scores would improve

3 comments
AuthorsOpenReview ↗

Response to Reviewer PRSz

Thank you for examining our supplementary caches and code so carefully.

1. On Why Within-Label Alignment Matters (Weakness 1 & Question 1)

We did not explain clearly enough why higher alignment is preferable. Binary models are indeed not trained to follow the grade order, which is why we check it after training. In three tasks the label is a threshold of the clinical factor (§3.1), so a model that decides from the finding responds more strongly to more severe findings. Alignment is not a quality score but a flag for models that do not follow this order. Pierson et al. (2021) trained on patient-reported pain, a subjective experience, whereas our labels and clinical factors are physicians' image-based grades or biopsy results (Table 1). Here, departing from the order means judging the same finding from other cues.

Question 1. Liver US ResNet18 has the highest full-image AUROC (96.2) but alignment −0.24, and loses 41.2 AUROC points without the fine band and 37.7 under blur, JPEG, and downsampling (Table 10). Retraining with the fine band removed from half of the training images preserved performance (97.4) and cut the corruption loss to 8.2 points, as it did for all 18 checkpoints of the two low-alignment backbones (28.2 to 9.7 points).

2. On Validating the Anchor Band with Measured Imaging Findings (Weakness 1 & Question 2)

Of our four cohorts, only Breast US (BUS-BRA) has expert lesion annotations (masks), so we measured the lesion morphology you mention there, without model outputs or BI-RADS (1 − solidity, depth/width ratio). The prespecified coarse band followed the measured findings best (6 of 8 models), alignment against the measured findings ranked the models almost identically to F_anchor|y (hereafter F; +0.976), and within the same BI-RADS grade the response still followed the findings (7 of 8). The other three cohorts lack such annotations (including knee joint-space width) and are left to future work.

3. On the Contribution beyond Anchor-Band AUROC and the Retention Endpoint (Weakness 2 & Question 3)

The covariance with A_anchor (Tables 6, 15) is correct. A_anchor might seem to substitute for F, but alignment examines whether the response follows the clinical order within a label, not the discrimination outcome; we did not explain this sufficiently. The audit's purpose is to judge black-box models using only clinical scales and forward passes (§6.4; 11 public checkpoints). A model that decides from the clinical finding responds to the anchor band much as to the full image, so both are high, which is what the audit is designed to check (Table 2).

Question 3. We did not emphasize this enough in the main text. If two models have equal full-image and anchor-band AUROC, the remaining difference is the clinical ordering within a label, and it predicts retention of that ordering under corruption (+0.475 [+0.17, +0.71] after controlling for A_anchor and clean full-image alignment, Table 9). Retention is the property itself, measured the way robustness is measured by accuracy under corruption (Hendrycks & Dietterich, 2019).

4. On Binary Training and Ordinal Reference Models (Weakness 2 & Question 4)

We should have stated this rationale. The audit is a post-hoc check of trained models, not a method for training ordinal models, so the reference factor c is withheld from training and checkpoint selection (§3.1) and grades are thresholded into binary labels (Table 1; Breast US: label pathology, factor BI-RADS). CORAL, CORN, and multiclass models use c as training supervision and cannot be audited this way; for the same reason, public Fundus models trained on the DR grade were excluded (C.6).

5. On Class-Specific Alignment (Question 5)

We computed class-specific alignment with intervals and grade distributions for all 32 cells; the conclusions are unchanged (no significantly negative cell), and the tables will be added to the appendix.

6. On Liver US ResNet18 (Question 6)

The two measurements in Response 1 reach the same conclusion without band inputs (Table 10), and alignment was negative under all three normalizations (B.3). The 17-patient limitation is stated in §5.3 and D.1.

7. On Saturation, Logit Margins, ResNet50, and Undefined Scores (Question 7)

The positive-class probability is a strictly increasing function of the logit margin, so all rank-based audit quantities are identical (B.1), and worst-case re-ordering of float32 saturation ties changed F by at most 0.039, with no sign change. No panel checkpoint has an undefined F (B.4). ResNet50 was excluded because its Liver US anchor-band output is constant, leaving F undefined.

Revisions. Lesion-finding measurements (§6.3); class-specific alignment and the scale correction (D.1); saturation and the ResNet50 exclusion (B.1, C.2).

Thank you again for the careful review. We hope this addresses your concerns and would appreciate your reconsideration of the score.

Thank you for your detailed response and additional analyses. After reading your response, I have been waiting for the new results to be incorporated into the manuscript, but I haven't seen an updated version over the past several days.

Could you please let me know when the revised manuscript will be available, or where I can access the additional results? I would like to review them before reconsidering my assessment.

Dear Reviewer PRSz,

Thank you for your patience, and we sincerely apologize for the delay. As this is our first experience with the ICLR discussion process, we did not realize that the revised manuscript should be uploaded together with our response. We have now uploaded the revised PDF, in which all revised or added text is shown in teal.

The additional results described in our response can be found as follows:

  • Measured lesion morphology (Response 2): §6.5 and Appendix D.5 (Figure 6, Tables 20–21)
  • Retraining with the fine band removed from half of the training images (Response 1): §6.6 and Appendix D.8 (Figures 8–9, Tables 28–31)
  • Class-specific alignment and grade distributions (Response 5): Appendix D.1 (Table 19) and Appendix C.1 (Table 18)
  • Saturation ties and the ResNet50 exclusion (Response 7): Appendices B.1 and C.2

We also added a concept-level analysis of Knee X-ray concept bottleneck models (§6.5, Appendix D.6) and probe-design sensitivity analyses (Appendix D.7), and updated the abstract, contributions, and conclusion accordingly. The original tables and figures keep their numbers, so the references in our response remain valid.

Thank you again for your careful review.

Reviewer_UXTBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper evaluates how frozen, trained binary diagnostic models respond to ordered clinical severity/grading information within the same diagnostic label, and proposes this assessment as a retraining-free post-training auditing perspective. The authors conduct evaluations across 32 backbone–task combinations covering Liver US, Breast US, Knee X-ray, and Fundus images, as well as 11 public Breast US checkpoints.

Critical strengths

The authors systematically evaluate how trained diagnostic models respond to withheld clinical ordering information across multiple modalities and architectures, offering an interesting perspective for post-training model auditing. If such an audit axis had a clear, actionable downstream usage, it could be of broad interest to the medical AI trustworthiness community.

Critical weaknesses
  1. Unclear practical utility and importance of the proposed audit axis. Although the authors show that clinical-signal alignment captures properties distinct from standard AUROC, it remains unclear why this axis is practically important or how it should be used in decision-making. Without demonstrating actionable utility (e.g., improved out-of-distribution generalization or model selection), the practical value of the audit remains limited.
  2. Unjustified reliance on isolated scale-band inputs and severe OOD confounding. It is unclear why the primary metric focuses on a single Laplacian pyramid scale band rather than the full image. Feeding a single, per-image [0,1] min-max normalized Laplacian band into a frozen model creates inputs that deviate from the training distribution. Consequently, low clinical-signal alignment may simply reflect a backbone's vulnerability to out-of-distribution frequency-band isolation rather than a failure to learn clinical signals on natural images, which undermines the paper's main conclusions.
Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Response to Reviewer UXTB

Thank you for noting that the audit axis could be of broad interest if it had a clear downstream use.

1. On the Actionable Downstream Use of the Audit Axis (Weakness 1)

As you point out, the manuscript did not show its practical use clearly enough. Diagnostic performance evaluates only whether the label is predicted, not the severity ordering within a label. In three tasks the label itself is a threshold of the clinical factor (§3.1), so a model that decides from the label-defining finding should respond more strongly as the finding becomes more severe. The audit flags models that do not, using forward passes only, and indicates how to fix them.

Liver US ResNet18, which has the highest full-image AUROC (96.2), is an example. The audit flags it as low-alignment (F_anchor|y = −0.24, Table 2). Removing the fine scale from the full image costs this model 41.2 AUROC points (within 5 points for the four high-alignment models), and blur, JPEG compression, and downsampling cost 37.7 points (Table 10). Retraining with the same split and seed after removing the fine band from a random half of the training images preserved performance (96.2 to 97.4) and reduced the corruption loss to 8.2 points. The same held for all 18 checkpoints of the two low-alignment backbones (ResNet18, EfficientNet-B0) in the three coarse-anchor tasks (mean AUROC 88.3 before and after; mean corruption loss from 28.2 to 9.7 points). All of these were measured on original and corrupted images, without band inputs.

The same outputs also support model selection among candidates with similar performance (Table 2), stress-test prioritization (more vulnerable models caught than with full-image performance at every budget, Table 15), and auditing public checkpoints without access to the training pipeline (§6.4). We do not claim improved external generalization.

2. On Band-Restricted Inputs and Distribution Shift (Weakness 2)

The concern that band inputs may depart from the training distribution is valid. We noted this in §4.3, but the manuscript did not present enough evidence that it does not explain the results; thank you for raising this.

Why bands. The full-image response mixes all scales and hides which scale the ordering comes from. In Liver US, the two models with the highest full-image alignment (EfficientNet-B0 +0.48, DenseNet121 +0.46) had low anchor-band alignment, and their full-image alignment vanished when the fine band was removed (−0.15, +0.06), whereas it did not decrease for the four high-alignment models. Selecting by full-image alignment alone would favor models that obtain their ordering outside the clinical scale.

Distribution shift does not explain the results. We tested whether the shift explains the results in three ways.

  1. Controlling for three shift indices computed without the clinical factor left the association between F_anchor|y and retention of clinical ordering unchanged (from +0.475 to +0.48–+0.51, all intervals excluding 0).
  2. Changing the normalization preserved the model ranking (Breast US panel, Spearman +0.98, B.3).
  3. Measurements without band inputs (fine-band removal, corruption, and the correction in Response 1) lead to the same conclusion.

Revisions. We will add the scale correction, the shift-index controls, and full-image alignment after band removal to Appendix D.1, referenced from §4.3.

Thank you again for the careful review. We hope this addresses your concerns and would appreciate your reconsideration of the score.

Reviewer_FPHuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes a retraining-free audit for frozen diagnostic classifiers that assesses whether the model’s output increases with disease severity among images of the same diagnostic class. Each image is decomposed into frequency bands, and the model is queried on the band assumed to contain the relevant clinical finding. The audit metric is defined as the within-class rank correlation between this output and an ordinal severity grade withheld from training.

The audit is evaluated on four tasks (liver US, breast US, knee X-ray, and fundus photography), each with eight backbones trained with three seeds, and is applied to 11 publicly available breast ultrasound classifiers.

Critical strengths
  • The experimental setup is clean and well controlled.
  • Practical applicability. The audit needs only forward passes of the frozen model. Linked to the experimental setup in the previous point, it has been well demonstrated on public checkpoints
Critical weaknesses
  • The clinical knowledge used by the audit reduces to the ordinality of the severity grade and a per-task choice between fine and coarse frequency bands. The clinical findings motivating the choices are not measured, so the audit cannot establish that the model responds to these findings or to any band-limited content correlated with severity.

  • As the authors acknowledge, F_anchor|y strongly correlates with the anchor-band diagnostic AUROC, A_anchor, a quantity that requires no clinical factors. The authors also state that, because of this covariation, the unique contributions of the two quantities cannot be interpreted separately. Their own results show that, once A_anchor is accounted for, F_anchor|y no longer predicts corruption vulnerability, and that A_anchor ranks models for stress testing as well or better than F_anchor|y. As a consequence, the motivation for introducing a metric that requires additional information becomes questionable.

Questions for authors
  1. Can the authors identify models with similar A_anchor but different F_anchor|y, and show that this difference predicts something A_anchor does not, such as performance on an external cohort or errors on clinically meaningful cases? This would directly justify the need for the clinical factor.
  2. Can the anchor-band response be related to direct measurements of the motivating findings?
  3. Are microaneurysms still visible after conversion to grayscale and subsampling?
1 comment
AuthorsOpenReview ↗

Response to Reviewer FPHu

Thank you for recognizing the controlled experimental design and the applicability to public checkpoints, and for pointing out specific weaknesses of the manuscript.

1. On Direct Measurement of the Motivating Findings (Weakness 1 & Question 2)

You are right that the manuscript does not directly measure the motivating findings, and we did not address this sufficiently. The audit uses only the grade order and a prespecified scale, so that it can be applied to graded cohorts without lesion annotation (§4.1). Of our four cohorts, only Breast US (BUS-BRA) provides expert lesion annotations (masks), so we measured the findings directly there. From the masks, we computed the shape findings listed in Table 1 (1 − solidity and depth/width ratio) without using model outputs or BI-RADS.

  • The prespecified coarse band followed the measured findings best (6 of 8 models; the fine band in none).
  • Alignment computed against the measured findings ranked the models almost identically to the BI-RADS-based F_anchor|y (Spearman +0.976).
  • Within the same BI-RADS grade, the coarse-band response still followed the measured findings (positive in 7 of 8 models), and the result was unchanged after controlling for lesion size.

The other three cohorts have no such finding annotations, and we leave them to future work, as stated in the Conclusion.

2. On What the Alignment Adds beyond Anchor-Band Discriminability (Weakness 2 & Question 1)

The observation that F_anchor|y (hereafter F) strongly correlates with A_anchor is correct. One might therefore regard A_anchor as a substitute for F, but we intended alignment to examine not the discrimination outcome but whether the model's response follows the clinical order within a label. We did not explain this distinction sufficiently in the manuscript. That the two are high together is also what the audit is designed to check. Its purpose is to judge the quality of black-box models, such as commercial models or public checkpoints whose training data and internals are inaccessible, using only clinically specified scales and forward passes (§6.4; 11 public checkpoints, Table 17). A model that decides from the clinical finding should separate the labels from that scale alone nearly as well as from the full image (A_anchor close to A_full), and its response should increase with finding severity within a label (high F). Good models therefore score high on both. For example, Liver US ConvNeXt-S and ResNet18 have similar full-image AUROC (95.5 and 96.2), but A_anchor is 83.3 versus 32.3 and F is +0.52 versus −0.24 (Table 2). Full-image accuracy alone would select ResNet18. Even after controlling for full-image performance, F explained corruption vulnerability (−0.693, Table 13).

Question 1. We reported these relationships in Tables 6, 9, and 15 but did not emphasize them sufficiently in the main text; thank you for pointing this out. Discrimination outcomes, such as external-cohort performance or errors on severe cases, are covered by A_anchor in the audit, and F adds no information beyond A_anchor for corruption vulnerability or stress-test prioritization (Tables 6, 15). What F covers separately is the clinical ordering within a label. After controlling for A_anchor and clean full-image alignment, F still predicted retention of clinical ordering under corruption (+0.475 [+0.17, +0.71], Table 9), and permuting the clinical factor drives F to zero while leaving A_anchor unchanged (B.2). This part cannot be measured without the clinical factor.

3. On Microaneurysm Visibility at the Audited Resolution (Question 3)

This is a fair point, and we thank you for raising it. After grayscale conversion and downsampling to 224×224 (C.1), the smallest microaneurysms span one or two pixels and are hard to resolve individually, and the fine band (period 2–6 pixels, Table 3) captures dot haemorrhages and larger microaneurysms. Consistently, alignment between DR 0 and DR 1, which differ only by microaneurysms, was near zero for all models (−0.04 to +0.06). In the revision, we will describe the Fundus finding in Table 1 as "small punctate lesions (dot haemorrhages and larger microaneurysms)".

Revisions. Lesion-finding measurements (§6.3, D.3), class-specific alignment (D.1), and the Fundus finding description in Table 1.

Thank you again for the careful review. We hope this addresses your concerns and would appreciate your reconsideration of the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.