acceptodds
Under review as a conference paper at ICLR 2027

DREAM: Error-Guided Learning for Robust Cancer Detection

Abstract

Segmentation models can surpass radiologists at detecting cancer on computed tomography (CT). However, models that perform well on standard datasets often degrade in real-world deployment, by producing more false negatives (missed tumors) and more false positives (false alarms). These errors concentrate on hard cases. Hard negatives are often non-cancer abnormalities that trigger false alarms. Hard positives are small early tumors that are often missed. To address these systematic errors, we propose Diagnostic Robustness by Error Analysis and Modeling (DREAM). The framework has three components. (1) Report-based mining: LLMs read 440,000 radiology reports to find hard negatives and hard positives. The corresponding CT scans are added to training, so the segmentation model learns from the cases it most often gets wrong. (2) Report-refined masks: an algorithm refines AI-generated tumor masks to match the tumor size, count, and location described in the radiology report. This avoids time-consuming manual annotation. (3) A cascade verifier combines the tumor segmentation model's output with a radiomics-based classifier of non-cancerous abnormalities to improve the tumor probability estimate and reduce the risk of mistaking other abnormalities for cancer. We evaluated DREAM on the detection of tumors in the pancreas, bladder, and gallbladder. DREAM consistently outperforms 13 state-of-the-art methods and public AI models on internal and external validation. Gains over the nnU-Net baseline reach up to +6.7 pp in AUC on tumor detection, and DREAM improves specificity by up to +40.7 pp on non-cancer abnormalities. In a reader study, DREAM detected 2.4 times more early pancreatic tumors than the average of three radiologists at matched specificity. These results show that AI errors cluster into systematic, identifiable patterns. Targeted mining, report-refined masks, and abnormality-aware verification are effective strategies to reduce these errors. Code and models will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
medical image segmentation, computed tomography

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vthMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DREAM (Diagnostic Robustness by Error Analysis and Modeling), a framework designed to improve the robustness of CT-based cancer detection by addressing two systematic failure modes: false negatives on small early-stage tumors and false positives caused by non-cancerous abnormalities. DREAM consists of three components. First, an LLM-based pipeline mines challenging positive and negative cases from approximately 440,000 radiology reports. Second, report-refined masks incorporate tumor size and anatomical location information from clinical reports to improve automatically generated segmentation pseudo-labels. Third, a cascade verifier combines segmentation-derived evidence with radiomics-based abnormality classification to refine tumor detection scores. The framework is evaluated on pancreatic, bladder, and gallbladder tumor detection using internal and external cohorts. The authors report substantial improvements over representative segmentation baselines, particularly on challenging cases, and provide component ablations, cross-backbone evaluations, and a radiologist reader study.

Critical strengths
  1. Clinically meaningful problem formulation and scalable hard-case acquisition.

The paper addresses two important failure modes of medical AI systems that are often insufficiently represented in conventional benchmark evaluations: missed early tumors and false alarms caused by non-cancer abnormalities.

The report-based mining pipeline provides a practical approach to identifying rare and diagnostically challenging cases without exhaustively processing large hospital imaging archives. Integrating report-derived information into training data construction is particularly valuable in settings where dense expert annotations are expensive.

The proposed framework therefore offers a meaningful systems-level contribution to improving medical AI robustness.

  1. Strong empirical validation across multiple organs and external cohorts.

The evaluation is a major strength of this submission. DREAM demonstrates improvements across pancreatic, bladder, and gallbladder tumor detection, including external validation on independently collected datasets.

The authors provide controlled ablations showing the contributions of hard-case mining, report-refined masks, and cascade verification. Additional experiments indicate that the training strategy can also benefit alternative segmentation backbones.

The hard-negative specificity analysis and small-tumor detection results are especially relevant because they directly evaluate the failure modes motivating the proposed approach.

Critical weaknesses
  1. Insufficient independent validation of report-refined masks and limitations of the cascade verification mechanism.

Although report-refined masks are an important component of DREAM, their spatial accuracy is not sufficiently validated against expert annotations.

The proposed refinement procedure uses reported tumor diameters and anatomical locations to constrain existing model predictions. However, matching the approximate tumor size and organ subregion does not guarantee accurate three-dimensional localization or boundary delineation, particularly when the initial segmentation model produces unreliable predictions.

The quality-filtering strategy relies on agreement between the original prediction and the refined pseudo-mask. Such agreement does not necessarily establish anatomical correctness, since both representations depend on the same underlying model. The manuscript mentions radiologist review of only 20 masks for selecting the filtering threshold. A larger independent expert-annotated evaluation would provide stronger evidence for the reliability of the generated supervision.

The cascade verifier also has a conceptual limitation. According to Eq. (2), when the segmentation score and cascade score differ by at least 0.5, the correction term becomes zero, and the final prediction remains identical to the original segmentation score. Therefore, strong disagreements cannot be corrected by the verifier, even when they arise from a potentially severe segmentation error.

Furthermore, the Trust Gate relies on rank correlation with the original segmentation score, which does not necessarily measure whether a cascade branch can correct segmentation mistakes. Additional comparisons with alternative calibration and fusion strategies would help establish the necessity of the proposed design.

  1. Some evaluation claims require more careful qualification and clarification.

The external validation is encouraging, but several aspects limit the interpretation of the results.

First, the PanTS benchmark uses binary pancreatic-lesion annotations without distinguishing malignant from non-malignant lesion subtypes, as acknowledged in Appendix G. Consequently, performance on this dataset supports pancreatic lesion detection rather than specifically establishing pancreatic cancer detection performance.

Second, the radiologist reader study includes only 74 enriched cases and three readers, two of whom are relatively junior. Although the reported improvement is promising, the small study and absence of prior imaging or clinical context limit its generalizability to routine radiological practice.

Third, the reported E-Coast training cohort size differs between the main text (3,151) and Appendix G (3,440). The authors should clarify the origin of this discrepancy and specify the exact cohort used for each training stage.

Finally, the detailed component ablations primarily focus on pancreatic tumor detection. Although the full pipeline is evaluated across three organs, organ-specific ablations would strengthen the claim that each component contributes consistently across different cancer types.

These concerns do not invalidate the reported improvements, but resolving them would improve the rigor and interpretability of the clinical and methodological claims.

Questions for authors

Q1. Reliability of report-refined pseudo-labels.

Could the authors provide a quantitative comparison between report-refined masks and radiologist-drawn tumor annotations on an independent subset of cases? In particular, how frequently does matching the reported tumor diameter result in incorrect tumor localization or boundary delineation? Such an analysis would help establish whether the proposed quality-filtering criterion reliably identifies inaccurate pseudo-labels.

Q2. Design and limitations of the cascade verifier.

Equation (2) suppresses the correction term when the segmentation and cascade scores strongly disagree. How does the verifier handle high-confidence false-positive predictions in which the radiomics classifier correctly indicates a non-cancer abnormality? Could the authors compare the proposed trust-gated fusion against standard weighted averaging, learned calibration, and alternative confidence-based fusion strategies? A component-level analysis of hard-negative cases would be particularly informative.

Q3. Generalizability of the three components.

The detailed ablation study focuses primarily on pancreatic cancer detection. Could the authors provide corresponding component-level results for bladder and gallbladder tumor detection? This would help determine whether the improvements from report mining, pseudo-mask refinement, and cascade verification are consistent across organs or primarily driven by the larger pancreatic cohort.

Q4. Cohort definitions and clinical interpretation.

Could the authors clarify the discrepancy between the E-Coast sample counts reported in the main text and Appendix G? Additionally, given that PanTS contains pancreatic-lesion annotations without subtype labels, could the authors more explicitly distinguish lesion-detection results from malignancy-specific detection claims?

5 comments
AuthorsOpenReview ↗

W1 & Q1. Report-refined masks are not validated against expert annotations, and the quality filter compares each mask with the same model's output. On an independent expert-annotated set, how often are refined masks wrongly placed or delineated, and does the filter reliably catch inaccurate pseudo-labels?

Thank you; we ran this comparison on an independent expert-annotated set. Report-refined masks are inexpensive silver-standard labels: they need no manual drawing, but they are approximate masks, and the quality filter aims to remove inaccurate ones. Even so, training with them performed better than training with the model’s raw masks.

On an independent public cohort (PANORAMA: 365 PDAC CTs, where lesions were manually delineated by trained investigators supervised by an expert pancreatic radiologist), we ran the refinement exactly as shipped, giving it only the initial model’s output, the organ (pancreas), and a tumor diameter measured from the expert mask as a radiologist would report it. We quantitatively compared the raw and retained refined masks against the expert annotations:

MaskCasesDiceNSD 2 mmHD95No overlap
Raw (model’s thresholded mask)3650.470.3628.5 mm6.3%
Refined + quality filter (kept)990.570.4114.4 mm0%

DREAM’s label pipeline keeps 99 of 365 masks (27%). The retained refined masks have better aggregate agreement with the expert annotations than the raw masks evaluated across the full cohort: median Dice 0.57 vs 0.47, NSD 0.41 vs 0.36, and HD95 14.4 vs 28.5 mm. No retained mask had zero overlap with the expert annotation, compared with 6.3% of raw masks.

These results quantify the quality of the pseudo-masks retained by the combined refinement-and-filtering pipeline and its trade-off between mask quality and training-set coverage. The retained masks remain approximate labels rather than expert-quality delineations.

AuthorsOpenReview ↗

W1 & Q2. Eq. (2) makes no correction when the segmentation and cascade scores differ by 0.5 or more, so the verifier cannot fix a confident segmentation error. How are such false positives handled, how does the rule compare with weighted averaging, learned calibration and confidence-based fusion, and what does each component do on hard negatives?

(1) High-confidence false positives that the radiomics arm flags as non-cancer.

DREAM targets these before the verifier, through hard-negative mining: we fine-tune the segmentation model on 673 report-mined hard negatives. On 656 pancreatic non-cancer abnormalities, the full DREAM pipeline raises specificity over the nnU-Net baseline from 57.0% to 74.4%, with gains up to +40.7 percentage points (pp) on cancer look-alikes such as fluid collection (Table 2; see (3) for the components).

When the segmentation and cascade scores strongly disagree, the verifier leaves the segmentation score unchanged by design. When the segmentation score D and the cascade score C differ by 0.5 or more, the agreement factor in Eq. (2) is zero, so the correction is zero. On PanTS and N-California, all four non-cancer scans with D ≥ 0.9 (three on PanTS, one on N-California) had C < 0.5 and received zero correction. These four cases therefore remain uncorrected by the verifier.

We retain segmentation as the primary score because spatial supervision can improve robustness to spurious correlations [1], while background bias can impair classifier generalization [2]. For example, suppose all cancer scans came from hospital A and all normal scans from hospital B. A classifier could then detect cancer from a hospital mark in the image. Segmentation supervision additionally requires the model to localize the tumor, which the mark alone does not identify. This provides an additional constraint against such shortcuts, although it does not eliminate them. DREAM therefore allows the classifiers to make bounded adjustments to the segmentation score.

Our fusion experiments show the generalization trade-off. A weighted average whose weight we learned on the calibration split put all of the weight on the arms. It raised AUC on N-California, the hospital system the classifiers were trained on, from 92.77 (D alone) to 98.25, but lowered AUC on the external PanTS cohort from 90.69 to 83.45. Our rule scored above D alone on both (97.29 and 91.47; see (2)).

(2) Comparison with weighted averaging, learned calibration and confidence-based fusion.

The table compares our rule with four alternatives on the three test cohorts. We fitted every weight, cutoff and gate decision on the 543-scan calibration split and froze them before scoring any test cohort. Values are AUC (%), with the change versus our rule in percentage points shown in parentheses.

RulePanTSN-CaliforniaEuro
D alone (segmentation only)90.7 (−0.8)92.8 (−4.5)96.2 (−0.9)
Ours: Eq. (2)91.597.397.1
Weighted average, weight learned on the calibration split83.5 (−8.0)98.3 (+1.0)92.5 (−4.6)
Logistic fusion of D and all three arms49.9 (−41.6)95.1 (−2.2)68.7 (−28.4)
Confidence-based selector87.8 (−3.6)93.9 (−3.4)92.9 (−4.2)

Values are AUC (%), with differences versus our rule in percentage points

The learned weighted average, logistic fusion and confidence selector fall below D alone on PanTS and Euro, where the classifier arms transfer poorly. The learned calibration we tested preserved the ranking of D, so its AUC matched D alone.

(3) What each component does on hard negatives.

Hard-negative mining directly introduces cancer look-alikes into segmentation training, and how the cases are mined matters. Mining hard cases from images instead of reports leaves more false alarms on cancer look-alikes: specificity on duct dilatation falls from 69.6% to 59.4%, and on atrophy from 86.4% to 77.2% (Appendix E).

The verifier can move a score by at most 2.5 percentile points, and only when D and C differ by less than 0.5. It therefore makes small adjustments to the ranking but cannot correct the high-scoring false positives for which segmentation and cascade scores differ by 0.5 or more.

References

[1] Hooper, Sarah, Mayee Chen, Khaled Saab, Kush Bhatia, Curtis Langlotz, and Christopher Ré. “A case for reframing automated medical image classification as segmentation.” NeurIPS (2023).

[2] Bassi, Pedro R. A. S., Sergio S. J. Dertkigil, and Andrea Cavalli. “Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization.” Nature Communications 15, 291 (2024).

AuthorsOpenReview ↗

W2 & Q3. The component ablations cover only the pancreas. Do report mining, mask refinement and cascade verification also help bladder and gallbladder detection, or are the gains driven by the larger pancreatic cohort?

Thank you for this suggestion. We extended the ablation study to the external Euro cohorts for bladder (171 tumors, 227 non-cancer scans) and gallbladder (28 tumors, 243 non-cancer scans). The full DREAM pipeline achieves the highest AUC and sensitivity among the evaluated configurations for both organs, supporting its generalizability beyond pancreatic cancer.

We report AUC and sensitivity at a 95% specificity target, selecting the lowest threshold achieving at least 95% specificity on each test cohort.

ModelBladder AUCBladder sensitivity (%)Gallbladder AUCGallbladder sensitivity (%)
nnU-Net, no fine-tuning0.80725.70.84346.4
Random cases instead of report-mined0.80924.80.81839.3
Report-mined cases, no refinement0.81026.40.83439.3
DREAM, full pipeline0.83429.00.89553.6

Compared with the base nnU-Net model, DREAM improves AUC by 2.7 percentage points for bladder and 5.2 percentage points for gallbladder, with sensitivity gains of 3.3 and 7.2 percentage points, respectively. It also outperforms both random-case fine-tuning and report-mined training without refinement on both metrics in both organs. Thus, although the magnitude of improvement varies across organs, the benefit of the complete DREAM pipeline is observed in both additional external cohorts and is not confined to pancreatic cancer.

Reviewer_wvPkOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DREAM improves CT tumor detection by targeting systematic false positives and false negatives. An LLM pipeline searches 440,000 reports for candidate hard cases, followed by temporal report review and image-based error selection. Report descriptions of tumor location, size, and count refine model-generated masks for training. A cascade then combines segmentation-derived features and radiomics classifiers, using a correlation-based trust gate and a bounded score adjustment. The framework is evaluated for pancreas, bladder, and gallbladder tumors. The principal pancreatic external result improves PanTS AUC from 0.824 to 0.915 and segmentation DSC from approximately 0.43 to 0.60. Appendix E includes equal-size random-sampling and image-mining controls, mask-refinement ablations, and robustness checks. A small reader study reports higher sensitivity than three radiologists on an enriched pancreatic cohort.

Critical strengths

S1 The work addresses a consequential data bottleneck with a testable workflow

Using inexpensive text to identify rare candidate failures before retrieving heavy 3D images is well motivated. More importantly, the paper does not rely only on a comparison to an unchanged baseline. Table 3 replaces mined cases with the same number of random positives and negatives and includes an image-mining alternative with matched fine-tuning settings. Without the cascade, report-refined masks raise PanTS AUC from 0.862 to 0.907 and DSC from 0.40 to 0.60. That is meaningful evidence that the report information adds supervision beyond naive thresholded masks. The contribution can be valuable even without a new segmentation architecture. [Secs. 2.1-2.2; Appendix E, Table 3, pp. 16-17.]

S2 There is substantial evidence beyond a single internal benchmark

DREAM is evaluated on external pancreatic data and on two additional organs. The reported pancreatic Euro AUC improves from 0.940 to 0.971, while the PanTS improvement is approximately nine AUC percentage points. Appendix E also shows gains when the training recipe is applied to other backbone families. These results make a purely internal overfitting explanation less likely. The authors appropriately acknowledge that the reader study is small and enriched, and Appendix G explicitly discusses limitations of the PanTS lesion labels and size proxy. Those qualifications are useful and should be retained in the main claims. [Table 1 and caption, p. 8; Table 4, p. 18; Appendix G, pp. 18-19.]

Critical weaknesses

W1 The verifier is trained on a different discrimination problem from the one motivating it

Appendix C trains Cseg on 1,500 tumor-positive scans and 739 strictly clean-healthy negatives; Ctumor_abn uses the same pool and filtering. Yet the stated purpose of the verifier is to distinguish cancer from non-cancer abnormalities. Removing those abnormalities from the negative class removes precisely the difficult contrast the verifier is meant to resolve. The justification is particularly concerning: excluding such cases changes the association between duct-volume features and tumor status from AUC 0.31 to 0.78, described as making the association clinically correct. That reversal demonstrates dependence on the sampled population; it does not show that the resulting association will distinguish cancer from pancreatitis. Although the upstream radiomics classifier includes abnormalities, the downstream decision rule is still fitted against clean-healthy negatives.

The trust gate does not settle this issue. It retains arms that agree in rank with the segmentation score on development data, rather than arms that correct its mistakes. Agreement can retain a shared bias, while a genuinely corrective signal can disagree with the baseline. If every arm fails, Eq. 1 falls back to Cseg anyway. The development-set rank transform also makes Eq. 2's adjustment an operation on percentiles, not calibrated cancer probabilities. The reported AUC gains remain meaningful as ranking results, but they do not by themselves establish either calibrated risk estimates or the claimed abnormality-aware verification mechanism. The large N-Cal. cascade increment and much smaller PanTS increment make this distinction consequential. [Sec. 2; Appendix C; Table 3.]

W2 The evaluation changes the target population and operating point across key claims

The small-tumor comparison needs a common denominator before its sensitivity differences can be interpreted. Sec. 3 and Appendix G describe 94 pancreatic tumors and 952 negatives in N-California, whereas Table 5 reports cohorts of 1,051 and 993 scans with 99 and 94 tumors. It is unclear whether these are model-specific eligibility sets, different versions, or reporting errors. If case inclusion depends on the model, a difference in sensitivity mixes prediction quality with a change in case difficulty. Size stratification adds another dependency: Appendix G reports that half of the 26 cases with an independent size measurement change stratum under the mask-derived proxy. Small mask volume and independently measured small cancer are therefore not interchangeable endpoints. PanTS also includes pancreatic lesions beyond malignant cancer, as the authors acknowledge.

The operating-point comparisons require similar care. Table 1 uses validation-frozen thresholds, while Table 3 reports F1-optimal sensitivity/specificity without identifying the selection set. On PanTS the full model consequently appears at about 87%/84% in Table 1 and 75.5%/96.0% in Table 3. Those are not contradictory if the thresholds differ, but a specificity increase at a different sensitivity does not isolate improved discrimination in the intended operating region. This also affects the matched-specificity reader claim: with 31 negatives, 96.8% specificity corresponds to one false positive, so the threshold-selection procedure is central to the claimed 2.4-fold sensitivity advantage. AUC improvements support ranking quality; they do not automatically validate each threshold-specific or early-cancer claim. [Tables 1-3 and 5; Appendix G.]

Questions for authors

Q1 What population does the verifier learn to distinguish

Why are benign abnormalities excluded from the negative training class of both Cseg and Ctumor_abn when resolving those abnormalities is the stated objective? What in the existing evaluation distinguishes a tumor-versus-abnormality decision from a tumor-versus-healthy decision? Please also explain why agreement with D is a reliability criterion and why Cseg remains usable when it fails that criterion. The answer should reconcile the actual sampling and fusion rules with the proposed mechanism.

Q2 Are the reported comparisons on the same patients and decision rule

Please reconcile the N-California counts in Table 5 with the 94-positive/952-negative cohort, and the E-Coast totals of 3,151 and 3,440. Was small-tumor eligibility fixed independently of model predictions and applied identically to every comparator? For the Table 3 F1-optimal values and the reader-study matched-specificity result, which cases were used to choose the threshold, and was it fixed before the evaluated cases were scored?

Q3 How do the training and calibration pools connect

Were segmentation training, radiomics training, cascade-arm training, meta-learner fitting, and the 543-case calibration set separated at the patient level from every test cohort, including longitudinal scans? Were the meta-learners fitted to out-of-fold base-model predictions or in-sample predictions? The stated within-component splits do not establish independence of the complete pipeline; a clear account of these existing data flows would resolve this concern.

Q4 What information remains in a report refined mask In Sec. 2.2, the search sphere has the reported tumor diameter, and the fallback selects K voxels corresponding to the tumor volume inferred from that diameter. What volume model determines K? If it is the volume of that same sphere, selecting its top K voxels would approximately fill the sphere and leave little dependence on the probability map. Please clarify the implemented rule. Also, Table 3 perturbs diameter but does not address a wrong location or a wrongly linked longitudinal report. Which of these errors can the quality filter actually detect, given that its reference is the same model output used to construct the mask?

7 comments
AuthorsOpenReview ↗

W1 & Q1. The verifier's arms train against clean-healthy negatives, although the verifier exists to separate cancer from non-cancer abnormalities. Why is agreement with D a reliability criterion, why fall back to Cseg when an arm fails, and does Eq. (2) produce calibrated probabilities?

Thank you for the careful reading. Each part of DREAM has its own job and its own training data. The segmentation model, which produces the tumor score D, learns to reject non-cancer abnormalities: we fine-tune it on 673 report-mined hard negatives, non-cancer scans on which an earlier model raised a false alarm. The radiomics classifier also learns abnormalities. It predicts 17 classes, including tumor, pancreatitis and duct dilation, and the arm Ctumor is its tumor output, trained against all other scans, abnormal ones included. The arms Cseg and Ctumor_abn have a narrower job: they confirm tumor versus normal pancreas. Cseg reads 26 measurements of the segmentation mask, such as duct and pancreas volumes, and Ctumor_abn reads all 17 outputs of the radiomics classifier. Finally, Eq. (2) keeps D in charge and lets the arms adjust it slightly.

"Although the upstream radiomics classifier includes abnormalities, the downstream decision rule is still fitted against clean-healthy negatives."

Correct; this is by design because adding abnormal cases to the training sets for Cseg and Ctumor_abn does not improve detection. Both Cseg and Ctumor_abn are fitted on tumor scans versus clean negatives; Ctumor_abn sees abnormalities only through its inputs, the 17 classifier outputs. These two arms are tumor-versus-normal arms with a limited role, not validated cancer-versus-abnormality discriminators. We kept abnormal scans out of Cseg's negatives because pancreatitis shares the features Cseg reads: it also dilates the duct and enlarges the pancreas, and with pancreatitis among the negatives, the duct-volume features reverse direction (AUC 0.31 versus 0.78; current Appendix C). Separating cancer from abnormalities is the job of the segmentation model, trained on the hard negatives, and of Ctumor.

"It retains arms that agree in rank with the segmentation score on development data, rather than arms that correct its mistakes. Agreement can retain a shared bias, while a genuinely corrective signal can disagree with the baseline."

Segmentation-based detection transfers across hospitals better than image-level classifiers [1]. DREAM therefore trusts the segmentation score and lets the classifiers adjust it, not overrule it. The gate keeps an arm only if its ranking agrees with D on 543 held-out calibration scans (Spearman ρ > 0.20). Agreement with D is a consistency check, not a guarantee of correctness: it can keep an error the arm shares with D or drop a corrective signal.

Letting the classifiers correct D costs generalization. We tested a weighted average whose weight we learned on the calibration split; the learned weighting put all the weight on the classifier arms. This rule raised AUC on N-California, the hospital system the classifiers were trained on, from 92.8 (D alone) to 98.3. It lowered AUC on the external PanTS cohort from 90.7 to 83.5. Our rule raised AUC on both: to 97.3 on N-California and 91.5 on PanTS. Eq. (2) caps the change at γ/2 = 0.025, which is 2.5 percentile points of D, and sets it to zero when D and the arms strongly disagree (|D − C| ≥ 0.5). An arm that transfers poorly therefore cannot pull D far.

"If every arm fails, Eq. (1) falls back to Cseg anyway."

The fallback is a fixed design choice with bounded influence. Cseg is the only arm computed from the segmentation output alone, and Eq. (2) still anchors the final score on D: it moves D by at most 0.025, and not at all when Cseg and D strongly disagree. This bound limits the influence of the fallback arm; our external-cohort results evaluate the resulting pipeline empirically.

"The development-set rank transform also makes Eq. (2)'s adjustment an operation on percentiles, not calibrated cancer probabilities."

Agreed. D and C are percentiles on the calibration split, so Eq. (2) adjusts percentiles. We report discrimination and detection performance, including AUC and sensitivity/specificity, but do not interpret the output as a calibrated cancer probability; the revision calls it a tumor score.

Non-cancer abnormalities in the evaluation. On 656 N-California non-cancer scans with pancreatic abnormalities, specificity rises from 57.0% (nnU-Net) to 74.4% (DREAM). The largest gains are on cancer look-alikes: fluid collection (+40.7 pp), duct dilatation (+23.2 pp) and pancreatitis (+20.0 pp) (Table 2).

Reference

[1] Hooper, Sarah, Mayee Chen, Khaled Saab, Kush Bhatia, Curtis Langlotz, and Christopher Ré. "A case for reframing automated medical image classification as segmentation." NeurIPS (2023).

AuthorsOpenReview ↗

W2 & Q2. The N-California counts differ between Sec. 3 (94 tumors, 952 negatives) and Table 5, PanTS size uses a mask-derived proxy, and the Table 3 and reader-study thresholds do not say which cases set them. Are all comparisons made on the same patients, with a decision rule fixed before scoring

Counts. Thank you, and sorry for these count errors during a previous revision. The revision scores every model in Table 5 on the Sec. 3 list (94 tumors of 2 cm or less and 952 non-cancer scans). For E-Coast, Appendix G’s 3,440 should read 3,151, as in Sec. 3; DREAM fine-tunes on these plus 719 report-mined hard cases (46 positive, 673 negative), for 3,870 scans in total. The case inclusion criteria do not depend on the model.

PanTS size definition. Agreed. For simplicity, the submitted version sized each PanTS lesion by the sphere-equivalent diameter of its reference-mask volume. We now use the lesion’s longest axial diameter, measured on the same reference mask with the published RadGPT code [1], which follows the WHO standard radiologists use to measure tumors. Under this definition, the ≤2 cm stratum holds 58 lesions (previously 67), and DREAM reaches an AUC of 0.859 vs. 0.739 for the nnU-Net baseline (difference 0.121, 95% CI 0.064 to 0.175), so the conclusion holds. Because PanTS also includes non-malignant pancreatic lesions, we relabel these rows “pancreatic lesions” and base all small-cancer claims on N-California, where malignancy is confirmed by pathology reports.

Operating points. The operating points differ between tables because the tables ask different questions. In Table 1, DREAM and nnU-Net are compared at the same specificity on PanTS (84.0%), and each is evaluated at its best balanced-accuracy point on N-California. SwinUNETR, MedNeXt, U-Mamba and Merlin are shown at their best-F1 points, and the public checkpoints (Universal Model, PANORAMA, nnU-Net (PanTS)) are evaluated at their released decision rules. We corrected the Table 1 caption accordingly. Table 3 shows DREAM at its best-F1 threshold, and its lower block compares all four variants at the same 95.4% specificity. The PanTS points in Table 1 and Table 3 are therefore two points on the same DREAM ROC curve (Fig. 4c), and they do not contradict each other. We want to emphasis that the comparison does not depend on these thresholds: AUC, which uses no threshold, also places DREAM above the nnU-Net baseline on both internal and external cohorts, and the full ROC curves are in Fig. 4. The revision adds, for every model, sensitivity and specificity at a cutoff fixed on a held-out calibration split before scoring the test cohorts. DREAM’s reader-study point uses the default rule of our released evaluator: a scan is positive if the largest predicted tumor exceeds 2 mm.

Reference

[1] Bassi, Pedro R. A. S., et al. "RadGPT: Constructing 3D Image-Text Tumor Datasets." ICCV (2025), pp. 23720–23730.

AuthorsOpenReview ↗

Q3. Were segmentation training, radiomics training, cascade-arm training, meta-learner fitting and the 543-scan calibration set separated from every test cohort at the patient level, including longitudinal scans, and were the meta-learners fitted out of fold?

Yes, by scan identifier, segmentation training, radiomics training, cascade-arm training, meta-learner fitting, and the 543-case calibration set are separated at the patient level from every cohort. And we established the separation at patient level and longitudinal scans as well. We fitted the meta-learners out of fold.

Following your suggestion, we added a new appendix that documents the data flow and separation for every stage.

Reviewer_S1YrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a new method for robust cancer detection on CT images. The main idea is to identify difficult positive and negative cases from large-scale radiology reports and use these cases to improve the segmentation model. This work also introduced report-refined masks to generate annotations without manual segmentation and a cascade verifier to reduce false-positive predictions. Experiments on pancreatic, bladder, and gallbladder cancers demonstrated the effectiveness of the proposed method.

Critical strengths
  1. This paper addresses an important clinical problem. Current cancer detection models may achieve good results on public benchmarks but still fail on difficult clinical cases. The proposed framework focuses on false negatives and false positives, which are meaningful to clinicians.
  2. The report-based mining strategy is interesting and practical. Compared with directly processing a large number of CT volumes, using radiology reports to identify difficult cases can significantly reduce the data collection and annotation cost. 3., The experimental results are promising. The authors evaluated the method on multiple internal and external datasets, including data from many hospitals. The improvements in detecting small tumors and reducing false positives are encouraging.
Critical weaknesses
  1. My main concern is the novelty of the proposed method. The framework mainly combines hard-example mining, pseudo-label refinement, radiomics classification, and score fusion. These components are not fundamentally new, although their combination is meaningful for the clinical problem.
  2. The method strongly depends on the quality of radiology reports. However, the reports may contain incorrect or incomplete descriptions, especially for small tumors that are difficult to identify. Therefore, the generated labels and masks may not always be reliable.
  3. The report-refined masks still depend on the predictions of the original segmentation model. If the model completely misses a tumor, it may be difficult to generate the correct mask only based on the tumor description in the report.
  4. Although the reader study is interesting, the number of radiologists and cases is relatively limited. The current results cannot directly demonstrate that the proposed method performs better than radiologists in routine clinical practice.
Questions for authors
  1. Clarify the main methodological contribution compared with existing hard-example mining and pseudo-label refinement methods. The authors should better explain what is new beyond combining these components.
  2. Please discuss how errors in radiology reports and original segmentation predictions affect the reliability of the generated masks, based on the current quality-control strategy.
  3. Please also clarify the limitations of the reader study and avoid over-claimed statements about the superiority over radiologists in real clinical practice.
6 comments
AuthorsOpenReview ↗

W1 & Q1. The framework combines hard-example mining, pseudo-label refinement, radiomics classification and score fusion, components that are not new on their own. What is the methodological contribution beyond the combination?

Thank you for your comment. The combination of components is not the main novelty. DREAM contributes two new ideas: mining hard cases from reports instead of images, and report-guided label refinement. Below, we compare it with related methods, theoretically and experimentally.

1. Report-based hard negative mining. Our method is the first to mine radiology reports for hard negatives. Previous works mine hard cases from images instead. However, as the reviewer mentioned, image-based mining, which runs the model over every image, is infeasible for medical data. For example, pancreatitis is rare (about 34 per 100,000 people per year [1]), but more common than pancreatic cancer. So, it is a major source of false alarms. Mining enough pancreatitis cases from images means downloading and running inference on hundreds of thousands of 3D CTs from hospital systems, possibly taking years. DREAM reads 440,000 reports with LLMs in five days and downloads only a few thousand hard scans.

Experimental result: Report-based mining outperforms image-based mining. In the appendix Table 3, we reported this comparison. Image-based mining lowers specificity from 96.0% to 91.7% and AUC from 0.907 to 0.895 in the PanTS dataset. The specificity gap grows on hard negatives: duct-dilatation specificity 69.6% vs 59.4%, atrophy 86.4% vs 77.2%.

2. Report-guided label refinement.

Previous report-guided label refinement methods mainly detect and remove false-positive lesions, but leave each segmented lesion as the model drew it. Our model removes false-positive lesions, recovers false negatives, and adjusts segmented tumors to match the tumor sizes in radiology reports. Therefore, DREAM exploits more report information in refining pseudo-labels (e.g., tumor sizes). In contrast, Bosma et al. [2] read only the number of significant lesions from the report and keep that many top-confidence candidates, filtering false positives, but not considering tumor size. [3] reads only the lesion location and keeps the candidates that fall in the reported region. Lesion-Harvester [4] uses no reports; it mines unlabeled lesions in DeepLesion with a proposal generator and a proposal classifier. DREAM edits each mask to match the report. For every reported tumor, it reads the diameter, places the tumor at the most likely position inside the reported organ, and keeps the K most probable voxels, with K set by the reported tumor volume. This improves tumor sizes and recovers tumors that a fixed threshold removes. DREAM then detects unreliable masks and discards them (see our answer to Q2). The revision adds this comparison and these citations to Related Work and Section 2.2.

Experimental result: thanks to your comment, we also included a comparison to the alternative pseudo-label refinement method from Bosma et al., on the PanTS dataset. We trained the same model with the same training data and recipe, and changed only the mask refinement method: none (thresholded model masks), Bosma et al., or DREAM.

Training labels for the 46 casesAUC (%)
Thresholded model masks89.4
Bosma et al. pseudo-labels90.1
DREAM refined masks (ours)91.5

[1] Xiao, Amy Y., Marianne L. Y. Tan, Landy M. Wu, Varsha M. Asrani, John A. Windsor, Dhiraj Yadav, and Maxim S. Petrov. “Global incidence and mortality of pancreatic diseases: a systematic review, meta-analysis, and meta-regression of population-based cohort studies.” The Lancet Gastroenterology & Hepatology (2016).

[2] Bosma, Joeran S., Anindo Saha, Matin Hosseinzadeh, Ivan Slootweg, Maarten de Rooij, and Henkjan Huisman. “Semisupervised learning with report-guided pseudo labels for deep learning–based prostate cancer detection using biparametric MRI.” Radiology: Artificial Intelligence (2023).

[3] Chen, Alex, Nathan Lay, Stephanie Harmon, Kutsev Ozyoruk, Enis Yilmaz, Brad J. Wood, Peter A. Pinto, Peter L. Choyke, and Baris Turkbey. “Location-based radiology report-guided semi-supervised learning for prostate cancer detection.” IEEE ISBI (2024).

[4] Cai, Jinzheng, Adam P. Harrison, Youjing Zheng, Ke Yan, Yuankai Huo, Jing Xiao, Lin Yang, and Le Lu. “Lesion-Harvester: Iteratively mining unlabeled lesions and hard-negative examples at scale.” IEEE Transactions on Medical Imaging (2021).

AuthorsOpenReview ↗

W2 & Q2. Reports can be wrong or incomplete, so the labels and masks DREAM builds from them may be unreliable. How do report errors affect the generated masks under the current quality control?

We agree, reports can be wrong or incomplete, and small tumors are the hardest case. DREAM limits how the two most damaging report errors reach the model: missed tumors and tumors that are not there, using follow-up report analysis and pathology confirmation. Our ablations show that the smaller errors that reach the model do not lower detection. The revision moves this ablation from the appendix (Table 3) to the main paper.

Minimizing report errors. The analysis of follow-up reports minimize missed tumors. For each negative case in our dataset (no cancer patient), we ensure the patient’s reports still show no cancer for at least 2 years after all the patient’s training scans. A missed cancer would likely become visible and be detected in these 2+ year follow ups. Therefore, longitudinal report analysis avoids training on reports where radiologists missed tumors.

Pathology confirmation. For positive cases, confirmation from pathology reports, the gold standard in cancer detection, ensures that all training reports contain real cancer, not false positives. Therefore, our methods out the most damaging errors that radiology reports can have: missed tumors and false positives. Sec. 2.1 (Step II, "Temporal report-based refinement") describes the follow-up rule and the pathology check. So the big report errors, missed tumors and tumors that are not there, are screened out before a mask enters training

Lesser errors can still affect the mask, such as imprecise locations and wrong tumor measurements. All 46 mined positives are tumors of 2 cm or smaller, the case the reviewer raises.

Location errors. A wrong location in the report never reaches the mask, so the quality control does not need to filter it: refinement reads only the organ from the report and centers the lesion on the peak of the model's tumor probability in that organ. On the other hand, a mask can still be misplaced when the model's own prediction is wrong. The quality control drops such a mask when the model shows no clear tumor: on PANORAMA, where expert masks exist, it rejected all 21 masks in the wrong location (see our answer to the following W3). It cannot catch an error the model shares. To measure what such errors cost, we moved training masks of mined positives between the pancreatic head and tail and retrained. Moving 4 of the 46 masks lowered PanTS AUC by up to 1.5%.

Size errors (current Table 3). We perturbed the reported diameters of the 46 positives by ±10%, ±25%, ±50% and ±100%, regenerated the masks and retrained. Against a retrain with unperturbed reports, no noise level lowered AUC by more than 0.01 on PanTS or N-California.

Reference

[1] Rolnick, David, Andreas Veit, Serge Belongie, and Nir Shavit. "Deep learning is robust to massive label noise." arXiv:1705.10694 (2017).

AuthorsOpenReview ↗

W3. Report-refined masks start from the original model's predictions. If the model misses a tumor completely, can the report description alone produce a correct mask?

We agree that the report alone cannot reliably produce a correct mask when the model provides no useful localization signal. Section 2.2 (“Discarding unreliable masks”) stated this limitation. Refinement corrects imperfect predictions, while a quality filter rejects masks with insufficient model support. The report constrains the number, size, and organ of the tumors; the model’s probability map determines their positions.

What refinement fixes. A tumor can be missed by a fixed threshold even when the model retains useful localization information. Often, the model’s probability at the tumor is below 0.5 but still the highest in the organ. A fixed 0.5 threshold then gives an empty mask, while report-guided refinement can recover the lesion.

What refinement cannot fix, and how quality control addresses it. When the model provides no useful localization signal, refinement may place the mask at an incorrect location within the organ. The quality filter assesses the model’s support for the refined mask and rejects masks with low scores, sacrificing training examples to reduce incorrect labels.

In our all-zero-prediction test, the refined mask was misplaced (Dice 0) and received a quality score of 2.9 out of 10, below the 4.78 cutoff, so it was rejected.

We also evaluated the filter on external PANORAMA scans with expert masks. Because reports were unavailable, we derived tumor diameters from the expert masks. The filter rejected all 21 incorrectly localized masks in this experiment. The filter is deliberately strict: it retains 27% of masks, trading training cases for cleaner labels.

These results demonstrate rejection of unsupported masks in the tested cases. The filter does not guarantee rejection of every localization error, particularly when the model confidently supports an incorrect location.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Thank You and Summary of Contributions

Dear Reviewers and AC,

Thank you for your careful reviews and the discussion. We replied to each review below and uploaded a revised PDF, with new and changed text in blue.

Contributions. DREAM learns from the rare cases that cancer detection models get wrong.

  • It finds them by reading radiology reports instead of images: an LLM read 440,000 reports in five days, a "scalable hard-case acquisition" (vthM) that reviewers found "interesting and practical" (S1Yr).
  • It writes the training masks from the same reports, using the size and location of each tumor, with no manual drawing. Manual masks for 3,125 pancreatic tumors took eight radiologists five years; report-refined masks add "supervision beyond naive thresholded masks" (wvPk).
  • A cascade verifier adds radiomics classifiers without losing generalization. Classifiers and radiomics often fail across hospitals, so when they disagree with the segmentation model, DREAM keeps the segmentation score. A new experiment added after your comments (Table 10) shows why: a standard combination of the segmentation score and the classifiers, with its weight learned on our calibration split, raises AUC by 5.5 pp on N-California, where the classifiers were trained, but lowers it by 3.7 to 7.2 pp on the two external cohorts. DREAM's rule uses the same classifiers to raise AUC on N-California (0.928 to 0.973) and Euro (0.962 to 0.971) without losing it on PanTS (0.907 to 0.915).

Results. DREAM raises specificity on non-cancer abnormalities by +17.4 pp on average (up to +40.7 pp), raises AUC on small (2 cm or less) pathology-confirmed pancreatic cancers from 0.919 to 0.973, and raises external PanTS lesion-detection AUC from 0.824 to 0.915, with significant gains in bladder and gallbladder tumor detection as well.

Why it matters. Medical AI datasets are usually random samples of hospital archives, where hard negatives and hard positives are rare: 1.75% of our reports describe pancreatic hard negatives and 0.05% hard positives. Due to this rarity, models seldom learn, for example, how to distinguish hard negatives from cancer. Yet pancreatitis (one type of hard negative, a cancer look-alike) alone is about seven times more common than pancreatic cancer, so a model that mistakes it for cancer raises more false alarms than true detections. We believe datasets should be built from the cases models get wrong, not sampled at random.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.