acceptodds
Under review as a conference paper at ICLR 2027

ReVision3D: Attribution-Guided Recursive Self-Improvement for 3D Medical Perception

Abstract

Recursive self-improvement (RSI) offers a promising path for overcoming the limited visual capability of current medical imaging agents. Yet applying RSI to volumetric imaging remains difficult: failures can arise from acquisition, perception, training recipe, or downstream inference, while self-generated feedback and logged trajectories provide little guidance on which component should change. We introduce **ReVision3D**, an RSI system that leverages 3D volumes with spatially grounded annotations to determine where visual evidence is lost and recursively improve the corresponding visual capability. A frozen language-model designer proposes revisions to acquisition, perception, training, or inference, while the verifier and system-level objective remain fixed. Our key insight is that an annotated volume forms an **exact replay world for view rendering and spatial verification**: unvisited views can be rendered on demand, and localized predictions can be checked directly against reference masks. This grounded feedback directs targeted revision, while only changes that improve beyond measured seed noise are retained. Each accepted change triggers renewed attribution, allowing the dominant bottleneck to shift across rounds. On abdominal CT, attribution identifies perception as the dominant remaining limitation. Revising that level enables ReVision3D to achieve 79% liver recall and 83% kidney recall at under 0.4 false positives per patient, outperforming the evaluated frozen multimodal foundation models, with the largest gains on small lesions.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Recursive self-improvement, reinforcement learning with verifiable rewards, vision-language models, replay environments, 3D medical imaging, multi-modal agent

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Y4oSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ReVision3D, an attribution-guided recursive self-improvement framework for 3D medical perception. Annotated CT volumes serve as replay environments for rendering previously unvisited views and verifying localized predictions against reference masks. The framework distinguishes failures in acquisition, perception, training, and inference, and uses a frozen LLM designer to propose targeted revisions. A fixed external objective and noise-aware acceptance criteria determine whether revisions are retained.

Experiments on LiTS and KiTS23 demonstrate substantial improvements over thirteen frozen vision–language models, supported by matched-protocol comparisons and component ablations. The revised reader achieves 79% liver-lesion recall and 83% kidney-tumor recall at fewer than 0.4 false positives per patient, with particularly strong improvements on small lesions. The analyses further show that recipe-level revisions primarily refine existing detections, whereas perception adaptation expands the detectable lesion set.

Critical strengths
  1. The paper addresses an important limitation of existing self-improvement approaches: knowing that a system failed does not reveal which component should change. Using annotated volumes for counterfactual view rendering and spatial verification provides a concrete, externally grounded mechanism for diagnosing failures. Separating proposal generation from a fixed evaluation and acceptance process also makes the improvement procedure more auditable. This conceptual contribution is the main reason for my positive assessment.

  2. The evaluation includes thirteen frozen VLM baselines, a matched dense-reading comparison, and ablations separating reading protocol, training recipe, and perception adaptation. These analyses support the conclusion that the gains are not simply explained by observing more slices. Particularly compelling is the finding that recipe-level improvements largely refine existing detections, whereas perception adaptation substantially expands the detectable lesion set. The large small-lesion gains and reporting of rejected revisions strengthen the empirical contribution.

Critical weaknesses
  1. Validation is restricted to two abdominal CT datasets, with test cohorts of 31 liver patients and 70 kidney patients. Some important subgroups are particularly small—for example, the kidney test cohort contains only 14 lesions in the 1–5 mL range. Although the reported gains are encouraging, these sample sizes limit confidence in subgroup robustness and generalization to different acquisition protocols, patient populations, or organs. Broader evaluation is the main missing evidence preventing a stronger recommendation.

  2. The paper explicitly states that the routing rule in Eq. (5) was formulated retrospectively and did not schedule the reported experiments. Consequently, the results provide stronger evidence for the effectiveness of the selected interventions than for the complete attribution-driven decision process. This does not negate the framework’s value, but it leaves uncertainty about whether the proposed routing procedure reliably selects useful interventions in a new setting.

Questions for authors
  1. Table 2 reports liver test (J) values of 0.71 and 0.54 for the two seeds of the designer-selected configuration. Are the overall and small-lesion recall improvements consistent across these seeds? Per-seed results with patient-level confidence intervals would help assess robustness given the limited test cohorts.

  2. Since Eq. (5) was formulated retrospectively, what procedure selected intervention levels during the reported experiments? Please distinguish decisions made by the designer from those made by researchers, and clarify which parts of the attribution-guided loop were executed automatically.

  3. Do the authors have any existing results on an independent cohort or acquisition-protocol subgroups that could help assess generalization? Such evidence would materially strengthen my confidence. Evaluation on additional organs or modalities would be valuable future work rather than a prerequisite for acceptance.

  4. The reproducibility statement mentions that code, designer prompts, playbooks, and trial registries are prepared for release. Will these materials, together with the trained reader adapters and evaluation scripts, be publicly available? Please clarify the release timeline and whether an anonymized repository is available during review. Open-source availability is important to my assessment because independently reproducing the recursive improvement process requires more than access to the final model.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Y4oS (Part 1 of 3)

We thank the reviewer for the thoughtful review and the accurate summary of the paper. We appreciate the recognition of the problem the paper addresses (knowing that a system failed does not reveal which component should change), of annotated volumes as a concrete, externally grounded mechanism for diagnosing failures, and of the separation between proposal generation and a fixed evaluation and acceptance process. We are also glad the reviewer found the presentation clear and the evaluation convincing, in particular the matched dense-reading comparison, the finding that recipe revisions refine existing detections while perception adaptation expands the detectable set, and the reporting of rejected revisions. We answer the four questions in order. The per-seed and subgroup numbers come from the saved test predictions of the submitted readers; no reader was re-run on test.

Response to Q1: Per-Seed Results with Patient-Level Confidence Intervals

Yes: detection is consistent across seeds. The table gives both seeds of the designer's configuration at the operating threshold chosen on development (the same threshold for both seeds and for the baseline: support ≥2 on liver, ≥1 on kidney), with 95% patient-bootstrap intervals.

Test setRecall [95% CI] @ FP/patient1–5 mL5–20 mL20–100 mL>100 mL
Liver, designer seed 00.79 [0.70, 0.86] @ 0.350.69 [0.52, 0.83]0.84 [0.68, 0.95]1.000.82 [0.50, 1.00]
Liver, designer seed 10.82 [0.73, 0.92] @ 0.870.75 [0.59, 0.92]0.81 [0.69, 0.95]1.000.91 [0.67, 1.00]
Liver, hand adapters0.52 [0.42, 0.62] @ 0.580.38 [0.24, 0.56]0.48 [0.29, 0.61]0.82 [0.71, 1.00]0.82 [0.50, 1.00]
Kidney, designer seed 00.83 [0.73, 0.93] @ 0.370.57 [0.33, 0.82]0.71 [0.52, 0.95]0.96 [0.87, 1.00]1.00
Kidney, designer seed 10.81 [0.70, 0.93] @ 0.530.50 [0.22, 0.79]0.71 [0.52, 0.95]0.96 [0.87, 1.00]1.00
Kidney, hand adapters0.70 [0.58, 0.83] @ 0.460.29 [0.07, 0.53]0.50 [0.30, 0.78]0.92 [0.80, 1.00]1.00

Overall recall is 0.79 and 0.82 on liver and 0.83 and 0.81 on kidney, and small-lesion (1–5 mL) recall is 0.69 and 0.75 on liver and 0.57 and 0.50 on kidney, against 0.38 and 0.29 for the hand-chosen adapters. Both seeds therefore expand the detectable lesion set by a similar amount. What differs between seeds is the false-positive rate at that threshold: seed 1 produces more false positives (0.87 vs 0.35 per patient on liver, 0.53 vs 0.37 on kidney). This explains the J values in Table 2 (liver 0.71 / 0.54, kidney 0.80 / 0.78, against 0.39 and 0.68 for the hand-chosen adapters): J enforces hard false-positive budgets, so under the ≤0.5 FP budget seed 1 moves one threshold step tighter, to 0.51 recall at 0.32 FP on liver and 0.75 at 0.17 on kidney. The seed effect is thus one of false-positive calibration, not of which lesions the reader can detect, and both seeds exceed the hand-chosen adapters on J on both organs. We will report both seeds in the abstract and Table 1 with these intervals. Three further seeds of the designer's configuration are training now, and we will add them to this table in this thread.

AuthorsOpenReview ↗

Response to Reviewer Y4oS (Part 2 of 3)

Response to W1 and Q3: Cohort Size and Generalization across Acquisition Protocols and Organs

The cohorts are the complete held-out test splits of both datasets; to quantify the subgroup uncertainty the reviewer points to, we add patient-bootstrap intervals to every per-size cell; the kidney 1–5 mL bin (n = 14) has an interval of [0.33, 0.82] for seed 0. On protocol: the test volumes span 0.7–1.5 mm slice spacing on liver and 0.5–5.0 mm on kidney. Recall of seed 0 at its ≤0.5 FP operating point (liver support ≥2; kidney support ≥1; hand adapters at theirs), by spacing group:

Spacing group (patients)Designer seed 0Hand adapters
Liver ≤1.0 mm (22)0.80 @ 0.32 FP0.25 @ 0.05
Liver 1.0–2.5 mm (9)0.74 @ 0.440.26 @ 0.11
Kidney ≤1.0 mm (14)0.75 @ 0.860.44 @ 1.07
Kidney 1.0–2.5 mm (8)1.00 @ 0.121.00 @ 0.25
Kidney >2.5 mm (48)0.82 @ 0.270.74 @ 0.31

Recall is higher or equal in every group (the eight kidney patients at 1.0–2.5 mm are a tie at 1.00, with fewer false positives); thin-slice kidney volumes draw more false positives for both readers (more slices are read), which we will note. An independent liver or kidney cohort is the natural next step, and the limitations section says so.

We do have one existing result on a further organ, which we add to the paper: the same readers were also trained and evaluated on pancreatic tumours (PanTS; 100 training, 20 development, 30 test patients), a much harder target on which the current candidate space is not sufficient. Test J is 0.05 with the frozen encoder and 0.08 / 0.13 for the two seeds of the designer's configuration, with 0.90 / 0.75 of lesions never detected on any slice. The attribution places the loss at perception, as on liver and kidney, but the perception revisions in the current candidate space reduce it only slightly (never-detected share 0.95 with the frozen encoder). This is the bounded case the third contribution describes: the loop locates the limitation and does not claim progress it has not verified. We add it as a third column of Table 1 and Table A6 and discuss it in Section 6.

Response to W2 and Q2: Selection of Intervention Levels

We did, at every decision point; the designer chose what to try within a level, and the fixed gate accepted or rejected. Recipes were searched first because that loop was implemented first; the vision-encoder adapters were our decision after the designer's playbook diagnosed a perceptual plateau ("hand, on designer's diagnosis" in Table A9); the perception-configuration search (Table A8) and the inference programs were the designer's under the gate. Eq. (5) was written after the liver-study decisions and scheduled nothing. Automated: proposal generation, training, evaluation, acceptance and playbooks. Not automated: the choice and order of levels. We will add a table of decision points with the decision-maker (given in full in our response to Reviewer i6HP).

We also ran a control during the discussion period: the perception search of Table A8 repeated with the attribution terms removed from the designer's prompt and feedback, everything else identical. The designer proposed the same six configurations in the same order and the same configuration was accepted. This matches the reviewer's reading: the results establish the effectiveness of the selected interventions and the value of attribution as a measurement of what they changed, and the accepted revision is robust to whether the designer sees the attribution terms. Whether the routing rule selects useful levels in a new setting is the question for prospective work, and the second contribution and the abstract are scoped accordingly.

Response to Q4: Code and Model Release

Yes. The supplementary zip already contains the core implementation (replay world, objective, attribution, acceptance rule, designer interface and inference programs, about 450 lines), a demo that runs on synthetic volumes without API keys, and the verbatim designer prompts, playbooks and 18 trial registries behind Tables A8–A11 and A16. We are preparing an anonymized repository with the full training and evaluation pipeline (data preparation, reader training, the three search levels, test scoring), the saved development and test reads of all four readers, from which every table can be recomputed without a GPU, and the scripts for the new analyses in this rebuttal; we will post the link in this thread during the discussion period. The trained adapters will be released with the camera-ready version.

AuthorsOpenReview ↗

Response to Reviewer Y4oS (Part 3 of 3)

Additional Update: Acceptance Rule at the Inference Level

While preparing the threshold details requested by Reviewer i6HP we tightened the acceptance test at the inference level, whose bootstrap had re-picked operating thresholds within each resample. The new rule chooses thresholds on other development patients; under it we re-scored all 60 logged proposals (none beats the reader alone at P ≥ 0.9) and re-ran the search; the one program per organ that the stricter rule accepts on development does not improve on the reader alone on test (liver exceeds every false-positive budget; kidney ΔJ +0.009, interval includes zero). We remove the inference-level test claims (Table 2 bottom rows and the corresponding sentences in the introduction and Section 5.3). The perception-level results, on which the reviewer's assessment rests, use a different test (retrained readers against the seed-noise floor, with integer thresholds that transfer from development to test) and are unaffected; the full table is in our response to Reviewer i6HP.

I thank the authors for the detailed response and the experiments. I appreciate the clarifications on the method's positioning and limitations. I therefore decide to maintain my score.

Reviewer_VKC1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes to apply the concept of recursive self-improvment to 3D medical vision-language models. The proposed method was evaluated on two public abdominal CT datasets for liver and kidney.

Critical strengths

None

Critical weaknesses
  • The paper contains many unsupported claims. Without solid demonstration or proof of the fundamental limitations of 3D medical VLMs at the begining of the paper, the proposed technical contributions are not properly motivated.

  • The work introduces three technical contributions including volumetric replay, attribution-guided RSI, and bounded visual self-improvement. However, the motivations under these designs are unclear. They are not properly formulated in the method section. They are not synergistically connected in the framework.

Questions for authors
  1. The logical flow of the abstract does not hold, and the core idea is unclear. The following questions need to be addressed: why an RSI system requires a frozen language-model designer and a verifier; why view rendering exists in an RSI system; what “attribution identifies perception” and “revising that level” means.

  2. In the introduction section, the authors claim that the model "observes only a small fraction of the available image space" and "a missing finding can result from failed acquistion, inconsistent detection ...". These are all unsupported claims that are neither discovered by references nor by experiments of this work. The authors should evaluate whether these claims can stand or not.

  3. The method section exists similar expression issues. It is difficult to interpret and the logical flow does not hold. The authors should define what is "replay", "attribution", and "revision" in their framework clearly. The logical flow also needs to be significantly improved.

4 comments
AuthorsOpenReview ↗

Response to Reviewer VKC1 (part 1 of 2)

We thank the reviewer for the time spent on our submission and for the direct feedback on its presentation. The review asks that the three components be defined and connected more explicitly, and that the framing claims be tied to evidence. We answer the three questions directly, point to where the evidence is in the submitted paper, and say how we will rewrite the abstract, introduction and Section 3.

Response to W2 and Q1: Motivation of the Components and Logic of the Abstract

Why a frozen language-model designer and a separate verifier. The system being improved is a visual reader (a 4B vision-language model) and the pipeline around it. The designer is a language model that proposes changes to that pipeline. It is frozen because what must improve is the reader's perception, not the proposer, and because the proposer must not be able to alter the measure of success. The verifier is the fixed, mask-grounded check of whether a reported location hits an annotated lesion (Section 3, grounded verification). Keeping designer, reader and verifier separate means no component can redefine what counts as a detection; every accepted change to the reader in the paper is a gain on this fixed check, above seed noise, on development patients the reader never trained on, and it carries over to the held-out test patients (liver recall 0.79 against 0.52 for the hand-chosen configuration and kidney 0.83 against 0.70, at the same development-chosen threshold).

Why view rendering. A lesion can be missed because no rendered slice showed it (acquisition) or because it was shown and not recognized (perception). Separating the two requires knowing what an unvisited slice would have shown. With annotated volumes that slice can be rendered after the fact and its lesion coverage computed exactly (Section 3, counterfactual exposure). Logged trajectories cannot provide this. This is what "replay" means here: the annotated volume is an environment in which any view can be rendered and any prediction checked.

"Attribution identifies perception" and "revising that level". Eq. (3) splits the lesions missed at a false-positive budget into three measured terms: not exposed by any rendered view, exposed but never detected on any slice, and detected on some slice but discarded when slice findings are merged. Table A6 gives the split: with the frozen encoder, 0.42 of liver lesions are exposed and never detected and 0.13 are detected then discarded; on kidney 0.45 and 0.00. The dominant term, exposed but never detected, is a perception failure by construction. "Revising that level" means changing the perception configuration: which vision-encoder blocks carry adapters, their rank, and the view scale (Section 3; Table A8). It is the only intervention that lowers the never-detected term (0.42 → 0.25 on liver, 0.45 → 0.14 on kidney, Table A6); recipe changes move only the discarded term (0.13 → 0.00; Table A9, Figure 3b).

Response to W1 and Q2: Evidence for the Claims in the Introduction

"Observes only a small fraction of the available image space." This describes the survey protocol under which the thirteen frontier models were evaluated and under which volumetric agents typically operate: eight axial slices per volume. On our test patients a dense read at 5 mm spacing covers 43 liver and 32 kidney slices per patient on average, so the 8-slice survey shows about one slice in five (19% on liver, 25% on kidney), and a smaller fraction of the native 0.5–1.5 mm slices. We will put these numbers in the introduction.

"A missed finding may result from failed acquisition, inconsistent detection, ..." This is the decomposition that Eq. (3) defines and Tables A1 and A6 measure, not an assumption. Table A1 tests the acquisition explanation directly: giving the frontier models slices through every lesion changes recall by 0.05 or less for nine of the thirteen, with the largest change +0.11 (GPT-6 Astra, 0.32 → 0.43), so for these readers the missed findings are not an acquisition failure. Table A6 then locates the loss for our reader at each stage. We will move a short summary of this evidence into the introduction so that each claim is tied to its table when first made.

AuthorsOpenReview ↗

Response to Reviewer VKC1 (Part 2 of 2)

Response to Q3: Definitions and Logical Flow of the Method Section

We will open Section 3 with a definitions box. Replay world: an annotated volume, in which any view can be rendered, its lesion exposure computed from geometry, and any prediction verified against the masks. Attribution: the Eq. (3) decomposition of the missed lesions into acquisition, perception and aggregation terms, computed with the replay world. Revision: a designer-proposed change to one of four levels (which views are acquired, the perception configuration, the training recipe, the inference program), kept only if it improves the fixed objective beyond seed noise on held-out development patients. The loop is then one sentence: attribute the loss, revise the level that owns it, verify, recompute the attribution, repeat. In the reported experiments the steps within a level (proposal, training, verification, acceptance) were automated and the choice of level was made by us from the measured attribution; the revision states this division explicitly. The three contributions correspond to these three terms, and we will present them in that order with a step-by-step walkthrough of Figure 1 and a rewritten abstract.

During the discussion period we also ran new experiments (per-seed results with confidence intervals, a search without attribution feedback, acquisition-protocol subgroups); they are summarized in our general response and none of them changes the perception-level results above.

AuthorsOpenReview ↗

Follow-Up on the Rebuttal about the potential concerns

Dear Reviewer VKC1,

Thank you for your time and for the direct feedback on the presentation. As the discussion period is closing, we would like to summarize briefly how we have responded:

  • Q1 and W2. We explained why the system separates a frozen designer, the reader and a fixed verifier, why view rendering is needed to separate acquisition from perception failures, and what "attribution identifies perception" and "revising that level" mean, with the supporting numbers (Table A6).
  • Q2 and W1. We tied each of the two introduction claims to its evidence (the 8-slice survey shows 19–25% of the dense read; Table A1 tests the acquisition explanation directly for thirteen models).
  • Q3. We gave explicit definitions of replay, attribution and revision, which open Section 3 in the revision together with a step-by-step walkthrough of Figure 1 and a rewritten abstract.

We also posted new experiments during the discussion period (per-seed results, controls for the search, additional seeds), summarized in the general response. If any part of the method is still unclear, we would be glad to clarify in the remaining time. If the clarifications address your concerns about motivation and logical flow, we would be grateful if you would consider updating your score.

AuthorsOpenReview ↗

Additional Follow-Up on the Rebuttal Concerns

Dear Reviewer VKC1,

With the discussion period closing today, we would like to thank you for your time and to ask, respectfully, whether our responses have changed your reading of the paper. Your concerns were about motivation, unsupported claims and logical flow. In brief:

The motivation in three sentences. A reader of CT volumes can miss a lesion because it was never shown the right slice, or because it was shown the slice and did not recognize the lesion. An annotated volume lets us tell these apart exactly, by rendering any slice after the fact and checking any prediction against the mask. We measure which failure dominates, change the component responsible, and keep the change only if a fixed, mask-grounded check improves beyond seed noise.

The claims are measured, not assumed. The 8-slice survey shows the model 19–25% of the slices a dense read covers. Giving thirteen frontier models slices through every lesion changes recall by 0.05 or less for nine of them (Table A1), so their misses are not an acquisition failure. For our reader, 0.42 of liver lesions and 0.45 of kidney lesions are shown and never detected with a frozen encoder; adapting the vision encoder lowers this to 0.25 and 0.14 (Table A6).

The result is robust. During the discussion we added five seeds, two equal-budget controls and protocol subgroups. Every converged seed exceeds every baseline seed on both organs (mean J 0.66 vs 0.37 on liver, 0.81 vs 0.66 on kidney), and the gain is reached whether the revision is proposed by the designer or drawn at random. The limits (two seeds that do not converge, and pancreas) are reported in the thread.

The revision opens Section 3 with explicit definitions of replay, attribution and revision and rewrites the abstract along the three sentences above. If these clarifications address your concerns, we would be sincerely grateful if you would consider updating your score. We would gladly clarify anything further today.

Reviewer_i6HPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReVision3D studies how to improve a VLM-based CT lesion-localization pipeline using mask-grounded feedback. Annotated volumes provide alternative views and spatial verification; a failure decomposition motivates revisions to acquisition, perception, training recipe, or inference. A frozen LLM designer proposes changes, which are evaluated under a fixed development objective and acceptance rule. The explicit rule for choosing the next intervention level is retrospective. Experiments on LiTS and KiTS23 report 0.79 liver-lesion recall and 0.83 kidney-tumor recall at 0.35 and 0.37 false positives per patient, respectively, for the selected reader. Same-protocol controls indicate that vision adaptation supplies the largest gain, particularly for small lesions; inference programs further improve the objective by combining successive readers.

Critical strengths

S1. Grounded and auditable feedback. The separation of geometric exposure, localized detection, and downstream retention provides a useful way to analyze failures (Section 3). Fixed spatial verification and a fixed system objective prevent candidate revisions from redefining success. Tables A10-A11 also report rejected searches and seed variation. This makes the limitations of local improvement visible and gives the framework value beyond reporting its best final checkpoint. S2. Meaningful evidence for improved perception. Table A3 compares readers under the same dense reading protocol, so increased slice coverage alone does not explain the gain. Table 2 reports test mean J of 0.75/0.66 across two seeds for the selected configuration, versus 0.54 for the hand-selected configuration. The lower never-detected shares in Figure 3b and the small-lesion results in Table 1 jointly support an expanded set of detectable lesions. These are useful empirical findings.

Critical weaknesses

W1. Eq. (5) was defined retrospectively and did not guide the reported experiments (Section 3), leaving the benefit of attribution-guided routing untested. The J = 0.529 recipe is also credited to the designer in Table A9 but called hand-designed in Table A10. Please clarify the actual search sequence and recipe source, and compare with an equal-budget search without attribution. W2. Table 1 claims recall at no more than 0.5 FP/patient, yet the same liver recalls in Tables A1-A2 correspond to 1.13 FP/patient for Gemini 3.1 Pro and 5.19 for Mistral Large 3. Please correct these operating points or the table description. The matched Gemini comparison in Table A3 still provides useful evidence.

Questions for authors

Q1. Clarification of W1: Please provide the actual sequence of level choices, proposal sources, configurations, and matched-seed acceptance results. Was attribution available before each choice? Are the J = 0.529 entries in Tables A9 and A10 the same recipe/checkpoint, and why are they credited differently? Q2. Validation of W1: Can a limited comparison on the existing tasks isolate attribution feedback from a fixed sequence or proposals without attribution under the same candidate space, data, seeds, and budget? Which conclusions currently rest on retrospective analysis? Q3. Clarification of W2: Are Table 1's survey values raw operating points or calibrated points satisfying the stated budget? Please give each model's corresponding false-positive rate and thresholding procedure, distinguishing development-selected thresholds transferred to test from test FROC envelopes used to summarize J.

6 comments
AuthorsOpenReview ↗

Response to Reviewer i6HP (Part 1 of 4)

We thank the reviewer for a careful and constructive review. We are glad the reviewer found the grounded, auditable feedback useful (the separation of geometric exposure, localized detection and downstream retention, with a fixed verifier and objective so that candidate revisions cannot redefine success), valued the reporting of rejected searches and seed variation, and judged the same-protocol comparisons to be meaningful evidence that perception improved beyond what denser reading explains. Both points the reviewer raises (W1, W2) are addressed below with the full record and with new results: we answer the three questions in order and report the comparison requested in Q2, which we ran during the discussion period.

Response to W1 and Q1: Search Sequence, Proposal Sources, and Tables A9–A10

StepLevelWho chose the levelWho proposed / decided within the levelOutcome (development J)
1Warm start, then RL (liver study)usfixed pipeline, no search0.379 → 0.463
2Recipe (liver study; Table A10)us: the recipe loop was implemented firstdesigner: 10 recipes + 2 full-length promotions; the fixed gate rejected all (two-seed floor 0.050)no change; playbook: "nothing beat noise", recall at support ≥1 pinned at 0.57–0.62, asks for "a lever on perception"
3Perception, 4 adapted vision blocks (liver study)us, after reading that playbook (Table A9: "hand, on designer's diagnosis")configuration chosen by us0.463 → 0.575, accepted against the seed floor
4Recipe again, from the adapter reader (Table A11)usdesigner: 4 proposals; gate rejected allno change
5Inference (liver study)usdesigner program over the system's readers; accepted by the P(ΔJ>0) rule0.575 → 0.608 (rule corrected, see Q3)
6Perception, multi-organ cohort (Table A8)usdesigner chose every configuration; accepted at trial scale, confirmed at full scale on two seeds0.25 → 0.48 (trial); 0.44 → 0.58 J_macro (full)
7Inference, multi-organ readersusdesigner programs; accepted by the same rulesee Q3

Was attribution available before each choice? At the perception level (step 6) the designer's trial feedback contained the Eq. (3) split per organ (never detected / discarded / kept; the designer prompt is in the supplementary material). At the recipe level (steps 2 and 4) the feedback contained J, the operating points and recall by lesion size; the playbook's diagnosis rests on recall at support ≥1, which is the complement of the never-detected term. The formal rule, Eq. (5), was written after steps 1–5 and scheduled none of them. In short: proposal generation, training, evaluation, acceptance and playbooks were automated; the choice and order of levels were ours. We will add this table to Section 4.

Tables A9 and A10 refer to the same reader and checkpoint: the hand-designed learnability recipe ("hand*", single seed) of Table A10. The A9 row label is corrected to match. The earlier label came from a three-seed replication at the 100-step trial length that the submitted version does not report: 0.525 ± 0.030 (designer's best recipe), 0.524 ± 0.015 (our recipe) and 0.485 ± 0.028 (naive failure mining), mean ± s.d. of J. We will add it as a table; it also shows that 0.529 lies within the hand recipe's own seed range. We will relabel the row "hand-designed recipe, single seed (designer's best recipe equal within noise)" and state that no designer recipe was accepted at this level. The trajectory's conclusion, that recipes moved precision but not the set of detectable lesions (maximum recall 0.57–0.58 through steps 1–4, 0.70 after step 3), is unchanged.

AuthorsOpenReview ↗

Response to Reviewer i6HP (Part 2 of 4)

Response to W1 and Q2: Isolating Attribution Feedback under an Equal Budget

Which conclusions rest on retrospective analysis: only two statements, that Eq. (5) would have selected the perception level at both decision points and that its ranking matches the observed order of gains. They are labelled as retrospective in Sections 5.2, 5.3 and 6. The acceptance decisions, the same-protocol controls and the test evaluation do not depend on Eq. (5).

We ran the second comparison the reviewer proposes: proposals without attribution under the same candidate space, data, seeds and budget. The perception-level search of Table A8 was repeated with the same designer (Claude Opus 5), the same playbook, the same training subset and development patients, the same two baseline seeds and noise floor, the same budget (six proposals, two replications) and the same acceptance rule. The only change: the per-organ attribution terms (never detected / discarded / kept) were removed from the system prompt and from the feedback on every trial, leaving J, the operating points and recall by lesion size.

Perception search, trial scaleWith attribution (Table A8)Without attribution (new)
Proposals 1–64 blocks at 2× views → 12 blocks → 24 blocks → rank 32 → 2 epochs → lr 3e-4the same six configurations, in the same order
Accepted configuration24 blocks, rank 32, 2× views, 2 epochsthe same
What the designer's rationales citenever-detected share, recall by sizerecall by size, operating points

Because the configurations coincide trial by trial, the outcome is identical. Within this level the attribution terms were not needed: recall by lesion size carried the same information for this designer. The accepted revision is thus robust to the form of the feedback (one run per arm). The role of attribution that the evidence supports is the one in Table A6 and Figure 3b: it measures what each revision changed, and shows that only perception revisions reduce the never-detected term. The second contribution, the abstract and Section 6 are stated in these terms in the revision. A random-search arm (six configurations drawn uniformly from the same space, same budget and rule) is training now and we will post it in this thread.

On the choice between levels, the record is this: under the same designer, gate and development patients, two recipe rounds (21 GPU trials) produced no accepted change, and ten perception trials produced an accepted change that replicated on a second seed and on test. This establishes that the levels are not interchangeable, which is the paper's empirical claim. A prospective test of Eq. (5) as a scheduler against a fixed order is the natural next experiment, and the revision scopes the claim to what is shown.

AuthorsOpenReview ↗

Response to Reviewer i6HP (Part 3 of 4)

Response to W2 and Q3: Operating Points and Thresholding Procedure in Table 1

Frontier models. They emit one set of findings without a confidence score, so each has a single raw operating point; Table 1 reports it, and the caption's "≤0.5 FP" applies to our rows only. Gemini 3.1 Pro's 0.35 recall is at 1.13 FP/patient and Mistral Large 3's 0.12 at 5.19 (Table A2; kidney in Table A5). We will add the FP column to Table 1, mark the frontier rows as raw operating points, and keep Table A3 as the matched-protocol comparison.

Our readers. J is a FROC-style summary: for each of four FP budgets, the best recall over the eight integer support thresholds of the dense read, a step function without interpolation, computed on the evaluated set. The per-size cells in Tables 1 and A3 use the most permissive support threshold within 0.5 FP/patient, chosen on the evaluated set. For the designer's reader (seed 0, the reader of Table 1) this threshold is identical on development and test (support ≥2 on liver, ≥1 on kidney), so the headline operating point is not test-selected. For the liver baselines and for the second designer seed, the development threshold (also support ≥2) yields more than 0.5 FP/patient on test, so the Table 1 rule places them one step tighter. We add development-transferred rows, where every reader uses the same development-chosen threshold:

Liver test (31 patients)Table 1 rule (threshold chosen on test)Development threshold carried to test
Frozen encoderrecall 0.19 @ 0.00 FP; 1–5 mL 0.060.38 @ 0.65 FP; 1–5 mL 0.19
Hand adapters0.25 @ 0.06; 1–5 mL 0.120.52 @ 0.58; 1–5 mL 0.38
Designer (seed 0)0.79 @ 0.35; 1–5 mL 0.69same threshold, same values: 0.79 @ 0.35; 1–5 mL 0.69
Designer (seed 1)0.51 @ 0.32; 1–5 mL 0.330.82 @ 0.87; 1–5 mL 0.75

The designer's advantage holds under both rules and for both seeds, and the paired macro-J gain (+0.21 [+0.14, +0.28]) does not depend on a single threshold. With a common development-chosen threshold the two designer seeds detect the same share of lesions (0.79 and 0.82; 1–5 mL 0.69 and 0.75, against 0.38 for the hand adapters) and differ in false-positive rate; the Table 1 rule widens the small-lesion gap to the baselines, so the revision presents both rules side by side, with kidney in the same format.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

Summary of New Experiments and Revisions

We thank the three reviewers. The reviews asked for evidence we could produce during the discussion period and we further add into our rebuttal reply. All test-set numbers below come from the saved test predictions of the submitted readers and no reader was retrained for test and nothing was tuned on test.

A. New Experiments

  1. Search without attribution feedback (Reviewers i6HP, Y4oS). We re-ran the perception-level search with the same designer, playbook, candidate space, training subset, development patients, seeds, baselines and budget, removing the attribution terms from the designer's prompt and feedback. The designer proposed the same six configurations in the same order as in Table A8, and the same configuration was accepted. The accepted perception revision is therefore robust to what the designer is shown: it is reached from the fixed objective and size-resolved recall alone. This sharpens what attribution contributes (item 6). A random-search arm at the same budget is training and will be posted in this thread.
  2. Per-seed results with patient-bootstrap intervals (Y4oS). Detection is consistent across seeds: at the development-chosen threshold, recall is 0.79 and 0.82 on liver and 0.83 and 0.81 on kidney, against 0.52 and 0.70 for the hand-chosen configuration. The seeds differ in false-positive rate (0.35 vs 0.87 per patient on liver), so under a hard ≤0.5 FP budget seed 1 operates one step tighter (0.51 on liver, 0.75 on kidney). Three further seeds are training and will be posted in this thread. Both seeds exceed the hand-chosen configuration on J on both organs.
  3. Development-transferred thresholds (i6HP). The headline operating point of the designer's reader uses the same threshold on development and test. For the liver baselines, Table 1's rule shows them one step tighter than a transferred threshold would; we give both. The designer's advantage holds either way (paired macro-J gain +0.21 [+0.14, +0.28], interval excludes zero).
  4. Acquisition-protocol subgroups (Y4oS). Recall is higher than or equal to the hand-chosen configuration's in every slice-spacing group on both organs.
  5. Inference level under a stricter acceptance rule (i6HP). Answering the reviewer's threshold question, we tightened the acceptance test for inference programs, which had re-picked operating thresholds inside each bootstrap resample. The new rule chooses thresholds on other development patients. Under it we re-scored all 60 logged proposals (none beats the reader alone), re-ran the search (one program accepted per organ on development), and scored those once on test with frozen thresholds: neither improves on the reader alone. The stricter gate therefore does what the paper asks of a gate, and the single reader stands as the system result.

B. Revisions to the Paper

  1. Claims about attribution. As the paper states, Eq. (5) was formulated retrospectively and the order of levels was set by us; experiment 1 adds that the accepted perception revision does not depend on feeding the attribution terms to the designer. The claim we make is therefore precise: attribution measures which failure term each revision moved (Table A6, Figure 3b), and perception revisions are the ones that reduce the never-detected term. We state the second contribution, the abstract and Section 6 in exactly these terms, and add the table of decision points (reply to i6HP).
  2. Inference level. The system result is the reader alone; we remove the inference-level test row (Table 2 bottom rows, "0.82 / 0.83") and the corresponding sentences in the introduction and Section 5.3. The perception-level results use a different test (retrained readers against the seed-noise floor, integer thresholds) and are unaffected.
  3. Table 1 caption. The thirteen frontier-model rows are raw operating points (these models emit no confidence score); "≤0.5 FP per patient" applies to our rows only. We add an FP column (e.g. Gemini 3.1 Pro 1.13, Mistral Large 3 5.19 FP/patient on liver). Table A3 remains the matched-protocol comparison.
  4. Table A9. The J = 0.529 row is the hand-designed recipe of Table A10 (single seed), not a designer recipe. No designer recipe was accepted at the recipe level.
  5. Both seeds in the abstract and Table 1, with intervals; a third organ (pancreas) is added as the bounded case of the third contribution: attribution localizes the loss to perception, and the current candidate space does not yet remove it (reply to Y4oS).
  6. Presentation (VKC1). A definitions box (replay world, attribution, revision), a rewritten abstract, and a step-by-step walkthrough of Figure 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.