acceptodds
Under review as a conference paper at ICLR 2027

3DMedEvo: Self-Evolving VLM Agents with Evidence-Grounded Multimodal Skills for 3D Medical Image Analysis

Abstract

3D CT analysis integrates volumetric perception with clinical reasoning, and improves as experience accumulates over successive readings. Existing 3D medical methods typically use task-specific models or task-agnostic end-to-end predictors and rely on external teacher supervision. Memory-augmented approaches instead reuse experience across cases but often retain raw, noisy, and poorly curated interaction traces as text that does not fully preserve spatial detail. More importantly, they lack adoption-aware estimates of memory utility for future reasoning. Here, we present **3DMedEvo**, a **self-evolving runtime framework** that distills informative trajectories from frozen vision–language model (VLM) agents reading CT scans into structured multimodal skills through outcome-guided reflection, without additional teacher supervision. The resulting skills combine reusable procedures with visual exemplars in a repository organized into multiple branches, with guidance grounded in clinical evidence. 3DMedEvo further calibrates skill reliability from observed utility using an adoption-aware **Skill Credit Score (SCS)**, which guides retrieval for each task and uncertainty-aware governance. In turn, retrieved skills support reasoning by frozen VLM agents within a closed loop of skill acquisition, evaluation, and revision. Experiments on two established clinical benchmarks covering thoracic and abdominal CT show that 3DMedEvo outperforms reasoning agents using tools and representative agents using memory across recognition, quantitative measurement, visual reasoning, and medical reasoning tasks. The framework also generalizes across VLM backbones, offering a practical approach to self-evolution in 3D medical image analysis. All code and data will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, 3D Medical Image Analysis, Self-Evolving Memory

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zfEhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

3DMedEvo improves frozen models’ performance on CT question answering through an evolving multimodal skill repository, combining trajectory-to-skill distillation, clinical evidence checking, adoption-aware credit, and posterior-interval governance. The paper reports notable benchmark improvements, but questions remain regarding the visual input pipeline, novelty relative to closely related methods, and evidence for generalization through continual evolution.

Critical strengths

1.Substantial empirical improvements. Under frozen memory, the method improves overall accuracy across two primary backbones and two benchmarks, outperforming the strongest same-backbone memory baseline by approximately 4.3–6.9 percentage points.

2.Extensive supplementary analyses. Ablations, cross-backbone evaluation, repository transfer, construction-order experiments, and physician verification provide complementary evidence supporting the framework.

3.Transparent failure analysis. The case studies disclose concrete limitations, including tool incompatibility, infeasible recovery actions, and insufficient interaction budgets.

Critical weaknesses

W1. The visual input pipeline for GLM-5.2 is unclear

The official GLM-5.2 documentation lists text as its input modality, whereas Appendices A and G describe supplying current-case images and tool-returned images to the same frozen backbone. The paper does not specify the actual model endpoint, visual frontend, or image-to-text mechanism. A text model can process structured measurements, but this does not explain how chest images and historical visual exemplars inform its decisions. This gap affects the interpretation of the multimodal mechanism, visual ablations, and baseline comparisons, and requires clarification.

W2. Methodological advances over closely related frameworks are not sufficiently established

SkeMex already includes trajectory distillation, skill creation and revision, branch-aware retrieval, adoption attribution, relative-advantage estimation, and repository governance. The main additions in 3DMedEvo are visual exemplars, clinical evidence checking, and posterior-interval governance. The ablations support the usefulness of these components, but do not sufficiently separate the contributions of additional information, additional processing, and new algorithmic mechanisms. Comparisons against adapted baselines with the same visual information, evidence access, and comparable budgets would better establish the methodological advance.

W3. Evidence for continual improvement on unseen patients is limited

The evolving-memory protocol revisits the same evaluation questions, retaining skills and correctness feedback derived from reference answers across rounds. The R1→R3 gains in Table 2 are 0.92–1.72 percentage points, primarily supporting adaptation to repeated tasks rather than establishing continual improvement on unseen patients. The partial-freezing comparison in Table 3 provides some support, but an independent held-out evaluation and uncertainty estimates across multiple runs are still missing. This concern does not negate the frozen-memory results.

Questions for authors

1.What exact model ID and visual processing pipeline were used for GLM-5.2? If an additional vision model was involved, was it equally available to all baselines?

2.Can the authors compare against an adapted SkeMex under matched visual information, evidence access, and comparable budgets?

3.Starting from the same end-of-R1 snapshot, how do freezing and continued evolution compare on a patient-disjoint held-out set that never contributes to updates? Are the gains stable across runs?

4.How frequently are tool-incompatible skills retrieved? Can the authors release the evidence-retrieval configuration, evidence snapshots, and task-baseline estimation procedure?

7 comments

We greatly appreciate your detailed and constructive review of our work. We are grateful for your recognition of the "substantial empirical improvements" of our method and the "extensive supplementary analyses" that support the framework. We are also encouraged by your positive assessment of our "transparent failure analysis". In addition, we are very thankful for your insightful questions and suggestions; we take every comment seriously and provide point-by-point responses below.

W1 / Q1: The visual input pipeline for GLM-5.2 is unclear — The official GLM-5.2 documentation lists text as its input modality, whereas Appendices A and G describe supplying current-case images and tool-returned images to the same frozen backbone. The paper does not specify the actual model endpoint, visual frontend, or image-to-text mechanism. A text model can process structured measurements, but this does not explain how chest images and historical visual exemplars inform its decisions. This gap affects the interpretation of the multimodal mechanism, visual ablations, and baseline comparisons, and requires clarification. What exact model ID and visual processing pipeline were used for GLM-5.2? If an additional vision model was involved, was it equally available to all baselines?

Thank you for raising this important point. We would like to clarify the visual input pipeline of GLM-5.2, which follows its official configuration.

The backbone is the official GLM-5.2 model. Since GLM-5.2 accepts text only, Z.ai provides an official Vision MCP Server at https://docs.z.ai/devpack/mcp/vision-mcp-server that gives it image understanding through a GLM vision model, and we follow this official configuration with its default vision model GLM-5.3-Flash. All images take this route, including the views of the current case, the images returned by CT tools, and the visual exemplars of retrieved skills. The vision model reads each image and returns a textual description to GLM-5.2, which can then compare the current case with historical exemplars. GLM-5.2 remains the only model that plans tool calls, uses skills, answers the question, and performs every LLM-based stage of skill evolution. Both models stay frozen.

This vision model belongs to the backbone configuration rather than to 3DMedEvo. All GLM-5.2 methods in the paper, including ReAct and every memory baseline, use the same vision model with the same settings, so the baseline comparisons and visual ablations are all made under an identical visual frontend. In addition, 3DMedEvo achieves consistent gains with Qwen3.6-Plus, which reads images natively, indicating that our conclusions do not depend on this visual frontend. We will add this clarification to the implementation details in the main text.

W2 / Q2: Methodological advances over closely related frameworks are not sufficiently established — SkeMex already includes trajectory distillation, skill creation and revision, branch-aware retrieval, adoption attribution, relative-advantage estimation, and repository governance. The main additions in 3DMedEvo are visual exemplars, clinical evidence checking, and posterior-interval governance. The ablations support the usefulness of these components, but do not sufficiently separate the contributions of additional information, additional processing, and new algorithmic mechanisms. Comparisons against adapted baselines with the same visual information, evidence access, and comparable budgets would better establish the methodological advance. Can the authors compare against an adapted SkeMex under matched visual information, evidence access, and comparable budgets?

Thank you for your valuable feedback regarding the methodological advance over SkeMex. Following your advice, we compared 3DMedEvo with an adapted SkeMex under matched visual information and evidence access, which separates the contribution of the new mechanisms from that of the additional information.

The adapted SkeMex keeps its own utility estimation, skill revision, and repository governance, and receives the two kinds of information that 3DMedEvo adds: historical CT exemplars are attached to its retrieved skills, and the same clinical references are supplied during its skill learning. All three methods use Qwen3.6-Plus with the same acquisition cases, current-case images, CT tools, and tool budget per question, and are evaluated with frozen memory on all 1,740 DeepTumorVQA questions, as in Table 1.

Regarding budgets, we match the budget available to each method rather than the cost it finally consumes. The consumed cost is determined by each framework's own learning pipeline: SkeMex and 3DMedEvo distill, review, and govern skills through different stages, so forcing identical consumption would require truncating one pipeline and would no longer compare the two methods as designed. The resulting costs are nevertheless of a similar scale. In Table 7, building the library with Qwen3.6-Plus takes 69.40M tokens for 3DMedEvo and 59.80M for SkeMex, and the adapted SkeMex additionally processes the added exemplars and references.

MethodVisual exemplarsClinical referencesAccuracy (%)
SkeMex✗✗49.71
Adapted SkeMex✓✓51.49
3DMedEvo✓✓54.71

The matched information improves SkeMex by 1.78 points, while 3DMedEvo remains 3.22 points higher with the same information. Most of the 5.00-point advantage of 3DMedEvo over SkeMex therefore comes from how the information is used, namely evidence-checked skill synthesis, adoption-aware credit, and posterior-interval governance, rather than from the additional inputs. The ablations in Figure 3 are consistent with this: with the information unchanged, crediting all injected skills instead of only the adopted ones lowers accuracy by 3.7 points, and replacing credit intervals with direct SCS comparisons also lowers accuracy.

W3 / Q3: Evidence for continual improvement on unseen patients is limited — The evolving-memory protocol revisits the same evaluation questions, retaining skills and correctness feedback derived from reference answers across rounds. The R1→R3 gains in Table 2 are 0.92–1.72 percentage points, primarily supporting adaptation to repeated tasks rather than establishing continual improvement on unseen patients. The partial-freezing comparison in Table 3 provides some support, but an independent held-out evaluation and uncertainty estimates across multiple runs are still missing. This concern does not negate the frozen-memory results. Starting from the same end-of-R1 snapshot, how do freezing and continued evolution compare on a patient-disjoint held-out set that never contributes to updates? Are the gains stable across runs?

Thank you for this constructive suggestion. We ran the requested comparison of freezing and continued evolution from the same end-of-R1 snapshot on a patient-disjoint held-out set that never contributes to updates.

Skill acquisition follows the DeepChestVQA evolving-memory setting in the paper. Freezing keeps the library saved at the end of R1, while continued evolution proceeds from this same snapshot for one and two more rounds, giving the R2 and R3 libraries. We repeat this comparison three times, where every run continues evolution from the same end-of-R1 snapshot, and report the mean and range. The held-out set contains 900 questions sampled from the Existence Detection task of the 3D-RAD dataset at https://huggingface.co/datasets/Tang-xiaoxiao/3D-RAD, each paired with the full 3D CT volume of a distinct CT-RATE patient who does not appear in the acquisition set. During held-out evaluation, each library and its credit statistics are frozen, so these cases never contribute to updates. All conditions use Qwen3.6-Plus with the same CT tools, prompts, and tool budget as in the paper, and the no-skill baseline uses the same tools with an empty library.

MemoryMean accuracy (%)Range (%)
No skill, same tools54.78–
Frozen at the end of R157.1156.78–57.67
Continued evolution to R260.7460.56–60.89
Continued evolution to R362.4161.89–62.78

Continued evolution exceeds freezing by 3.63 points after one more round and by 5.30 points after two, and the R3 library is 7.63 points above the no-skill baseline. Since the held-out patients never contribute feedback or skills, this gain reflects reusable skill improvement rather than adaptation to repeated questions. The gains are also stable across runs: the accuracy of each library varies by at most 0.89 points, well below the gain of continued evolution, and the R3 library exceeds the frozen R1 library in every run. Consistently, Appendix B.5 rebuilds the library under four different case orders, and every order improves over ReAct by 7.07–9.02 points.

Reviewer_PbAGOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes 3DMedEvo, a self-evolving memory framework for 3D medical image analysis. It develops and iterates CT analysis trajectories into reusable multimodal skills. The framework further introduces adoption-aware skill credit and uncertainty-aware governance to guide skill retrieval and maintenance. Experiments demonstrate improvements over existing memory-based agents, with additional analyses of skill evolution and cross-model transferability.

Critical strengths
  1. The proposed multimodal skill design is suitable for 3D medical image analysis. Organizing skills into observation, interpretation, and decision-making branches, while combining procedural guidance with visual exemplars, provides a meaningful improvement over purely text-based memory representations.

  2. The adoption-aware skill credit and lifecycle management mechanisms are interesting. By considering whether retrieved skills are actually used and incorporating uncertainty into skill maintenance, the framework provides a more systematic approach to evaluating and updating reusable experience.

  3. The comparison experiments, the ablations and detailed evaluation and analysis is comprehensive. The supplementary material also provides useful implementation details and expert validation.

Critical weaknesses
  1. Although the frozen-memory experiments demonstrate the generation ability of 3DMedEvo, the evolving-memory evaluation repeatedly processes the same question set across rounds. It would be valuable to examine whether the framework can sustain improvements on continuously arriving unseen cases.

  2. It would be interesting to discuss whether dynamically retrieving based on intermediate observations could improve adaptability since 3D medical image analysis involves iterative exploration. In current pipeline, skill selection remains fixed throughout each case, but the agent may acquire new visual evidence and revise its diagnostic hypotheses during execution.

  3. The proposed framework consistently improves over the evaluated baselines, but the performance gain on several basic visual tasks (measurement, recognition) remains somehow moderate. It would be interesting to investigate whether stronger upstream visual tools, (as explored in previous works such as 3DMedAgent) could further improve the performance of 3DMedEvo.

  4. The relationship with existing self-evolving memory and skill frameworks could be discussed more clearly. In particular, the authors could further clarify the methodological distinctions between 3DMedEvo and recent approaches such as SkeMex regarding skill utility estimation, experience-based refinement, and memory lifecycle management.

  5. Reporting random-guessing accuracy for different task categories would help readers better interpret task difficulty and the significance of the reported improvements for the MCQ evaluation.

Questions for authors

See weakness.

5 comments

Thank you for your thoughtful review of our work. We are grateful for your recognition that our multimodal skill design is "suitable for 3D medical image analysis", that the adoption-aware skill credit and lifecycle management are "interesting", and that our experiments and analysis are "comprehensive". In addition, we are thankful for your insightful comments and suggestions; we take every comment seriously and provide point-by-point responses below.

W1: Although the frozen-memory experiments demonstrate the generation ability of 3DMedEvo, the evolving-memory evaluation repeatedly processes the same question set across rounds. It would be valuable to examine whether the framework can sustain improvements on continuously arriving unseen cases.

Thank you for raising this point. In the paper, each R1 question is answered before any feedback on it enters the bank, and Table 3 shows that freezing skills after the first 25% of evaluation questions lowers accuracy by 3.43 points compared with continued evolution. To test unseen cases directly, we further evaluated the skill library after each round on cases that never contribute feedback or skills.

We follow the DeepChestVQA evolving-memory setting in the paper and save the skill library at the end of the first, second, and third rounds, denoted R1, R2, and R3. The unseen cases are 900 questions sampled from the Existence Detection task of the 3D-RAD dataset at https://huggingface.co/datasets/Tang-xiaoxiao/3D-RAD, each paired with the full 3D CT volume of a distinct CT-RATE patient who does not appear in the acquisition set. Each library and its credit statistics are frozen during this evaluation, so nothing from these cases returns to skill learning or governance. All conditions use Qwen3.6-Plus with the same CT tools, prompts, and tool budget as in the paper, and the no-skill baseline uses the same tools with an empty library. The results of the three libraries are averaged over three runs.

ConditionAccuracy (%)
No skill, same tools54.78
R157.11
R260.74
R362.41

Accuracy on the unseen cases rises from 57.11% after the first round to 60.74% and 62.41% after the second and third. The R3 library is 5.30 points more accurate than the R1 library and 7.63 points more accurate than the no-skill baseline. As these cases are never used for feedback or skill learning, the results indicate that the framework sustains its improvement on unseen cases as evolution continues, rather than only on the revisited question set. We will include this evaluation in the revision.

W2: It would be interesting to discuss whether dynamically retrieving based on intermediate observations could improve adaptability since 3D medical image analysis involves iterative exploration. In current pipeline, skill selection remains fixed throughout each case, but the agent may acquire new visual evidence and revise its diagnostic hypotheses during execution.

Thank you for this insightful suggestion. In the current pipeline, the controller already adapts its reasoning and tool calls to new observations, while skills are selected once per case. This keeps the procedural guidance consistent within a case and ties the adoption feedback of each case to a single known selection during memory construction and evolution.

Following your suggestion, we examined whether retrieval conditioned on intermediate observations brings further gains. On all 1,740 DeepTumorVQA questions, we used Qwen3.6-Plus with the frozen library from Table 1 and compared fixed retrieval with a variant that re-selects skills once after the first successful measurement, based on the original question and the tool observations collected so far. Both policies share the same library, image inputs, retrieval scoring, skill budget, and tool budget.

Retrieval policyCorrect / totalAccuracy (%)
Fixed retrieval952 / 1,74054.71
One observation-conditioned refresh973 / 1,74055.92

The observation-conditioned refresh improves accuracy by 1.21 points over fixed retrieval. This suggests that refreshing skills with newly acquired visual evidence complements the controller's own adaptation, and that 3DMedEvo is compatible with dynamic retrieval.

W3: The proposed framework consistently improves over the evaluated baselines, but the performance gain on several basic visual tasks (measurement, recognition) remains somehow moderate. It would be interesting to investigate whether stronger upstream visual tools, (as explored in previous works such as 3DMedAgent) could further improve the performance of 3DMedEvo.

Thank you for this valuable suggestion. We examined whether a stronger visual backend further improves 3DMedEvo, as recognition and measurement are limited by the quality of the upstream visual tools. The original backend uses TotalSegmentator-based organ segmentations and measurements. The upgraded one keeps these and adds VISTA3D for liver and pancreatic lesion segmentation from the 3D CT volume, prompted only with fixed class names, and lesion volumes are computed from its predicted masks.

To isolate the effect of the backend, we kept the backbone, frozen skill library, retrieved skills, prompts, and tool-call budget fixed, and applied the same upgrade to memory-free ReAct for reference. The results on all 1,740 DeepTumorVQA questions are summarized in the table below.

BackboneMethodRecognitionMeasurementVisual ReasoningMedical ReasoningOverall
GLM-5.2ReAct53.4144.6040.4351.6646.03
GLM-5.2ReAct + VISTA3D57.3949.4340.7152.5748.10
GLM-5.23DMedEvo56.2553.4150.0758.9153.68
GLM-5.23DMedEvo + VISTA3D66.7663.0752.0661.0358.97
Qwen3.6-PlusReAct49.1546.5944.2647.4346.32
Qwen3.6-PlusReAct + VISTA3D53.9852.8444.8248.0448.91
Qwen3.6-Plus3DMedEvo60.5151.1449.7962.8454.71
Qwen3.6-Plus3DMedEvo + VISTA3D69.8961.6551.6365.2659.94

The stronger backend improves 3DMedEvo by 5.29 and 5.23 points overall with GLM-5.2 and Qwen3.6-Plus, mainly in recognition and measurement, which rely most directly on lesion segmentation. ReAct gains 2.07 and 2.59 points from the same upgrade, so the margin of 3DMedEvo over ReAct widens to 10.87 and 11.03 points. As the skill library is not updated here, this suggests that stronger visual tools and learned skills are complementary.

Reviewer_bCx5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes 3DMedEvo, a memory-based framework for improving frozen VLM agents on 3D medical image analysis. It distills completed CT-reading trajectories into multimodal skills organized into Observe, Interpret, and Decide branches. Candidate skills are checked against clinical evidence, while adoption feedback and task outcomes update a Skill Credit Score (SCS) used for retrieval and skill governance. Experiments on DeepChestVQA and DeepTumorVQA evaluate both a frozen skill library and online memory evolution. The paper also provides component ablations, cross-backbone transfer results, and physician verification of adoption labels and skill quality.

Critical strengths
  1. The paper addresses a meaningful problem: turning case-specific CT-reading experience into reusable guidance for subsequent cases. Combining scoped procedures, visual exemplars, and clinical evidence is a reasonable design for this setting.
  2. The experimental results support the effectiveness of the method. With frozen memory, improvements over the strongest same-backbone memory baselines range from 4.25 to 6.87 percentage points across the two benchmarks and primary backbones. Transferring the fixed library from abdominal to chest CT tasks provides useful evidence of generalization beyond the acquisition cases.
  3. The ablations are informative and cover memory content, branch organization, skill synthesis, credit assignment, and governance. The physician verification further supports the usefulness of the adoption labels and their relationship to clinical skill quality.
Critical weaknesses
  1. The interpretation of credit after skill revision needs clarification. Each acquisition window executes a frozen skill version, but Algorithm 1 first applies revisions, discounts earlier statistics, and then adds the current window’s feedback at full weight. Appendix C.3 explicitly retains the feedback from the window that prompted the revision without discounting it. These outcomes were still produced using the previous version. Inheriting them may be useful when revisions are local, but it is unclear how the resulting SCS should be interpreted as evidence about the revised skill. In particular, failures that motivated a correction could lower its credit before the corrected procedure has been used.
  2. The evidence pooling used in skill consolidation may affect the interpretation of the posterior intervals. Near-duplicate skills are identified through applicability-description similarity, and their credit statistics are added together. Since multiple skills can be adopted in the same trajectory, it would be helpful to clarify whether shared trajectories are deduplicated during pooling. Otherwise, the interval could become narrower without a corresponding increase in independent evidence. Similar applicability descriptions also do not necessarily imply equivalent procedures or validation conditions. The physician checks support label agreement and the association between SCS and skill quality, while the uncertainty estimates after revision and merging remain less directly validated.
  3. The evolving-memory protocol provides limited evidence of continued generalization to unseen cases. The same evaluation questions are revisited for three rounds, with feedback and memory carried across rounds. This is a useful adaptation setting, but the gains may include benefits from repeated exposure to particular cases as well as reusable skill improvement. The additional gains from R1 to R3 are 0.92–1.72 percentage points, with much of the advantage over the baselines already present in R1. The frozen-memory results provide separate evidence of generalization; an untouched, CT-disjoint evaluation set would help establish whether continued evolution further improves performance on new cases.
Questions for authors
  1. How are adoption statistics associated with skill versions in the implementation? Why is feedback from the window that triggers a revision added at full weight, although that window used the previous version? A comparison with resetting credit or consistently discounting all previous-version evidence would help clarify this choice.
  2. When consolidating skills, are shared trajectories deduplicated? Are procedure content and validation conditions checked in addition to applicability similarity before their statistics are pooled?
  3. Can you evaluate successive memory snapshots on a fixed set of CT volumes that never contribute feedback or skills? This would help distinguish improvement on new cases from adaptation to the repeatedly evaluated cases.
4 comments

We sincerely thank you for your careful and thorough review of our work. We are very grateful for your recognition that our paper addresses "a meaningful problem" and that combining procedures, visual exemplars, and clinical evidence is "a reasonable design". We also greatly appreciate your assessment that our results provide "useful evidence of generalization" and that our ablations are "informative". In addition, we are very thankful for your insightful questions and suggestions; we take every comment seriously and provide point-by-point responses below.

W1 / Q1: The interpretation of credit after skill revision needs clarification. Each acquisition window executes a frozen skill version, but Algorithm 1 first applies revisions, discounts earlier statistics, and then adds the current window's feedback at full weight. Appendix C.3 explicitly retains the feedback from the window that prompted the revision without discounting it. These outcomes were still produced using the previous version. Inheriting them may be useful when revisions are local, but it is unclear how the resulting SCS should be interpreted as evidence about the revised skill. In particular, failures that motivated a correction could lower its credit before the corrected procedure has been used. How are adoption statistics associated with skill versions in the implementation? Why is feedback from the window that triggers a revision added at full weight, although that window used the previous version? A comparison with resetting credit or consistently discounting all previous-version evidence would help clarify this choice.

Thank you for this insightful question. In our implementation, each adoption record stores the skill identity, the executed version, the adoption label, and the credited outcome, and credit accumulates along the revision lineage of a skill rather than per version. The SCS after a revision is therefore a prior inherited from this lineage, which later adoptions of the revised version update.

Let HH be the evidence from earlier windows and CC the feedback from the window that triggered the revision. Our rule and the two alternatives you suggested retain:

E_{\mathrm{uniform}}=\gamma(H+C),\qquad E_{\mathrm{reset}}=0,\qquad \gamma=0.5.$$ The discount is applied once when a revision is released. We keep $C$ at full weight because a revision alone does not show that the failure behind it has been fixed, so the revised skill should regain credit through new adoptions. Since an effective correction may then start with a lower score, we compared the three rules empirically. We replayed the DeepTumorVQA acquisition history of Qwen3.6-Plus, recomputing SCS and relative advantage under each rule while keeping skill contents, retrieval paths, merges, and status decisions fixed. The three rules therefore differ only in the credit of the final library used for test-time retrieval. | Revision-credit policy | Correct / total | Accuracy (%) | |---|---:|---:| | Current inheritance | 952 / 1,740 | **54.71** | | Uniform discounting | 926 / 1,740 | 53.22 | | Reset | 897 / 1,740 | 51.55 | **Current inheritance outperforms uniform discounting by 1.49 points and reset by 3.16 points.** Figure 3c also shows that keeping earlier evidence at full weight with $\gamma=1$ lowers accuracy on DeepChestVQA. Together, these results indicate that discounted retention works better than either discarding or fully retaining earlier evidence. Regarding the concern that a correction may be penalized before it is used, we traced all 428 content revisions in the same history until the next content update, merge, deprecation, or the end of acquisition. Of these, 259 (60.51%) were adopted again after the revision, whereas only 27 (6.31%) were deprecated before any new adoption; the rest had not yet been adopted again when tracing ended. Among the revisions with new adoptions, 257 have a post-revision score that can be recomputed from the log, and for these we compared the post-revision SCS under each rule with the 1,829 outcomes later credited to the revised versions. The mean squared error is 0.1975 for current inheritance, 0.1952 for uniform discounting, and 0.2500 for reset. Inherited credit thus predicts the outcomes of the revised skill better than reset, and the two retention rules are nearly equivalent.

W2 / Q2: The evidence pooling used in skill consolidation may affect the interpretation of the posterior intervals. Near-duplicate skills are identified through applicability-description similarity, and their credit statistics are added together. Since multiple skills can be adopted in the same trajectory, it would be helpful to clarify whether shared trajectories are deduplicated during pooling. Otherwise, the interval could become narrower without a corresponding increase in independent evidence. Similar applicability descriptions also do not necessarily imply equivalent procedures or validation conditions. The physician checks support label agreement and the association between SCS and skill quality, while the uncertainty estimates after revision and merging remain less directly validated. When consolidating skills, are shared trajectories deduplicated? Are procedure content and validation conditions checked in addition to applicability similarity before their statistics are pooled?

Thank you for raising this point. Shared trajectories are not deduplicated during pooling, and near-duplicates are identified by applicability similarity alone. Pooling follows the definition of SCS, which credits each adoption of a skill, so a trajectory that adopted both skills contributes one adoption to each.

To examine whether these choices affect the intervals, we replayed the 263 merges logged during skill acquisition with Qwen3.6-Plus on DeepTumorVQA. Only 47 of them involve shared trajectories, and counting each shared trajectory once widens their intervals by 24.35% on average but changes only 8 of the 263 governance decisions. The merged pairs are also close in content, with a mean similarity of 0.93 for procedures and 0.92 for validation conditions. Hence, shared trajectories have a limited effect on governance, and skills with similar applicability largely share their procedures and validation conditions.

We further evaluated two stricter rules in the same setting. Deduplicated counts each shared trajectory once, and Strict additionally merges a pair only when its applicability, procedure, and validation-condition similarities all reach 0.90. The frozen-memory results on all 1,740 DeepTumorVQA questions are as follows:

ConfigurationAccuracy (%)
Original54.71
Deduplicated54.43
Strict54.20

Accuracy changes by at most 0.51 points, indicating that the reported gains do not rely on repeated credit or on merging skills with dissimilar procedures. Neither stricter rule improves accuracy, while Figure 3c shows that disabling merging lowers accuracy on DeepChestVQA by 1.85 points, which supports the current consolidation design.

W3 / Q3: The evolving-memory protocol provides limited evidence of continued generalization to unseen cases. The same evaluation questions are revisited for three rounds, with feedback and memory carried across rounds. This is a useful adaptation setting, but the gains may include benefits from repeated exposure to particular cases as well as reusable skill improvement. The additional gains from R1 to R3 are 0.92–1.72 percentage points, with much of the advantage over the baselines already present in R1. The frozen-memory results provide separate evidence of generalization; an untouched, CT-disjoint evaluation set would help establish whether continued evolution further improves performance on new cases. Can you evaluate successive memory snapshots on a fixed set of CT volumes that never contribute feedback or skills? This would help distinguish improvement on new cases from adaptation to the repeatedly evaluated cases.

Thank you for this insightful suggestion. To separate improvement on new cases from adaptation to the repeatedly evaluated cases, we evaluated successive memory snapshots on a fixed held-out set of CT volumes that never contribute feedback or skills.

Skill acquisition follows the DeepChestVQA evolving-memory setting in the paper, and R1, R2, and R3 denote the skill libraries saved at the end of the first, second, and third rounds. The held-out set contains 900 questions sampled from the Existence Detection task of the 3D-RAD dataset at https://huggingface.co/datasets/Tang-xiaoxiao/3D-RAD, each paired with the full 3D CT volume of a distinct CT-RATE patient who does not appear in the acquisition set. During held-out evaluation, each library and its credit statistics are frozen, so nothing from these cases returns to skill learning or governance. All conditions use Qwen3.6-Plus with the same CT tools, prompts, and tool budget as in the paper, and the no-skill baseline uses the same tools with an empty library. The results of the three libraries are averaged over three runs.

ConditionAccuracy (%)
No skill, same tools54.78
R157.11
R260.74
R362.41

Accuracy increases with every additional round. R3 exceeds R1 by 5.30 points, which isolates the benefit of continued evolution, and exceeds the no-skill baseline by 7.63 points, which reflects the overall contribution of the accumulated memory. Since the held-out cases never contribute feedback or skills, these gains reflect reusable skill improvement rather than adaptation to repeatedly evaluated cases. We will include this evaluation in the revision. Thank you again for your careful and constructive review. We hope these responses address your concerns, and we are glad to clarify anything further.

Thank you again for your careful review. As the discussion period draws to a close, we would be grateful to hear whether any concerns remain after our responses. We would be happy to provide further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.