- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CF-Evolve, a Proposer–Verifier–Solver framework that improves a frozen Solver through learned counterfactual guidance and accumulated memory. The Proposer is trained to correct erroneous decisions while preserving correct ones. Experiments cover clinical prediction, diagnosis, business reasoning, and fact verification.
Critical strengths
- The distinction between beneficial corrections and harmful revisions is well motivated and more informative than reporting accuracy alone.
- Learned guidance produces substantial improvements in some settings, particularly DiagnosisArena.
- The appendix transparently reports implementation details, negative transfer results, and limitations of the theoretical guarantees.
Critical weaknesses
1.The clinical curriculum progressively enables patient-specific counterfactual guidance, rather than demonstrating transfer from earlier cases to new ones. The separate held-out memory experiment yields approximately one percentage point of improvement, with confidence intervals including zero. *Thus, evidence for sustained self-evolution is limited. *
-
Appendix D states that the clinical RL experiment omits the correctness veto in Equation (5) and assigns zero weight to the Verifier reward. Consequently, the results do not directly validate the complete proposed optimization mechanism. I concern about the reported implementation differs from the proposed objective.
-
A matched, supervised correction or evidence-review baseline is needed to distinguish counterfactual benefits from additional supervision, model calls, and acceptance policies. Currently, the contribution of counterfactual reasoning is insufficiently isolated.
Questions for authors
- Which experiments implement Equation (5), including the correctness veto and Verifier feedback?
- How does CF-Evolve compare with an ordinary correction Proposer under matched supervision and inference budgets?
- Can the authors demonstrate sequential memory transfer on independent new cases while keeping current-case counterfactual processing fixed?
- Can the main clinical results be reproduced on a test cohort untouched during development?
3 comments
The core architectural idea a learnable feedback model, a frozen execution model, and reward signals tied to whether revisions yield correct outputs has been explored in prior work like RL4F (ACL 2023), Retroformer (ICLR 2024), and CTRL (ICML 2025). Other lines of work such as Reflexion, ReasoningBank, Agent Hospital and GSEM pursue memory-based self-evolution without weight updates. In clinical diagnosis, counterfactual case modification was studied by You et al. (2026).The main incremental advance here is limiting feedback to structured counterfactuals and drawing a clear distinction between correction and retention. The theoretical analysis appears to be a tailored instantiation of known results. I recommend discussing RL4F, Retroformer and CTRL and making the key differences explicit.
Thank you for your suggestion!
Dear Reviewer VmcY,
We appreciate your suggestion to clarify the differences from additional baselines. We carefully reviewed the reference you suggested and added a new paragraph to the Related Work section discussing all of the relevant methods you mentioned. Here we provide further details to clarify the distinctions. Specifically, our novelty lies in three aspects:
-
Feedback design: RL4F and CTRL directly critique a candidate answer, while Retroformer reflects on a failed trajectory. In contrast, our feedback is built on causally grounded counterfactuals: it edits the evidence in the case itself.
-
Supervision: preserving a correct answer is an explicit training objective in our method. We assess whether a counterfactual is useful or harmful, functioning as a safety guard. For example, our instruction-tuning data include preservation examples, and the clinical reward penalizes harmful revisions more heavily than it rewards corrections.
-
Task formulation: CTRL is a code-generation method trained against unit tests, while Retroformer is trained on environment returns from repeated attempts at the same task. Because we use counterfactuals to support decision-making, our method evaluates each counterfactual by its effect on the Solver’s decision for the original labeled case, together with feedback from the Verifier.
We have added new self-evolution experiments in Section 4.3 of the revised draft. Beyond memory evolution, i.e. growing the CF context while keeping the Proposer fixed, we now study evolution of the Proposer itself. Specifically, we progressively tune the Proposer using batches of increasingly high-quality counterfactual instruction-tuning examples. This produces a clearer trend: performance improves across all tasks as training grows, with larger gains in more challenging settings. These results show that memory evolution and Proposer evolution are complementary forms of self-evolution, each providing distinct insights. Our approach therefore extends beyond memory evolution alone.
Circle back to the additional related works, Retroformer is methodologically closest to our approach. RL4F and CTRL update only the feedback model’s weights, while Reflexion, ReasoningBank, Agent Hospital, and GSEM update only memory. In contrast, both our method and Retroformer update model behavior as well as memory. We will prioritize an implementation-based comparison with Retroformer, along with other methods you mentioned, and further clarify the distinctions between these methods in the revision.
Thank you again for your valuable suggestions for helping us further improve our work. Please do not hesitate to let us know if you have additional questions.
Sincerely,
Authors
Thank you for your constructive feedback. To further clarify the contributions and mechanisms of CF-Evolve, we have conducted additional experiments examining the Verifier's role, comparisons with ordinary correction, and the impact of accumulated counterfactual supervision on challenging tasks. We also provide additional clinical evaluations to assess generalization.
1. Reward Ablation and Verifier Contribution
The clinical RL experiment reported in the appendix was a preliminary demonstration. Due to runtime constraints, retrieval-based verification (RAG) had not yet been integrated into that run. We have now supplemented it with evaluations incorporating the correctness veto and the Verifier, using the same 10,471 IHM records and full-clinical inputs. Adding the Verifier on top of the veto improves AUROC and AUPRC by 0.108 and 0.077 percentage points, respectively. The three configurations are summarized below.
Table R3-1. IHM reward ablation. All metrics are percentages.
2. Comparison with Direct Correction
Structured counterfactual guidance enables more effective and selective correction than ordinary diagnostic feedback by identifying evidence boundaries and guiding targeted revisions while preserving correct predictions.
Under matched supervision and inference budgets, CF-Evolve outperformed answer-blind Direct Correction across all six model–dataset comparisons on DiagnosisArena and MMMU Business with Qwen, Llama, and MedGemma. The gains over Direct Correction were particularly substantial on DiagnosisArena (+28.3, +32.6, and +22.5 percentage points, respectively), compared with +0.7, +4.8, and +11.0 points on MMMU Business. The correction–harm analysis further illustrates this advantage: on DiagnosisArena, CF-Evolve achieved 43 corrections with zero harms for Llama, versus zero corrections and two harms for Direct Correction. On MMMU Business, harmful revisions decreased from 24 to 13 for Llama and from 31 to 1 for MedGemma.
3. CF Supervision and Harder Tasks
Our self-evolution experiments aim to improve the Proposer by learning from accumulated counterfactual experience, rather than simply reusing historical cases, which can introduce inapplicable reasoning. We trained the IHM Proposer on progressively accumulated cases ordered from easy to hard and evaluated LOS Proposers trained with increasing amounts of CF supervision. On IHM, the balanced-accuracy gain over no CF increases from 5.95 percentage points on the 24-hour task to 8.76 and 9.33 points on the 72- and 168-hour tasks, respectively. On LOS, 2-class improves accuracy by 2.05 and 6-class improves 4.83 points. These results suggest that additional CF supervision can provide greater benefits in more challenging settings, although the relationship is not uniformly monotonic.
4. Data Usage and Additional Evaluation
The 3,000 CF training examples were constructed from training records, not from test cases. The “test set” comes from the early oracle experiment, where we analyzed reasoning failure modes and evaluated the potential benefit of counterfactual guidance. In subsequent pipeline development and evaluation, those inspected cases were treated as validation data and excluded from the final test set.
To avoid potential data leak problem, we also assess whether the benefits extend beyond the oracle-analysis cases, we additionally evaluated the original Proposer examples on 20,000 additional MIMIC-IV stays and 325 eICU cases. The Proposer and Solver weights remained frozen, with no additional training on these evaluation records.
Table R3-2. Additional MIMIC-IV and external eICU evaluation. All metrics are percentages.
CF improves all four metrics on the additional MIMIC-IV cohort, including gains of 4.27 percentage points in AUROC and 3.21 points in AUPRC. On the external eICU cohort, AUROC and AUPRC improve by 7.47 and 12.18 points, respectively, while F1 decreases. These results support benefits beyond the original oracle-analysis cases and demonstrate transfer of the MIMIC-trained Proposer to an external clinical dataset.