- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RSIMed, a retrieval-grounded self-improvement framework for medical foundation models under limited labeled clinical data. The framework combines a frozen task proposer, evidence retrieval, task validation, and reinforcement learning. The proposer generates medical questions from retrieved references and adapts their coverage and difficulty using feedback from the solver. The solver is trained through supervised fine-tuning followed by reinforcement learning with semantic rewards.
Critical strengths
- This article addresses the problem that clinical annotation data is limited and the generated answers cannot be verified through executable tests.
- Splitting the solver and task proposer is an intuitive and efficient solution.
- RSIMed achieves substantial improvements on rare-disease diagnosis, including gains over supervised initialization and static synthetic-data training.
Critical weaknesses
- The ablation experiment failed to truly reflect the proposed innovation. The main comparison is against reinforcement learning on labeled cases or a fixed synthetic-task pool. Continuous generation changes several factors simultaneously, including the number, diversity, and difficulty of available tasks. In particular, the static control uses 15,000 generated tasks, whereas the continuous run analysis 43,698 generated task instances served during training.
- The validator rejects answers directly contradicted by retrieved evidence but accepts tasks when the evidence is inconclusive. It remains unclear how incorrect reference answers affect reinforcement learning or whether the validator consistently controls their accumulation.
- The results in out-of-domain scenarios undermine the conclusion that the full framework consistently enhances general clinical capabilities. In the HealthBench Professional benchmark, a variant utilizing only RL improved the score from 0.291 to 0.303, whereas SFT+RL achieved a score of only 0.280.
- Figure 2 would be more informative if it could be populated with actual content.
Questions for authors
Please see the weakness.
2 comments
Thanks for the review
Dear Reviewer n57U,
Thanks for spending the time to write the review. As the discussion period ends today, we hope our response addresses your concerns. We are happy to take any follow up clarifying questions you may have.
Thanks for the review and response to questions
We thank the reviewer for the review. We address each point below.
We ran the requested experiments. We see that with the number of fresh tasks matched, feedback still makes the solver learn faster. The results below are done with the RSIMed-9B recipe with no feedback and matched sample sizes. We use the same Qwen3.5-9B proposer/judge at training time, and GPT-5.5-Chat judge for validation scoring.
Note the accuracy here is the mean from step 200 to 440, which is far from the saturation performance. We see that even at matched sample size, RSIMed-9B is still the best model with highest learning signal. We further see that the feedback allows the proposer to propose a higher proportion of useful tasks, at 19.8% vs 11.0% for no feedback variant.
We would like to further note that the static pool variant fell from its 0.441 peak to 0.359 by step 155, after only 9,920 draws from its 15,000 tasks. Therefore, repeated training is not a cause for its collapse, and even if it has 43k tasks, it will still collapse well before all tasks are consumed.
We see that wrong keys stay rare and carry little learning signal. We audited a random sample of 100 accepted short-answer tasks from each fifth of training in RSIMed-27B and RSIMed-9B with GPT-5.5-Chat as automatic judge. We see that:
They do not accumulate. The valid share rises from the first fifth of training to the last in both runs: from 78.2% to 86.5% in RSIMed-27B and from 72.0% to 80.0% in RSIMed-9B.
Invalid tasks usually do not get any learning signal. 73% of invalid tasks get no rollout that matches the key, against 54% of valid tasks, so they give no correctness signal.
We further replayed a strict version of the validator, which accepts a task only when the retrieved evidence supports its key, with a frozen Qwen3.5-9B checker. The strict version raises the valid share from 81% to 88% but rejects 19% of the valid tasks. As a comparison, the paper's validator keeps 98% of the valid tasks and rejects 11% of the invalid tasks. In other words, requiring support removes 2.3 units of valid signal for every unit of invalid signal.
We would like to point out that the main focus of the work, and all improvements on the self improving loop, is on the RL part of the pipeline. Therefore, RL only result being better actually indicates that the self improving pipeline gives a more generalizable OOD performance that normal SFT on generated data does not provide, which strengthens our claim. If we break down the results by category, we see that RL only is at or above the untrained model in every category with zero targeted training data for HealthBench Pro:
With that said, we will further adjust the claim to make this more clear.
We will fill Figure 2 with the real task of Figure 3, from seed case to graded solver responses.
We will further make the minor fixes mentioned in the review, and consolidate the new results above into the main text.