- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a new iterative training method to optimize multimodal LLMs for medical reasoning. The core loop involves a student generating traces to medical problems, after which the traces are reviewed and tagged by the teacher to identify specific points in the traces which diverged from correct reasoning. The student then regenerates these tagged traces, keeping the correct prefix of the trace, and only regenerating the incorrect suffix. This results in higher quality student-generated traces for future supervision, which is combined with further evaluation on a feedback set to direct the model to identify the next best questions for future iterations of the loop. When applying these steps to finetune Qwen3.5-9B and other larger variants, the model significantly surpasses the base model's performance and outperforms other similarly sized competitor models.
Critical strengths
The method makes intuitive sense, and the authors present strong results to back it up. The recipe also generalizes across model sizes, which encourages follow ups with larger model size or increased rounds of iterations. I also found the paper well written and easy to follow the methodology.
Critical weaknesses
The paper mentions several baselines in the abstract, such as supervised finetuning, reinforcement learning, and on-policy distillation, but does not compare the performance of training with these methods to the results of the new proposed method. A comparison to these approaches would solidfy the contribution and show why the proposed method is better than using an existing approach. More concretely, these are some alternative approaches which I believe would be strong baselines to compare with:
- Supervised finetuning on teacher-generated traces (obtained with rejection sampling using the final target answer as the correctness signal)
- Reinforcement learning, with format reward and final answer correctness reward
- On-policy distillation using a strong open-source teacher model
Another aspect of the method which raises some concern with me is the assumption that the teacher can identify incorrect reasoning in the student trace. This is a strong assumption, and even if it is true, suggests that using the teacher alone for direct distillation or creating teacher-generated traces would be a potentially stronger baseline.
Table 1 is a bit misleading. For example, in the <10B open source model section of the table, the entire MedEvo row is bolded, when LingShu actually achieves higher performance on SLAKE and PathVQA.
Questions for authors
Have you tried any ablations where you compare different existing approaches such as SFT, RL, OPD, on the same dataset you use in your new approach?
Do you assume the teacher is strong enough to identify incorrect medical reasoning in the student trace? Are there any instances where you observe inaccuracies in the correct reasoning cutoff point identified by the teacher model?
Have you tried your approach using the same base model used to train the other models you compared with (Qwen2.5-VL-Instruct for example)