- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents MedFrame-R1, a framework for improving medical multimodal reasoning through supervised initialization and reinforcement learning. It argues that (1) most of the 51K training corpus yields zero advantage, (2) an LLM judge's binary correctness reward is hackable via "answer inclusion", and can be gated by an embedding-similarity threshold (cos ≥ 0.8), (3) hard questions and rare modalities remain underrepresented, motivating a reasoning-based supervised cold start, and (4) evaluation should be deterministic where possible and otherwise use three independent judges. The resulting 4B model scores 64.8 macro over 17 benchmarks vs 62.9 for the authors' reproduction of MediX-R1-4B and 62.2 for its Qwen3-VL-4B initialization.
Critical strengths
MedFrame-R1 achieves strong performance across standard medical VQA and QA benchmarks, outperforming the evaluated medical MLLM baselines in overall macro score.
The paper addresses practical challenges in medical multimodal RL. Uninformative rollout groups, unreliable judge rewards, and evaluation bias are relevant obstacles to developing reliable medical MLLMs.
The analysis of training behavior is informative. The reward curves and qualitative examples illustrate how increasing judge rewards can coexist with declining embedding agreement and overly broad answers.
Critical weaknesses
Insufficient controlled ablations. MedFrame-R1 combines DAPO-style sampling, AESL, reasoning-based SFT, and embedding-gated rewards, but the paper does not isolate their contributions to downstream performance on the main Qwen3-VL-4B backbone. Table 1 reports only the complete MedFrame-R1 system, while the cold-start comparisons use 8B and 27B backbones and do not assess subsequent RL performance. Consequently, it remains unclear which components account for the reported gains.
Insufficient independent validation of semantic reward gating. Section 4.3 primarily evaluates gating through embedding-based agreement with the reference—the same signal directly optimized during training. Although the qualitative examples illustrate reward hacking, the reported analysis does not establish whether gating improves answer correctness or reduces the frequency of reward hacking. A human-annotated audit of reward decisions or a downstream accuracy comparison with and without gating would provide stronger evidence.
Unclear methodological novelty. The overview in Section 3 describes the method as combining structured reasoning supervision, semantic reward validation, and informative example selection, but does not clearly distinguish newly proposed components from adopted techniques. The paper should articulate its contribution beyond integrating existing methods and explain what new capability or insight this integration provides.
Presentation needs improvement. Several sections contain long paragraphs that combine distinct topics, making the argument difficult to follow. In particular, the introduction mixes challenge descriptions with proposed solutions unevenly; a clearer correspondence between the four challenges and the contributions would improve readability. There are also typos, including “scores.R presents a separate challenge” at line 95 and “generates generates” in the Figure 2 caption.
Questions for authors
How often does a retained mixed group collapse to near-zero advantage after gating, and how often does a discarded group retain reward variance? Would selecting on the gated reward change the result?
We thank Reviewer Erkf for their valuable comments, for finding the training analysis informative, and for the helpful writing suggestions. We would like to address the reviewer's comments as follows.
W1: Controlled ablations on the 4B model: Please refer to the comment regarding the clarification on the contribution of each component. Point 3 of that comment covers the cold start on the same 4B model.
W2: Independent validation of the gate:
W3: Novelty:
We thank the reviewer for this helpful suggestion. We will make this clear in the introduction and Section 3 of the final paper.
W4: Presentation:
We thank the reviewer for pointing this out. In the final paper, the introduction will list the four problems one by one and link each to one contribution. We will also fix the typos pointed out by the reviewer.
Q1: Groups that lose their signal after the gate, and selecting on the gated reward:
We estimated this from the per-rollout training logs of a MedFrame-R1 run, which record the judge decision, the gate, and the reward of every rollout.
We thank the reviewer for this question and will add this analysis to the final paper.