- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents Context0, a zero-labeled self-evolutionary framework for context-aware clinical conversations. This method employs a Teacher–Doctor collaborative evolution mechanism: The Teacher uses the biomedical knowledge graph to generate multiple rounds of clinical conversations and dynamic scoring criteria, while the Doctor optimizes learning through the Rubric-based GRPO to learn how to utilize the existing context, identify missing key information, and avoid unnecessary inquiries.
Critical strengths
The method focuses on a practical but often overlooked issue in clinical conversations: not only do models need medical knowledge, they must also judge when to use existing information and when it requires further clarification.Combining context-sensitive task generation, dynamic scoring, and Teacher–Doctor co-evolution provides a valuable technical path to reduce reliance on manual annotation.
Critical weaknesses
-
Context0 mainly integrates existing Teacher Student self evolution, knowledge graph guided synthetic data generation, dynamic Rubric evaluation, and GRPO optimization. Compared with related works such as Agent0, R-Zero, Socratic Zero, and InfiMed ORBIT, the paper has not fully explained what irreplaceable new contributions its key mechanisms bring at the algorithm level. For ICLR, simply combining existing components into clinical dialogue tasks may not be sufficient to demonstrate significant technological innovation of the method.
-
Teacher-Doctor co-evolution is the most important technical claim of this paper. However, the existing ablation studies mainly focus on multi-round context, security filtering, and Rubric densification. The paper does not directly compare the effects of fixed Teacher, adaptive Teacher, static synthetic data training, and the complete bidirectional evolution mechanism. It also lacks performance changes across different evolution rounds. Therefore, it is currently impossible to determine whether the performance improvement comes from the true collaborative evolution, or merely from the larger-scale synthetic data and Rubric optimization. This directly weakens the causal evidence of the core contribution.
-
Although the paper compared Agent0, R-Zero and several general language models, it did not adequately report the synthetic sample size, training computational budget, and optimization settings of each training method for comparability. In particular, there was a lack of standard GRPO, static Rubric RL, and non-adaptive curriculum training baselines that used the same base model and the same synthetic data scale. Given that Context0 achieved a 37.5 percentage point improvement in context-awareness compared to Qwen3-4B-Instruct-2507, it is necessary to reasonably control training resources and data differences.
-
The main results are only from the Global Health subset of HealthBench and are not sufficient to demonstrate that the method generalizes to different diseases, healthcare systems, and real rounds of clinical interactions.More importantly, the paper emphasizes that the model can judge when clarification is needed and when a direct answer should be answered, but it lacks independent quantitative results for behaviors such as the need for clarification, redundant questioning, and incorrect contextual assumptions.
-
The training tasks, scoring criteria, and feedback are highly dependent on automatic generation and model judgment. Although the knowledge graph provides structured medical evidence, it cannot guarantee that the generated clinical scenarios, recommendations, and Rubrics are completely correct. The appendix only describes the disease distribution and scoring criteria composition of 77 generated cases, lacking the factual correctness, Rubric consistency, and proportion of potentially harmful recommendations independently evaluated by clinical experts. Since the model directly optimizes based on these automated signals, systematic errors in the synthetic data may be repeatedly reinforced, affecting the credibility and clinical safety of the training results.
-
The paper lists knowledge graph constraints and zero-annotation training as important features, but Appendix A.5 clearly states that the No-KG experiment only removed the knowledge graph triplet in the training anchor point and failed to confirm that all KG-derived Rubric templates were also disabled.Therefore, this experiment cannot completely isolate the role of the knowledge graph.In addition, Appendix A.2 explains that Zero-Label does not exclude manually designed templates and rules. Although this definition is reasonable in itself, the paper does not quantitatively analyze the impact of these artificial priors on performance.
-
There is a significant misrepresentation in Section 5.1 regarding the ablation results of Figure 4. For example, after removing multiple rounds of context, the clear context score decreased from 61.9% to 53.0%, an actual decrease of 8.9 percentage points, while the main text report decreased by 18.9 percentage points; Removing Rubric densification corresponds to a decrease of 18.9 percentage points. In addition, after removing multiple rounds of context, the unclear context score increased from 50.3% to 50.4%, rather than the decrease described in the main text.
Questions for authors
-
I particularly hope to see a comparison between the fixed Teacher and no Teacher updates, static course training, and complete co-evolution. It would be ideal to match the training data size and computing budget as well. This will directly affect my judgment on the innovation and core contribution of the method.
-
It is hoped to understand the error rate of medical facts in generating scenarios, the consistency between the Rubric judgment and the clinical expert evaluation, as well as the specific performance of the model in providing dangerous suggestions, omitting critical warnings, and unnecessary clarifications. This will influence my assessment of the credibility of the automated training signals.
-
In particular, it is necessary to confirm whether No-KG retains the KG-derived templates, to state the training budget of the main baseline, and to correct the inconsistent values in Section 5.1 and Figure 4.