- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces PACT (Periodic Anchor Consensus Training), a framework for incorporating multiple diagnostic reasoning paradigms into a single deployable LLM. PACT trains paradigm-specific LoRA branches and periodically merges them into a shared anchor using sign-consensus aggregation and rank projection. The paper also proposes DPS (Doctor-Patient-Supervisor), a data synthesis pipeline that generates multi-turn diagnostic dialogues under four reasoning paradigms while using privileged supervisors for quality control. Experiments on a newly constructed dynamic diagnosis benchmark show improvements over Mixed-SFT and a range of medical and proprietary baselines, with additional evaluations across model backbones, simulators, judges, and clinician ratings.
Critical strengths
- The paper addresses a meaningful aspect of interactive diagnosis that is often overlooked in static medical QA benchmarks. The focus on actively acquiring evidence over multiple turns is well motivated, and the idea of learning different inquiry strategies while retaining a single deployable adapter is practically appealing.
- The experimental evaluation is relatively comprehensive. In addition to comparisons against Mixed-SFT on the same training data, the paper evaluates multiple backbones, alternative judges and patient simulators, and includes a blinded clinician study. These checks make the empirical results more convincing than a benchmark-only comparison.
Critical weaknesses
-
It is difficult to separate the contribution of PACT itself from that of the DPS-generated training data. On Qwen3-8B, Mixed-SFT already improves Strict DA from 32.53 to 51.48, while PACT further improves it to 55.31. Thus, most of the overall gain appears to come from the synthesized supervision, whereas the incremental gain from PACT is 3.82 points. Moreover, the corresponding improvements in Exam F1 and Soft Score over Mixed-SFT are not statistically significant. This makes the comparisons against zero-shot proprietary and medical models less informative for assessing the contribution of the proposed training algorithm.
-
I am not fully convinced that PACT preserves genuinely distinct reasoning paradigms. The paper reports that the final branch updates have pairwise cosine similarities close to 1.0 and almost no sign conflict. This is useful for merging, but it also raises the possibility that the branches have largely collapsed toward a common solution. Table 18 shows some behavioral differences between branches, but the evidence is still limited. Since the central motivation is to combine complementary diagnostic strategies, I would like to see stronger evidence that such diversity remains meaningful and is actually exploited by the final anchor.
-
The role of privileged supervision in DPS is not fully disentangled from diagnosis-conditioned guidance. The Doctor-Supervisor has access to the full EMR and target diagnosis and may rewrite questions based on diagnostic drift, missing high-risk clues, or missing examinations. The reported 2.1% leakage rate appears to measure explicit label disclosure, but does not rule out more subtle forms of target-conditioned guidance through question selection. Table 6 mainly evaluates the quality of the synthesized data; downstream model performance for these DPS ablations would be more informative.
-
Some of the efficiency results are difficult to interpret. For example, on Qwen3.5-9B, PACT reduces the average number of rounds from 4.24 to 2.06 relative to Mixed-SFT, but symptom coverage also drops substantially from 93.52 to 79.52. A similar pattern appears in the English evaluation. Since the paper motivates PACT as improving evidence acquisition, it would be useful to distinguish efficient questioning from simply terminating the consultation earlier.
-
The aggregation ablation contains a result that I found difficult to reconcile with the branch-alignment analysis. Linear averaging achieves almost the same Strict DA as TIES (55.19 vs. 55.31), but its Exam F1 drops dramatically from 61.33 to 33.74; SLERP shows a similar pattern. Given that the PACT branches are reported to be almost perfectly aligned by the end of training, such a large difference is surprising. A clearer explanation of why the aggregation rule has such a large effect on examination behavior would help clarify the mechanism behind PACT.
Questions for authors
-
Can the authors provide additional quantitative evidence that the PACT branches remain meaningfully specialized after training, especially given their near-perfect parameter-space alignment? More importantly, is there evidence that the final anchor actually benefits from combining these distinct behaviors?
-
Could the authors report downstream diagnostic performance for models trained on DPS variants without the Doctor-Supervisor, without the Patient-Supervisor, and without privileged supervision? This would help clarify how much the supervisor design contributes beyond improving synthetic-data quality metrics.
-
Can the authors explain why Linear Avg and SLERP perform similarly to TIES on Strict DA but dramatically worse on Exam F1, despite the very high branch cosine similarity reported for PACT? Were all other factors, including trimming, rank projection, scaling, and checkpoints, controlled identically?
-
Were , , and selected exclusively on the validation set? Also, are the reported PACT versus Mixed-SFT results based on a single training run, or can the authors provide variance across independent training seeds?