acceptodds
Under review as a conference paper at ICLR 2027

PACT: Learning Diverse Diagnostic Strategies via Privileged Synthesis and Branch Consensus

Abstract

Artificial intelligence (AI)-based diagnosis has substantial potential to improve the accessibility, efficiency and quality of clinical care. Although existing AI diagnostic methods have demonstrated strong medical reasoning capabilities, most follow a single, fixed reasoning paradigm. In real-world clinical practice, however, physicians may adopt different strategies when questioning patients, gathering clinical evidence and formulating diagnoses. Here we introduce PACT (Periodic Anchor Consensus Training), a model-merging framework that separately learns paradigm-specific LoRA Branches and periodically merges them into a shared Anchor through sign consensus, enabling multiple diagnostic paradigms to be integrated while reducing cross-paradigm interference. Because high-quality clinical dialogues reflecting distinct diagnostic strategies are difficult to collect at scale, we further develop DPS (Doctor Patient Supervisor), which distils dialogues from large language models by simulating four diagnostic strategies; DPS uses complete electronic medical records (EMRs) for quality control while restricting the simulated doctor to patient-visible information, thereby preventing hidden clinical answers from leaking into the dialogues. We collect 10,000 de-identified internal-medicine EMRs to construct a dynamic, multi-turn diagnostic benchmark and show that PACT achieves state-of-the-art performance against strong baselines.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM,Consensus Training,AI diagnostic

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XHAaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PACT (Periodic Anchor Consensus Training), a framework for incorporating multiple diagnostic reasoning paradigms into a single deployable LLM. PACT trains paradigm-specific LoRA branches and periodically merges them into a shared anchor using sign-consensus aggregation and rank projection. The paper also proposes DPS (Doctor-Patient-Supervisor), a data synthesis pipeline that generates multi-turn diagnostic dialogues under four reasoning paradigms while using privileged supervisors for quality control. Experiments on a newly constructed dynamic diagnosis benchmark show improvements over Mixed-SFT and a range of medical and proprietary baselines, with additional evaluations across model backbones, simulators, judges, and clinician ratings.

Critical strengths
  1. The paper addresses a meaningful aspect of interactive diagnosis that is often overlooked in static medical QA benchmarks. The focus on actively acquiring evidence over multiple turns is well motivated, and the idea of learning different inquiry strategies while retaining a single deployable adapter is practically appealing.
  2. The experimental evaluation is relatively comprehensive. In addition to comparisons against Mixed-SFT on the same training data, the paper evaluates multiple backbones, alternative judges and patient simulators, and includes a blinded clinician study. These checks make the empirical results more convincing than a benchmark-only comparison.
Critical weaknesses
  • It is difficult to separate the contribution of PACT itself from that of the DPS-generated training data. On Qwen3-8B, Mixed-SFT already improves Strict DA from 32.53 to 51.48, while PACT further improves it to 55.31. Thus, most of the overall gain appears to come from the synthesized supervision, whereas the incremental gain from PACT is 3.82 points. Moreover, the corresponding improvements in Exam F1 and Soft Score over Mixed-SFT are not statistically significant. This makes the comparisons against zero-shot proprietary and medical models less informative for assessing the contribution of the proposed training algorithm.

  • I am not fully convinced that PACT preserves genuinely distinct reasoning paradigms. The paper reports that the final branch updates have pairwise cosine similarities close to 1.0 and almost no sign conflict. This is useful for merging, but it also raises the possibility that the branches have largely collapsed toward a common solution. Table 18 shows some behavioral differences between branches, but the evidence is still limited. Since the central motivation is to combine complementary diagnostic strategies, I would like to see stronger evidence that such diversity remains meaningful and is actually exploited by the final anchor.

  • The role of privileged supervision in DPS is not fully disentangled from diagnosis-conditioned guidance. The Doctor-Supervisor has access to the full EMR and target diagnosis and may rewrite questions based on diagnostic drift, missing high-risk clues, or missing examinations. The reported 2.1% leakage rate appears to measure explicit label disclosure, but does not rule out more subtle forms of target-conditioned guidance through question selection. Table 6 mainly evaluates the quality of the synthesized data; downstream model performance for these DPS ablations would be more informative.

  • Some of the efficiency results are difficult to interpret. For example, on Qwen3.5-9B, PACT reduces the average number of rounds from 4.24 to 2.06 relative to Mixed-SFT, but symptom coverage also drops substantially from 93.52 to 79.52. A similar pattern appears in the English evaluation. Since the paper motivates PACT as improving evidence acquisition, it would be useful to distinguish efficient questioning from simply terminating the consultation earlier.

  • The aggregation ablation contains a result that I found difficult to reconcile with the branch-alignment analysis. Linear averaging achieves almost the same Strict DA as TIES (55.19 vs. 55.31), but its Exam F1 drops dramatically from 61.33 to 33.74; SLERP shows a similar pattern. Given that the PACT branches are reported to be almost perfectly aligned by the end of training, such a large difference is surprising. A clearer explanation of why the aggregation rule has such a large effect on examination behavior would help clarify the mechanism behind PACT.

Questions for authors
  1. Can the authors provide additional quantitative evidence that the PACT branches remain meaningfully specialized after training, especially given their near-perfect parameter-space alignment? More importantly, is there evidence that the final anchor actually benefits from combining these distinct behaviors?

  2. Could the authors report downstream diagnostic performance for models trained on DPS variants without the Doctor-Supervisor, without the Patient-Supervisor, and without privileged supervision? This would help clarify how much the supervisor design contributes beyond improving synthetic-data quality metrics.

  3. Can the authors explain why Linear Avg and SLERP perform similarly to TIES on Strict DA but dramatically worse on Exam F1, despite the very high branch cosine similarity reported for PACT? Were all other factors, including trimming, rank projection, scaling, and checkpoints, controlled identically?

  4. Were λ\lambda, T∗T^*, and ρ\rho selected exclusively on the validation set? Also, are the reported PACT versus Mixed-SFT results based on a single training run, or can the authors provide variance across independent training seeds?

Reviewer_2wmvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study aims at the limitation that existing medical large language models generally follow a single and fixed reasoning strategy in multiple rounds of consultations, making it difficult to flexibly combine and switch between different diagnostic thinking paradigms. This study proposes a Periodic Anchor Consensus Training Framework (PACT) and a supporting Doctor-Patient-Supervisor (DPS) privileged information dialogue distillation framework.

Critical strengths

The author collected 10000 desensitized internal medicine electronic medical records, selected and synthesized 20684 high-quality multi paradigm training dialogues covering 5171 medical records and 145 diagnostic labels, and constructed a dynamic multi round diagnostic evaluation benchmark based on patient simulator interaction on 438 completely independent reserved medical records.

Critical weaknesses
  1. From the comprehensive analysis of Tables 2, 3, and Appendix Table 13, it can be seen that the substantial leap in model performance is almost entirely due to the DPS synthesized data itself (on Qwen3-8B, a simple mixed fine-tuning Mixed FFT can significantly improve the strict diagnostic accuracy from 32.53% of the baseline to 51.48%, far exceeding the strongest medical vertical baseline Doctor-R1's 33.50%), while the complex PACT algorithm only brings a 3.82 percentage point increase in strict accuracy on this basis.

  2. The core safety principle of multiple rounds of clinical questioning is to make a diagnosis after fully collecting medical history evidence. However, Appendix Table 15a and Table 4b of the main text reveal that PACT has a dangerous tendency of "rush-based" premature convergence. On the Qwen3.5-9B and Ministral-3-8B bases, PACT will sharply reduce the average number of questioning rounds from 4.24 rounds and 3.69 rounds of Mixed-SFT to 2.06 rounds and 2.00 rounds.

  3. The paper lacks the crucial control experiment of randomly and evenly dividing the same dataset into four sub-datasets with the same distribution (i.e., eliminating the paradigm differences among each branch) and running PACT. This weakness has a decisive impact on the overall assessment, as the existing evidence cannot prove that the gains of PACT stem from the "complementary combination of different clinical thinking paradigms".

  4. The paper emphasizes in the introduction that real clinical doctors would flexibly switch between different reasoning strategies during a single consultation based on newly emerging evidence (for example, they would first gradually track symptoms, then concurrently identify hypotheses, or prioritize the exclusion of critical and severe conditions). However, in terms of method implementation, each training dialogue generated by DPS is fixedly bound to a single reasoning paradigm from beginning to end, and does not synthesize the trajectory of dynamic switching of reasoning paradigms during the consultation.

  5. The author attempts to provide theoretical support for PACT in Appendix C, proving that branch drift is controlled (Proposition 1) and that low rank singular value projection errors are bounded. However, the author uses this theory as the basis for the reliability of the method in the main text, but acknowledges in Appendix C.3 and C.4 that Proposition 1 requires the objective function value of the optimization endpoint to not be inferior to the starting point, and must be compared on the exact same empirical loss. However, the adaptive optimizer (AdamW) and random small batch gradient descent used in actual training cannot guarantee the validity of this prerequisite.

  6. The baseline comparison suffers from severe imbalance in the domain-specific vocabulary distribution, and synthesis, simulation, and evaluation are highly coupled to a single model family. In the main experiments of Tables 1 and 2 in the main text, all closed-source cutting-edge large models (such as GPT-4.1, GPT-5, Gemini-3.1 Pro) and open-source medical models were directly evaluated under the zero-shot setting, while PACT was fully fine-tuned on 5,171 electronic medical records from the internal medicine department of the same hospital.

Questions for authors
  1. Why did PACT compress the average number of consultation rounds to 1.27 to 2.06 in Appendix Table 15a (Qwen3.5-9B and Department-3-8B) and Table 4b (English evaluation), resulting in a significant decrease of 13 to 14 percentage points in symptom coverage compared to Mixed FFT or pedestal models? Why is the relaxed diagnostic accuracy (Relaxed DA) of PACT lower than that of Mixed FFT on all four bases?

  2. Why does the F1 score in Appendix Figure 4 experience a cliff like drop when the synchronization strength is set to 100 or 400, or the aggregation period is set to 0.1 or 1/3? Compared to a single model Mixed FFT, how many times does PACT increase the actual training wall clock time and peak video memory by training four branches and materializing the matrix of one norm and singular value decomposition module by module?

  3. If the 957 troublesome medical cases that were excluded in Appendix B.4 are retained for training and testing, or if the privilege rewrite instructions for doctors' supervisors to add missing examinations at the end of the conversation are removed during training, will the strict diagnostic accuracy and F1 score of PACT remain at the current level? On completely different hospitals or public external multi-round medical benchmarks (zero-sample cross-hospital evaluation), will PACT still have a significant advantage over Mixed-SFT and GPT-4.1?

Reviewer_UqdvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PACT, a training framework for multi-turn medical diagnosis that aims to integrate multiple diagnostic inquiry strategies into a single deployable LoRA adapter. It trains one LoRA Branch per strategy, regularizes each Branch toward a shared Anchor, and periodically updates the Anchor via magnitude trimming, sign-consensus merging, and rank projection. The paper also introduces DPS, a role-separated dialogue-synthesis pipeline in which a Doctor only observes patient-visible information while privileged Supervisors with access to complete EMRs validate and minimally rewrite questions and answers. Experiments use 10,000 de-identified internal-medicine EMRs and a simulator-based multi-turn evaluation.

Critical strengths
  1. The paper identifies a concrete and practically relevant tension: mixing heterogeneous diagnostic-dialogue strategies can cause interference, while independently trained adapters are difficult to merge and deploy. PACT addresses this tension with a coherent training-time design rather than relying only on post-hoc merging. The ablations against Mixed-SFT, sequential training, FedAvg, and independent-Branch merges provide useful evidence that the proposed training loop is beneficial under the reported setting.

  2. The experimental package is relatively thorough. The paper evaluates four backbones, reports component ablations and robustness checks with alternative judges/simulators, and includes a small blinded physician evaluation. DPS's explicit partition of patient-visible versus privileged EMR information is also a sensible attempt to make synthetic diagnostic dialogues more realistic than unconstrained self-play.

Critical weaknesses
  1. The methodological novelty is limited relative to close prior ingredients. PACT builds on LoRA specialization [1], task-vector/model merging [2,3], sparse trimming and sign-conflict resolution in TIES-Merging [4], and SVD-based alignment/merging of LoRA updates in KnOTS [5]. Its periodic Branch--Anchor optimization and fusion loop is also related to iterative task-specific optimization and fusion in ColD Fusion [6], as well as shared-reference approaches for heterogeneous local updates in federated learning, including FedAvg [7], FedProx [8], and SCAFFOLD [9]. The specific combination of periodic Anchor regularization and TIES-style consensus merging for diagnostic-dialogue paradigms may be new; however, the paper does not adequately position PACT against these closest methods or isolate what is technically new beyond application-specific integration. In particular, comparison with ColD Fusion and a proximal/periodic FedAvg-style Branch baseline would be important.

  2. DPS is a sensible control mechanism, but its conceptual novelty should be positioned more carefully. Its role-separated use of privileged information is related to Learning Using Privileged Information [10], while diagnosis-aware critic/self-play for clinical dialogue is closely related to AMIE [11]. Thus, DPS appears best characterized as an EMR-grounded instantiation with explicit information-access controls, rather than a wholly new supervision paradigm. Moreover, because the Doctor-Supervisor sees the complete record, including diagnosis and unrevealed tests, the reported 2.1% explicit-label leakage rate does not exclude subtler leakage through question choice or wording. This concern is material because both the training supervision and primary evaluation rely substantially on LLM-generated/simulated interactions.

  3. The central empirical claims are limited by the synthetic evaluation pipeline and comparison protocol. PACT is trained on EMR-derived dialogues, whereas proprietary LLMs are evaluated zero-shot; thus the large comparisons to GPT-4.1/GPT-5 should not be read as an apples-to-apples model-capability comparison. More importantly, the Doctor-Supervisor has access to the full record, including diagnosis and unrevealed tests, when it validates or rewrites questions. Reporting 2.1% explicit label leakage does not rule out subtler diagnostic leakage through question selection or wording. Since both supervision and much of the assessment rely on LLM-generated/simulated interactions, this concern materially affects the strength of the claimed clinical improvement.

[1] LoRA: Low-Rank Adaptation of Large Language Models.

[2] Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy without Increasing Inference Time.

[3] Editing Models with Task Arithmetic.

[4] TIES-Merging: Resolving Interference When Merging Models.

[5] Model Merging with SVD to Tie the Knots.

[6] ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning.

[7] Communication-Efficient Learning of Deep Networks from Decentralized Data.

[8] Federated Optimization in Heterogeneous Networks.

[9] SCAFFOLD: Stochastic Controlled Averaging for Federated Learning.

[10] A New Learning Paradigm: Learning Using Privileged Information.

[11] Towards Conversational Diagnostic Artificial Intelligence.

Questions for authors
  1. Please quantify potential implicit privileged-information leakage from the Doctor-Supervisor, beyond explicit diagnosis-label leakage. For example, can you compare DPS against a version whose question supervisor is restricted to patient-visible information, or measure whether rewritten questions predict the gold diagnosis more strongly than questions generated without privileged access?

  2. Please clarify the fairness of the comparisons with proprietary models. Were all such models evaluated strictly zero-shot, while PACT was trained on EMR-derived data from the same source distribution? If so, please temper the “state-of-the-art” wording and add a comparison to a strong model receiving an analogous in-context or supervised adaptation budget where possible.

  3. How were patient-level or encounter-level overlaps controlled between the EMR records used for DPS synthesis and the 438-case evaluation set? Please provide label prevalence, institution/time split information, and details of de-identification/IRB or equivalent governance.

  4. Please clarify the novelty of the proposed method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.