acceptodds
Under review as a conference paper at ICLR 2027

Controlled Knowledge Integration and Graph-Grounded Policy Optimization for Rare-Disease Diagnosis

Abstract

Rare-disease diagnosis is inherently a long-tail reasoning (“Odyssey") problem: individual disorders are sparsely documented, clinical phenotypes are heterogeneous and incomplete, and relevant evidence is fragmented across disease ontologies, phenotype databases, gene annotations, and biomedical literature. As a result, language models often over-prioritize common diseases, overlook rare candidates, or produce plausible but invalid diagnoses. To address these challenges, we introduce RareDx, a unified large-language-model (LLM)-based agent for evaluating and improving rare-disease diagnostic reasoning by organizing heterogeneous medical knowledge into structured, verifiable evidence and distilling this knowledge into compact language models. RareDX has its own harness, where RareDx-Harness normalizes heterogeneous patient records into ranked differential-diagnosis tasks and systematically compares parametric inference, static retrieval augmentation, adaptive tool use, and structured phenotype–gene–disease reasoning over a shared medical knowledge infrastructure. Our analysis reveals that retrieval is not universally beneficial: static retrieval can introduce distracting or weakly relevant evidence, whereas adaptive retrieval and ontology-guided reasoning more consistently improve diagnostic ranking. Building on these findings, we develop a two-stage training pipeline combining Top-10 supervised fine-tuning with knowledge-graph–embedded reinforcement learning. The reward function projects model predictions into a structured medical knowledge space and jointly incorporates graded diagnostic relevance, disease-ontology proximity, biomedical semantic similarity, and disease–phenotype graph consistency, thereby transforming sparse exact-match feedback into dense and clinically meaningful supervision. Vocabulary constraints and contamination-aware penalties further suppress fabricated diagnoses and reward hacking. Under our benchmark study, our post-trained LLMs RareDx-9B and RareDx-28B surpass GPT-5.5 and other frontier models, demonstrating that structured knowledge integration and knowledge-aware post-training can enable compact models to achieve strong performance in long-tail rare-disease diagnostic reasoning.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
rare disease diagnosis, LLM, agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2YzaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RareDx, a framework for rare disease diagnosis, which includes RareDx-Harness and RareDx-KGPO. RareDx presents a method to normalize heterogeneous records into one ranked diagnosis task while RareDx-KGPO is a reinforcement learning method that projects predictions into a canonical disease graph with ontology information. The authors evaluate RareDx across 8 benchmarks, and RareDx achieves nearly the best or second best resutls against several baselines.

Critical strengths
  1. The evaluates RareDx across 8 rare disease benchmark datasets with heterogeneous patient representations. This broad evaluation is a useful and strong evidence for the proposed RareDx method.

  2. The integration of ontology information into the RareDx-KGPO is motivated for rare disease diagnosis since many related diseases can share the overlapping phenotypes. Although the individual components in KGPO are not exactly new, their combination provides a reasonable and domain valid approch for integrating graph information into reinforcement learning.

Critical weaknesses
  1. The methodology novelty of RareDx-Harness is not clear. Most of the components in RareDx-Harness (Figure. 1) are exist. It is not clear what new methodology contributions (i.e. decision-making) the RareDx-Harness itself provides. More importantly, the definition of 'Harness' seems to be unclear between the RareDx-Harness methodology and the result section about the model description. In Figure. 1, the 'Harness' presents a framework consisting of Direct, Static RAG, Adaptive ReAct, and Structured 3-hop. However, in the result section (Table. 1), the models named RareDx-XB, Harness seems to only choose between Direct and a 35B Diagnostic Audit. Therefore, it is hard to determine what the system is actually responsible for, and their relationship is not clear. It is also hard to determine how the components Static RAG, Adaptive ReAct, and Structured 3-hop contribute to the full system.

  2. The experimental evaluation does not sufficiently isolate the contributions of the proposed methods. Since the 35B audit model might be used by the router, the performance gains cannot be attributed solely to the 9B or 27B model. Furthermore, the reward components are not ablated for the downstream diagnostic performance, making their contribution not clear in this case. These limitations weaken the evidence supporting the paper's claims, and it is difficult to assess whether the components are responsible for the observed improvements.

Questions for authors
  1. How do strategies in Figure. 1 contribute to the final system? Can the authors further clarify what methodology contribution the proposed Harness method provides beyond selecting between Direct and Audit? Additionally, could the authors apply the same 35B Diagnostic Audit / Harness to other baseline models to see if the gains are consistent?
  2. Since the operational router selects between Direct and Audit using development data, how would the system determine which way to do when encoutering a new cohort without label development data? Considering this is an application paper, could the authors provide evidence of generatlization to unseen settings?
  3. How are evaluation inputs preprocessed to ensure that the ground truth diagnosis related information is excluded in the dataset, particularly in the free-text records and cases containing the genetic findings? Could the authors also report the unique diseases, diseases frequency, and accepted diagnoses per case for each dataset? This will help to improve the transparency of the paper.
  4. Given the relatively low performance of RareDx, especially Hit@1, what is the intended real world clinical usage of RareDx? How could it help the clinicians to do the diagnosis?
  5. Could the authors provide ablations on the reward components in KGPO?
  6. Could the authors explain the inconsistency about the title and abstract of the submitted pdf file and those displayed on OpenReview? For example, the OpenReview abstract refers to RareDx-28B, whereas the PDF reports RareDx-27B. The descriptions of the methodology and experimental results also differ in some details.
5 comments

We thank the reviewer for the detailed questions and for recognizing our broad evaluation and motivated ontology-grounded RL. Full results (Tables 1-6) are at https://anonymous.4open.science/r/RAREDX_rebuttal/rebuttal3.md.

W1 & Q1: Harness definition and final-system contribution

We agree that “Harness” denoted two objects. RareDx-Harness is an evaluation methodology, not a novel routing algorithm. It fixes the task, Top-10 contract, knowledge layer, normalizer, and metrics while varying evidence access. Figure 1's Direct, Static RAG, ReAct, and Structured 3-hop are controlled studies of when knowledge helps; they do not all feed the final prediction.

Operationally, a dataset-level router chooses frozen Direct or Diagnostic Audit. The 35B auditor sees the case and Direct list with a fixed prompt, greedy decoding, 384-token cap, and no retrieval/tools. We will rename the two objects and revise Figure 1 (Table 3).

We have not audited every baseline, so cannot claim model-invariant audit gains. Proprietary baselines had no audit; we withdraw any implication that the 9B policy alone exceeds GPT-5.5. The attribution is:

SystemEvaluationH@1H@5H@10
Base 9BFull 8-source9.5320.6924.77
KGPO DirectFull 8-source16.9025.5431.35
KGPO DirectStrict test16.2924.7130.54
Validation-selected Direct/AuditStrict test18.8130.2137.34

Thus policy training and compute-augmented auditing provide distinct gains; only the first two rows are a policy-only comparison. Table 4 gives every development decision and both frozen test candidates.

Q2: a new cohort without labeled development data

The router is not a case-level gate. For an unlabeled cohort, the prespecified default is Direct. Audit is enabled only if representative labeled local development data support it under the fixed Hit@10, Hit@5, Hit@1 rule. Eight source-held-out evaluations show breadth, not prospective cross-hospital validation; we will not claim zero-shot router generalization.

Q3: label isolation, overlap, and dataset composition

Loaders use only designated free text, phenotypes, and benchmark-supplied genetic findings. Gold diseases are loaded separately by the evaluator and, during training, remain in reward metadata rather than the prompt. Genetic findings are intended evidence, not labels.

Against all test sources we checked normalized identifiers, SHA-1 case text, and disease-HPO profiles at Jaccard at least 0.90. SFT/RL training have zero identifier and zero text matches. SFT has zero profile matches; RL has one with LIRICAL among 61,918 prompts. RL validation has 290 profile matches with Phenopackets but no identifier/text match, potentially favoring selection. Excluding Phenopackets leaves Harness Hit@10 at 37.47 versus GPT-5.5's 35.41. Retrieval contains disease-level ontology/gene records, not benchmark patient reports.

Table 1 reports cases, unique primary diagnoses, median/maximum disease frequency, and accepted strings per case. RDS/RDC have a median of six accepted strings because aliases are accepted for one primary diagnosis, not because each case has six independent gold diseases.

Q4: intended clinical use

RareDx is a differential-diagnosis support tool, not an autonomous diagnostician. Its Top-10 list helps prioritize phenotype review, gene interpretation, and confirmatory testing. Hit@1 measures first choice; Hit@5/10 measure screening retention. Modest Hit@1 is a limitation: predictions require expert verification and must not determine treatment. We will foreground the lack of prospective validation.

W2 & Q5: reward-component attribution

The audit tests reward behavior, not downstream causality. A phenotype neighbor scores 0.316 versus 0.008 without graph/phenotype credit. Without validity safeguards, a pseudo-disease changes from -0.175 to +0.320 and is rewarded in 82.5% of probes; placing gold at rank 11 changes 0.279 to 0.068 (Table 5). On 932 unseen cases, exact Top-1 rises from SFT's 78.22% to KGPO's 85.09%. This supports complete KGPO, not each term; matched train-from-SFT ablations are required, and we will narrow the claim.

Q6: OpenReview/PDF inconsistency

This is a version-control error, not a second model. “28B” was an earlier rounded label; the evaluated checkpoint is Qwen3.8-27B. We will make the OpenReview title/abstract match the PDF verbatim, use RareDx-27B throughout, and record the checkpoint identifier (Table 6).

We appreciate these important concerns. We will separate research from deployment, disclose overlap, and narrow unsupported claims. We would be grateful if you would kindly consider raising the score. Thank you again for your time and constructive feedback.

Thank the authors for the additional tables and clarifications. These additional results address my concerns about the Harness definition, dataset statistics, and evaluation.

My two major concerns are still not addressed.

  1. As mentioned in my Weakness 2, the contributions of Harness and KGPO are still not fully isolated. An important missing ablation is Base Qwen3.5-9B + Harness (35B Audit) without KGPO. Although the authors acknowledged that this factorial ablation is missing, I still consider it to be an important experiment for attributing the final-system gains.
  2. Could the author extend Table 5 to provide downstream ablations (i.e. Hit@1,5,10) for individual KGPO reward components?

A smaller concern: while the authors state that the gold labels are loaded separately, have you checked whether the evaluation inputs (i.e. free-text or genetic findings) can directly reveal the diagnoses? I have checked RareArena’s paper (Chen et al., 2026), and the authors stated that they perform potential diagnosis leakage screening (not even remove the labels, but change some wording accordingly) during the dataset construction. Is a similar screening step ensured for other benchmarks? Have the authors done the dataset screening and cleaning in advance?

These remaining concerns are the main reasons I am maintaining my current rating 2.

We thank the reviewer for the constructive follow-up and for confirming that our previous response resolved the questions on the Harness definition, dataset statistics, and evaluation setup. Full results (Tables 1-5) are at https://anonymous.4open.science/r/RAREDX_rebuttal/rebuttal4.md.

1. Factorial Ablation of Base Model, KGPO, and Harness

We evaluated Base Qwen3.5-9B + Harness (35B Audit) without KGPO (as well as Top-10 SFT-9B and unrouted Always-Audit) under both the Full 8-source protocol and the Strict 80% disjoint test split (20% dev-selected route). Complete breakdowns and 27B results are in Tables 1-2; key rows are:

SystemProtocolH@1H@5H@10
Base 9B + DirectFull / Strict test9.53 / 9.2920.69 / 20.1624.77 / 24.34
Base 9B + Harness (35B Audit)Full / Strict test11.83 / 11.3025.09 / 24.1030.00 / 28.83
Top-10 SFT-9B + HarnessFull / Strict test16.08 / 15.1227.73 / 26.6234.41 / 33.26
KGPO 9B + DirectFull / Strict test16.90 / 16.2925.54 / 24.7131.35 / 30.54
KGPO 9B + Harness (35B Audit)Full / Strict test20.11 / 18.8131.54 / 30.2138.34 / 37.34
  1. KGPO alone outperforms Base 9B + Harness without 35B Audit calls: Under both protocols, KGPO 9B + Direct surpasses Base 9B + Harness at every cutoff (+5.07 H@1 and +1.35 H@10 on Full; +4.99 H@1 and +1.71 H@10 on Strict test).
  2. Complementary roles: The 35B Audit has no retrieval or tools and only reranks the policy's Top-10 list under a 384-token cap. When Base 9B omits the true disease neighborhood (e.g., Phenopackets 20.6%, LIRICAL 19.7%; Table 2), Audit cannot recover it. KGPO populates the initial Top-10 list with ontology- and phenotype-relevant candidates, amplifying the Harness gain (+5.23 H@10 on Base 9B vs. +6.99 on KGPO 9B).

2. Downstream KGPO Reward Ablations

We extended our previous Table 5 by training matched 9B policies from the same SFT checkpoint, ablating each reward channel (gk,ok,pk,ekg_k, o_k, p_k, e_k) and safeguard (Cclean,PhedgeC_{\text{clean}}, P_{\text{hedge}}). Full results (including Harness scores, nDCG w=0.3w=0.3, and per-dataset scores) are in Tables 3-4; key Direct results are:

Reward Configuration (Trained from SFT-9B)Val Top-1Direct H@1Direct H@5Direct H@10OOV (%)
Stage 1: Top-10 SFT Initialization78.2213.4322.0327.594.18
Full RareDx-KGPO85.0916.9025.5431.350.62
w/o Graded-Gold Relevance (gk=0g_k = 0)79.5113.8423.1328.900.85
w/o Ontology Proximity (ok=0o_k = 0)82.6215.4623.6429.140.74
w/o Phenotype Profile (pk=0p_k = 0)82.1915.6823.7529.280.71
w/o Biomedical Embedding (ek=0e_k = 0)84.1216.3124.9530.610.68
Sparse Exact-Match Only (ok=ek=pk=0o_k=e_k=p_k=0)80.1514.5322.8928.240.92
w/o Vocab Gate & Contamination (Cclean=1C_{\text{clean}} = 1)74.4612.1419.8424.1328.45
w/o Output-Budget Gate (Phedge=1P_{\text{hedge}} = 1)81.3315.1323.4528.731.84

Removing gkg_k drops H@1 by 3.06 points because dense channels are capped (≤0.50\le 0.50). Removing oko_k or pkp_k reduces H@10 by 2.21 and 2.07 points (-3.11 for Sparse Exact-Match Only). Removing the vocabulary gate (CcleanC_{\text{clean}}) raises fabricated/OOV strings from 0.62% to 28.45% and collapses H@10 below SFT (24.13% vs. 27.59%).

3. Input Modality and Leakage Screening

  1. 0% free-text narrative in the six non-RareArena benchmarks: RareArena (RDS/RDC) uses published clinical case reports, requiring narrative wording screening (Chen et al., 2026a). In contrast, MyGene2, RAMEDIS, MME, HMS, LIRICAL, and Phenopackets (2,368 cases) contain 0.00% free-text clinical narratives (Table 5); inputs are strictly structured HPO term lists (e.g., HP:0001943) plus candidate gene symbols, precluding narrative wording leakage by construction.
  2. Programmatic screening across all 4,249 prompts: Scanning all serialized test prompts against the 27,554-disease vocabulary and accepted synonym sets G(x)\mathcal{G}(x) (exact substring and token-span Levenshtein distance ≤0.20\le 0.20) confirmed 0 / 2,368 (0.00%) matches in the six structured sets and 0.00% explicit gold-label presence in RDS/RDC (Table 5).
  3. Genetic findings: Genotype tracks supply candidate gene symbols rather than disease names, mapping to 3.9 to 5.2 distinct rare diseases per gene on average in our knowledge graph (pleiotropy; Table 5), which requires joint phenotype-genotype disambiguation (Base 9B reaches only 7.2%-10.2% H@1 on gene-containing sets).

We will incorporate Tables 1-5 into the revision. We would be grateful if you would kindly consider raising your score.

Reviewer_RLbLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RareDx, a framework for rare-disease diagnosis that combines controlled knowledge utilization with knowledge-graph-grounded policy optimization. The framework consists of two main components: RareDx-Harness, which evaluates different diagnostic strategies, including direct inference, retrieval-augmented generation, adaptive tool use, and structured reasoning; and RareDx-KGPO, which incorporates disease ontology relationships, biomedical semantic similarity, and phenotype information into reinforcement learning rewards.

Critical strengths
  1. The paper addresses rare-disease diagnosis, where incomplete phenotypes, sparse disease-specific data, and fragmented medical knowledge pose significant challenges.

  2. The proposed KGPO incorporates multiple sources of medical knowledge and introduces safeguards against invalid disease names and reward exploitation. The reward analyses and controlled adversarial probes provide useful evidence of the effectiveness of these constraints.

  3. The paper includes evaluations of different model scales, retrieval strategies, reward components, and computational efficiency.

Critical weaknesses
  1. The reported RareDx-9B Harness performance relies on an additional Qwen3.6-35B-A3B model for diagnostic auditing, whereas the proprietary baselines are evaluated without equivalent auditing. This introduces unequal inference resources and makes it difficult to attribute the reported performance advantage to the proposed KGPO method.

  2. The paper does not clearly report a comprehensive comparison among the original backbone, SFT-only model, SFT with standard GRPO, and SFT with RareDx-KGPO across the eight benchmarks. Therefore, the contributions of domain-specific supervised fine-tuning and the proposed knowledge-grounded reinforcement learning cannot be adequately separated.

  3. The framework combines a lot of techniques, including medical knowledge-graph matching, ontology-based similarity, retrieval-augmented reasoning, GRPO-based post-training, and rule-based reward constraints. Related studies, such as DeepRare[1] and RareDxR1[2], have already explored knowledge-enhanced reasoning and reinforcement learning for rare-disease diagnosis. The authors should clarify which technical innovations extend beyond the integration of established methods.

[1] “An Agentic System for Rare Disease Diagnosis with Traceable Reasoning,” Nature, vol. 651, no. 8106, pp. 775–784, 2026.

[2] "RareDxR1: Autonomous Medical Reasoning for Rare Disease Diagnosis Beyond Human Annotation." arXiv preprint arXiv:2607.00147 (2026).

Questions for authors

See Weakness.

3 comments

We thank the reviewer for the careful assessment, weak-accept rating, and recognition of our medical reward design, adversarial probes, broad evaluation, and efficiency study. Full results (Tables 1-5) are at https://anonymous.4open.science/r/RAREDX_rebuttal/rebuttal2.md.

W1: fair attribution under unequal inference resources

We agree. The submission did not distinguish sufficiently between a 9B policy result and a system result invoking a 35B auditor. Proprietary baselines receive no audit, so Harness is not evidence that KGPO alone surpasses GPT-5.5. We will remove that implication and separate Direct from Harness.

The fair policy-only comparison is:

SystemExtra auditorH@1H@5H@10
Qwen3.5-9BNo9.5320.6924.77
RareDx-9B, KGPO DirectNo16.9025.5431.35
Qwen3.6-35B-A3BNo17.9327.6832.98
GPT-5.5No22.0131.5936.74

Thus KGPO Direct improves the same 9B backbone by +7.37/+4.85/+6.58 without retrieval, tools, or an auxiliary model. It is competitive with, but does not exceed, the standalone 35B model, and remains below GPT-5.5. This is the appropriate evidence for post-training effectiveness.

Harness addresses a different question: whether a validation-selected audit can improve the deployed system. On the disjoint strict test split, the same frozen KGPO outputs improve from 16.29/24.71/30.54 to 18.81/30.21/37.34 after dataset-level Direct/Audit selection. We will report this +2.52/+5.50/+6.80 increment as an explicitly compute-augmented result, not as a 9B-only comparison. Table 3 at the link gives both candidates and the development-only decision for every dataset.

W2: separating base, SFT, standard GRPO, and KGPO

We agree that Table 1 should have made the training stages easier to isolate. Two controlled comparisons are currently supported:

  1. Across all eight benchmarks, Base to KGPO Direct improves the macro from 9.53/20.69/24.77 to 16.90/25.54/31.35; Table 2 at the link gives every dataset and cutoff.
  2. On the same 932-case unseen validation set, with identical prompt, parser, canonicalizer, and deterministic decoding, exact Top-1 rises from 78.22% for the Top-10 SFT checkpoint to 85.09% for KGPO at step 270 (+6.87 points). This is held-out accuracy, not training reward.

The archived standard-GRPO runs predate the finalized eight-source data and normalizer, so inserting them would not be controlled. The evidence isolates an incremental benefit of complete KGPO over SFT, but not a fully matched eight-benchmark SFT-versus-GRPO-versus-KGPO decomposition. We will state this boundary and avoid attributing the entire gain specifically to graded graph credit until that control is available.

W3: novelty relative to DeepRare and RareDxR1

We appreciate these closely related works and will sharpen the positioning. DeepRare contributes a large multi-agent clinical support system with more than 40 tools, self-reflection, and traceable external evidence. RareDxR1 contributes end-to-end reasoning from unstructured notes through knowledge internalization, Reflection-Enhanced Reasoning Sampling, and curriculum RL. We do not claim novelty for knowledge graphs, retrieval, or group-relative optimization individually.

RareDx targets two different gaps. First, RareDx-Harness holds the task, output contract, disease normalizer, and knowledge infrastructure fixed while comparing direct inference, static retrieval, adaptive tool use, and structured phenotype-gene-disease reasoning. This controlled design exposes when retrieval hurts, rather than assuming that additional context helps. Second, KGPO converts canonical disease-graph relations into a rank-sensitive training signal for a compact policy: curated relevance, ontology distance, semantic similarity, and phenotype consistency are combined with canonical-vocabulary projection and explicit anti-contamination/list safeguards. Unlike an inference-time agent, KGPO Direct internalizes this supervision and requires no tools or larger auditor. Table 5 at the link contrasts these objectives and mechanisms explicitly.

Moreover, DeepRare is based on models with much larger parameter size, and it does not involve model training and adaptation. RareDx-R1 seems a closed-source model, and in our benchmark we have compared our method versus GRPO training model. We will include more detailed discussion in our revised paper for clarification.

We hope the resource-matched reporting, SFT-controlled held-out gain, and clarified technical scope resolve the concerns. We would be grateful if you would kindly consider raising the score. Thank you again for your time and constructive feedback.

I appreciate the authors' thorough rebuttal and the extra validations provided during the discussion period. The new results help clarify the experimental setup and strengthen the presentation. However, taking into account the overall contribution, I feel my initial rating already represents the upper bound for this submission. Therefore, I will maintain my score.

Thank you for your comments and effort. We have clarified our main contribution in both paper and rebuttal information. We believe that we offer a strong and solid method for designing an agentic system based on post-training for rare disease diagnosis. Please let us know if you have any questions and we will be happy to discuss and address.

Reviewer_ngKqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RareDx combines Top-10 SFT, knowledge-graph-grounded policy optimization (KGPO), and a diagnostic evaluation harness. It studies retrieval and tool-based strategies, while the final router selects between Direct predictions and a separate 35B diagnostic auditor. Experiments cover eight benchmark sources.

Critical strengths
  1. The reward combines partial credit for related diagnoses with explicit constraints on invalid disease names and excessively long lists. These safeguards address practical failure modes of dense medical rewards.

  2. Evaluation covers heterogeneous diagnosis settings and includes a development/test routing audit. The finding that retrieval can hurt performance is useful, and the authors disclose the exploratory headline routing and use of the separate auditor.

Critical weaknesses
  1. The retrieval study does not clearly explain the final system gains. Figure 1 and Table 2 examine RAG, ReAct, and structured retrieval, whereas the operational router in Appendix C chooses Direct or a 35B auditor that uses no retrieval and can revise or add diagnoses. Table 2 also uses different backbones from the final RareDx checkpoints. These settings are disclosed, but the experiments do not establish how the proposed knowledge-use mechanisms contribute to the final gains beyond the additional diagnostic audit.

  2. KGPO's incremental training benefit is not isolated. Appendix E shows that SFT targets already incorporate KG phenotype and ontology neighbors, but Table 1 does not include SFT-only results. The reported comparisons also do not separate the benefit of graded medical supervision from that of the output safeguards. Training reward curves and fixed-output reward audits cannot establish how much the graded reward improves held-out diagnostic accuracy.

Questions for authors
  1. How do base, SFT-only, and KGPO outputs compare when processed with the same auditor, audit prompt, and routing-selection protocol?

  2. Could the authors report SFT-only results and compare KGPO with an exact-match-only version of the same rank-weighted reward, starting from the same SFT checkpoint and holding the RL setup, vocabulary/output safeguards, training data, and budget fixed?

3 comments

We thank the reviewer for the careful reading and for recognizing the value of the reward safeguards, heterogeneous evaluation, and leakage-controlled routing audit. Detailed results (Tables 1-3) are available at https://anonymous.4open.science/r/RAREDX_rebuttal/rebuttal1.md; we summarize the decisive evidence here.

W1 & Q1: attribution of the final-system gain

We agree that the submission did not separate the roles of the knowledge-use study, trained policy, and auditor sharply enough. Figure 1 and Table 2 are controlled mechanism studies: they test static RAG, ReAct, and structured HPO reasoning under their stated backbones. They were not intended to attribute the headline gain. The operational Harness instead chooses once per benchmark between (i) the frozen RareDx Top-10 list (Direct) and (ii) a Qwen3.6-35B-A3B Audit of that list. The auditor receives the original case and Direct list, uses one fixed prompt, greedy decoding, a 384-token cap, and no retrieval or tools. Routing is dataset-level, never case-level.

The strict 20/80 development/test audit cleanly isolates the auditor on frozen KGPO outputs:

9B system, disjoint testH@1H@5H@10
KGPO Direct16.2924.7130.54
Validation-selected Direct/Audit18.8130.2137.34
Auditor/router increment+2.52+5.50+6.80

These are eight-source macros recomputed from Appendix Table 5; Table 2 at the link gives every development choice and both test candidates. Selection uses development Hit@10, then Hit@5 and Hit@1; ties choose Direct. No test score enters routing. Separately, on the complete evaluation, the unadapted 9B backbone scores 9.53/20.69/24.77 and KGPO Direct scores 16.90/25.54/31.35. Thus the trained policy improves performance before auditing, while validation-selected auditing provides an additional gain. Neither result implies that retrieval causes the final gain. We will relabel the paper's Table 2 as a mechanism study, remove wording that conflates it with the operational router, and use the strict route as the primary routing estimate.

The reviewer is correct that a full Base/SFT/KGPO-by-Audit factorial table is absent. Auditing only KGPO cannot determine whether the auditor interacts differently with weaker policies. We will state this boundary explicitly rather than extrapolate across unmatched backbones.

W2 & Q2: incremental KGPO benefit and reward control

We recovered a checkpoint-controlled result from the archived trajectory:

Frozen 9B checkpointExact Top-1Change
Top-10 SFT initialization78.220
KGPO, final stage85.09+6.87

Both rows use the same 932 unseen validation cases, prompt, parser, canonicalizer, and deterministic decoding. This is held-out diagnostic accuracy, not training reward, and directly establishes an incremental RL benefit over SFT. Table 1 at the link additionally reports Base and KGPO Direct for every benchmark and cutoff.

For precision, the deployed reward is

H = \max_{1 \leq k \leq 10} \frac{c_k}{\log_2(k+1)}$$ where $c_k = \max\{g_k,o_k,e_k,p_k\}$ combines curated relevance, ontology proximity, embedding similarity, and phenotype consistency. Canonical-vocabulary projection precedes scoring; $q=1$ for an exact Top-1 diagnosis and otherwise $q=2^{-m}$ for $m$ contaminated lines. Hence $c_k$ supplies graded medical supervision, while vocabulary, contamination, and list-length terms constrain output behavior. We agree that fixed-output reward audits cannot replace the requested training ablation. No archived run satisfies all controls: the available “exact-rank” run starts from a different 27B RL checkpoint, so reporting it would be misleading. The valid control must start from the identical 9B SFT checkpoint and change only $$c_k \leftarrow \mathbf{1}\!\left[\mathrm{canon}(y_k) \in G\right]$$ while fixing data order, rollout budget, optimizer, vocabulary projection, and safeguards. Until this matched run is complete, we will narrow the claim: current evidence establishes the incremental benefit of the **complete KGPO objective**, but does not quantify how much arises from graded credit alone versus its safeguards. We appreciate that these questions exposed attribution ambiguity. The revision will clearly separate mechanism studies, KGPO Direct, and validation-selected auditing, and will add the SFT-controlled result without overstating the pending exact-only comparison. We hope these clarifications and the \+6.87-point held-out gain address the main soundness concern. We would be grateful if you would kindly consider raising the score. Thank you again for your time and constructive feedback.

Thank you for the response. The clarifications on the Harness, the policy-only comparison and the narrowed claims are helpful. My main concerns remain, though. Base/SFT + Audit is still missing, and so is the exact-match-only reward control that would test the central KGPO claim. The 932-case SFT vs. KGPO result is in-distribution and Top-1 only, and it is unclear whether the step-270 checkpoint was selected on these same cases. SFT-only results on the eight benchmarks and Base/SFT + Audit only need inference. I will keep my score for now and would reconsider if these results are provided.

We thank the reviewer for the constructive follow-up and willingness to reconsider the score. We have completed all requested evaluations; full per-benchmark results (Tables 1-5) are at https://anonymous.4open.science/r/RAREDX_rebuttal/rebuttal5.md.

1. SFT-Only on Eight Benchmarks & Base / SFT / KGPO + Audit (W1, W2 & Q1)

We evaluated frozen Base 9B, Top-10 SFT-9B, Exact-Match-Only RL, and KGPO 9B across all eight external benchmarks using the identical 35B auditor (same prompt, greedy decoding, 384-token cap, no tools) and strict 20/80 dev/test routing protocol (Tables 1-2 report unrouted Always-Audit and all per-dataset dev/test decisions):

9B PolicyFull DirectFull HarnessStrict DirectStrict Harness
Base 9B9.53/20.69/24.7711.83/25.09/30.009.29/20.16/24.3411.30/24.10/28.83
Top-10 SFT13.43/22.03/27.5916.08/27.73/34.4112.96/21.48/26.9215.12/26.62/33.26
KGPO16.90/25.54/31.3520.11/31.54/38.3416.29/24.71/30.5418.81/30.21/37.34
  1. Out-of-distribution SFT vs. KGPO across all cutoffs: Without auditing, KGPO improves over SFT-9B across the eight external benchmarks by +3.47 H@1, +3.51 H@5, and +3.76 H@10 on Full (+3.33/+3.23/+3.62 on Strict test; per-dataset scores in Table 4). While SFT targets include static KG neighbors, SFT fits token cross-entropy on one fixed list ordering; KGPO optimizes rank-discounted rewards across 16 rollouts, penalizing patient-inconsistent distractors.

  2. Policy vs. Auditor: Without 35B Audit calls, KGPO Direct (16.90/25.54/31.35) beats Base 9B + Harness (11.83/25.09/30.00) at all cutoffs and exceeds SFT + Harness at H@1 (16.90 vs. 16.08). Unrouted Always-Audit shows the same ordering on Strict test (Base 11.20/23.79/28.60, SFT 14.68/26.15/32.10, KGPO 17.91/29.18/35.46; Table 1). Under the identical routing protocol, auditing adds +4.49 H@10 to Base 9B, +6.34 to SFT, and +6.80 to KGPO on Strict test (Table 2), showing that stronger policies supply higher-recall Top-10 lists for the tool-free auditor to rerank.

2. Matched Exact-Match-Only RL Control & Safeguard Isolation (W2 & Q2)

From the identical 9B SFT checkpoint, data order, rollout budget, optimizer, vocabulary projection, and safeguards, we trained the exact-match-only control (c_k in {0, 1}) and safeguard/channel ablations (Tables 3-4):

9B Objective (From SFT)Direct H@1/5/10Harness H@10OOV (%)
Stage 1: Top-10 SFT13.43 / 22.03 / 27.5934.414.18
Exact-Match RL + Gates14.53 / 22.89 / 28.2435.040.92
Graded KG w/o Vocab Gate12.14 / 19.84 / 24.1331.0828.45
Graded KG w/o Budget Gate15.13 / 23.45 / 28.7335.601.84
Full RareDx-KGPO16.90 / 25.54 / 31.3538.340.62

Holding all safeguards fixed, Exact-Match RL yields modest gains over SFT (+1.10/+0.86/+0.65 Direct) because zero-hit rollout groups produce zero variance on hard rare diseases. Replacing binary match with graded KG credit adds +2.37 H@1, +2.65 H@5, and +3.11 H@10 (68%, 75%, and 83% of the total RL gain over SFT). Conversely, removing the vocabulary gate raises OOV pseudo-diseases to 28.45% and drops H@10 below SFT (24.13% vs. 27.59%). Single-channel ablations confirm complementary roles: removing g_k (13.84/23.13/28.90), o_k (15.46/23.64/29.14), p_k (15.68/23.75/29.28), or e_k (16.31/24.95/30.61) consistently degrades Direct ranking (Tables 3-4).

3. Checkpoint-Selection Audit: Step-270 vs. Final Step-300

The 932-case validation split selected step-270 (where SFT scores 78.22/84.12/86.48 at H@1/5/10 vs. KGPO's 85.09/89.70/91.63; Table 3). To verify that out-of-distribution gains do not rely on validation selection or Phenopackets overlap, Table 5 evaluates the unselected final checkpoint (step-300) and the 7-benchmark macro excluding Phenopackets:

  • Unselected final step-300 (8-benchmark Direct): Scores 16.82/25.46/31.24 (within 0.11 of step-270), outperforming SFT-9B by +3.39/+3.43/+3.65 and Exact-Match step-300 (14.46/22.81/28.15) by +2.36/+2.65/+3.09.

  • 7-benchmark macro (excluding Phenopackets): KGPO Direct improves over SFT-9B from 11.95/20.37/26.27 to 14.17/23.36/29.49 (step-270) and 14.11/23.29/29.39 (step-300).

We deeply appreciate the time, effort, and profound expertise you have dedicated to reviewing our work, for which we are immensely grateful. We will incorporate Tables 1-5 and all attribution clarifications into the revision. We hope these complete evaluations resolve your remaining concerns and would be sincerely grateful if you would kindly reconsider and raise your score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.