acceptodds
Under review as a conference paper at ICLR 2027

RareVA: Adaptive Agentic Verification for Rare Disease Candidate Prioritization from Facial Images under Data Scarcity

Abstract

Rare disease candidate prioritization from facial images is particularly challenging under extreme data scarcity, where genetic testing and structured clinical annotations may be unavailable and visually similar disorders can exhibit overlapping craniofacial phenotypes. Existing approaches typically formulate this setting as either direct visual classification or end-to-end vision-language reasoning, providing limited mechanisms for explicitly verifying why one candidate disease should be favored over competing alternatives. We propose **RareVA**, an image-first system that formulates rare disease candidate prioritization as candidate-conditioned disease verification. Rather than directly selecting a disease from the visual prediction space, RareVA first identifies a small set of plausible candidates, extracts image-derived facial phenotypes through a vision-language model, and verifies each candidate through agentic reasoning over evidence retrieved from a disease-phenotype knowledge graph. To improve computational efficiency, RareVA further introduces Adaptive Candidate Selection (ACS), which dynamically allocates the number of candidates requiring verification according to case difficulty. We evaluate RareVA on RDFace and further assess generalization on the independent GMDB benchmark. RareVA improves Top-1 accuracy from 25.66% to 34.51% on RDFace and from 7.56% to 12.25% on GMDB. ACS further reduces average output-token cost by 17.15% relative to the best-performing fixed-budget configuration while achieving comparable observed accuracy. These results support candidate-conditioned verification as a formulation for evidence-grounded rare disease candidate prioritization from facial images under severe data scarcity.

Primary area:
interpretability and explainable AI
Keywords:
Multi-agent systems, Vision-language models, Rare disease, Retrieval-augmented generation, Adaptive inference, AI for Health

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_p3QSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RareVA, a multi-agent framework for identifying rare diseases from facial images. RareVA uses a visual classifier to select top-k candidates and a VLM to extract phenotypes. Then, doctor agents verify selected candidates and a judge agent outputs the final prediction. The adaptive candidate Selection (ACS) policy is proposed to allocate token budgets. Experiments show that RareVA can improve accuracy while reduce token costs.

Critical strengths
  • The motivation is clear and easy to follow. The candidate-conditioned verification formulation provides a clearer and more explainable way to identify rare diseases.
  • Connecting knowledge bases like HPO with the image is practical.
  • The experimental section of RareVA provides detailed statistical metrics like confidence intervals, making the reported results more reliable.
Critical weaknesses
  • RareVA first uses a vision classifier to output the candidate diseases and a VLM extracts phenotypic evidence. Why don't let the VLM also output the candidate diseases?
  • Some experimental settings are not clear, making the method difficult to reproduce. What is the trainable part in RareVA? Is it only the ACS policy and vision classifier? What is the detailed training settings of ACS?
  • In the ablation of Table5, the performance of 'w/o ACS K=4' is higher than RareVA, and is statistically better. This makes the ACS less valuable if its function is just to save tokens. Since using a multi-agent system is inherently token-intensive, performance improvement is more important. And why isn't the bolded result the best one?
  • Setting the max tokens as 1k limits the capability of the LLM.
Questions for authors
  • What does the '¬' mean in Line 7 of Algorithm 1?
6 comments
AuthorsOpenReview ↗

Response to Reviewer p3QS — Part 1/2: Summary and W1, W2 — Candidate Generation, Trainable Components, and ACS Training Reproducibility

Dear Reviewer p3QS,

Thank you for your constructive feedback and for recognizing the motivation, interpretability, and statistical rigor of RareVA. To address your concerns, we conducted additional experiments on VLM-based candidate generation (W1, Table R12), ACS accuracy–efficiency trade-offs on RDFace and GMDB (W3, Tables R15–R16), and generation-length sensitivity (W4, Table R17). We also provide detailed clarification of RareVA's trainable components and ACS training protocol (W2, Tables R13–R14).

Notably, our results show that the task-specific visual classifier achieves substantially higher Top-5 candidate recall than direct VLM-based proposals (43.36% vs. 8.85–10.62%). Furthermore, ACS maintains competitive accuracy while reducing output-token usage by 17.15% on RDFace and 19.45% on GMDB relative to fixed K=4. Increasing the generation limit beyond 1,024 tokens also does not improve observed accuracy.

We have updated the revised manuscript to incorporate these additional analyses, implementation details, and clarifications. We hope these results help address your concerns. Our detailed, point-by-point responses are organized into two parts, beginning with W1 below.


W1. Why Not Use the VLM for Candidate Generation?

RareVA is designed to separate visual candidate generation from phenotype-grounded VLM verification (Section 3.1). The task-specific visual classifier proposes a bounded set of disease hypotheses from facial images, allowing VLM agents to focus on phenotype interpretation, KG-grounded evidence assessment, and candidate-conditioned reasoning (Section 3.3). This separation is a deliberate architectural choice rather than an assumption that VLMs cannot generate disease candidates.

To empirically assess the reviewer's suggestion, we conducted an additional comparison using the same VLM backbone (Gemini) as RareVA's reasoning agents: (1) direct VLM proposal, given the facial image and all 103 target disease labels; and (2) VLM with in-context learning (ICL), additionally given one labeled training image per disease. Both generate Top-5 candidates on the same RDFace test set.

Table R12. Top-5 ground-truth candidate recall on RDFace.

Candidate generation methodTop-5 Recall (%)
Direct VLM proposal8.85
VLM + 103-disease ICL10.62
VisCls (DenseNet-169)43.36

The additional results provide empirical support for our original architectural choice. As in Table R12, VisCls achieves substantially higher Top-5 recall (43.36%) than direct VLM proposal (8.85%) or VLM with ICL (10.62%) under the evaluated protocol. Thus, RareVA retains a task-specific visual candidate generator while using VLMs for structured verification rather than unrestricted disease proposal (Section 3.1; Appendix C.1).


W2. Trainable Components and ACS Training Reproducibility

We thank the reviewer for highlighting the need for clearer implementation details. Yes, only the visual classifier (VisCls) and ACS policy are trained specifically for RareVA. All VLM components remain frozen and operate through prompting, while phenotype normalization and KG retrieval require no additional training (Section 3; Appendix M). In Table R13, we summarize the details of each component in RareVA.

Table R13. Trainable components in RareVA.

ComponentImplementationTrainable?
Visual ClassifierDenseNet-169Yes
Phenotype ExtractorQwen3-VL-8BNo
ACSMLPYes
Planner / Doctor / JudgeGemini-2.5-FlashNo

ACS is trained through offline utility regression (Appendix E.2–E.3) using the 46-image RDFace validation split, with downstream rollouts for each K∈{2,3,4,5}K\in\{2,3,4,5\}. The utility balances prediction correctness and output-token cost (Appendix E.3, Eq. 12):

$

u(x,K)=1[y^K=yGT]−λ cost(x,K).u(x,K)=\mathbb{1}[\hat y_K=y_{\mathrm{GT}}]-\lambda\,\mathrm{cost}(x,K).

$

An MLP predicts utility from pre-reasoning routing features and a one-hot encoding of KK, selecting the highest-utility budget at inference. Hyperparameters are selected via 5-fold cross-validation on the validation split, followed by training a single policy checkpoint. The test set is excluded from training and model selection, and inference uses neither ground-truth labels nor downstream reasoning outcomes. The training configuration is summarized in Table R14.

Table R14. ACS training configuration (Appendix M, Table M.4).

ParameterValueParameterValue
Training dataVal (46 images)Candidate K{2, 3, 4, 5}
ObjectiveUtility regression (MSE)OptimizerAdamW
Learning rate1 × 10⁻³Weight decay1 × 10⁻⁴
Epochs25Batch size256
Hidden dimension64Dropout0.1
Cost metricOutput tokensUtility penalty λ≈1.16 × 10⁻⁵

We have incorporated these implementation details into Appendix M (Tables M.4) to improve reproducibility.

AuthorsOpenReview ↗

Response to Reviewer p3QS — Part 2/2: W3, W4, Q1 — ACS Effectiveness and Accuracy–Cost Trade-off, Generation Length, and Algorithm Notation

W3. ACS Effectiveness and Accuracy–Cost Trade-off

We thank the reviewer for raising this important concern regarding ACS effectiveness and the importance of predictive accuracy.

(1) We clarify the statistical comparison. Fixed K=4 does not significantly outperform ACS on RDFace (McNemar's exact p=1.0; paired bootstrap 95% CI for ACS − K=4: [−6.19, 4.42] pp; Section 4.7; Appendix K.3). The observed Top-1 accuracies are 35.40% and 34.51%, respectively.

(2) We distinguish ACS from the diagnostic verification framework. Without ACS, K becomes a globally selected hyperparameter; the candidate-conditioned Doctor–Judge framework remains unchanged. ACS instead learns case-dependent verification allocation from downstream utility (Section 3.3.1; Appendix E.2–E.4). Thus, the diagnostic accuracy contribution comes primarily from Doctor–Judge verification, while ACS addresses inference allocation under computational constraints (Sections 3.3.2–3.3.3; Section 4.8, Table 5).

Table R15. Accuracy–cost comparison on RDFace.

Verification policyTop-1 Acc. (%)Avg. output tokens
Fixed K=333.637,834
Fixed K=435.4010,064
Entropy-based routing29.207,154
ACS (Ours)34.518,338
Results from Section 4.8 (Table 5) and Appendix E.3 (Table E.1).

(3) We additionally evaluated ACS on the independent GMDB cohort (n=661) to examine its performance beyond RDFace. The RDFace-trained ACS policy was transferred without target-domain retraining, and all configurations used the same candidate-generation and Doctor–Judge verification protocol.

Table R16. External evaluation of verification budgets on GMDB.

Verification policyTop-1 Acc. (%)Avg. output tokens
Fixed K=311.657,674
Fixed K=411.9510,178
ACS (avg. K=3.21)12.258,198

Importantly, ACS achieves the highest observed accuracy on GMDB while reducing inference cost relative to fixed K=4. It improves Top-1 accuracy from 11.95% to 12.25% while using 19.45% fewer output tokens. Together, the RDFace and GMDB results indicate that adaptive verification does not necessarily require sacrificing predictive accuracy for efficiency, and provide additional evidence for transferring learned verification allocation across cohorts.

Finally, we have also corrected the bold formatting in Table 5 to highlight fixed K=4 (35.40%) as the highest-accuracy configuration on RDFace in the revised PDF.


W4. Sensitivity to Maximum Generation Length

We thank the reviewer for raising the concern that the 1,024-token generation limit may constrain the reasoning capability of the VLM agents. To directly assess this, we conducted an additional sensitivity analysis on RDFace by varying the maximum output tokens per generation from 800 to 4,096 (4× the original limit). We use fixed K=4 to isolate the effect of generation length from adaptive budget allocation, keeping all other inference settings unchanged (Appendix M, Tables M.2–M.3).

Table R17 (corrected). Sensitivity to maximum output tokens per generation on RDFace (fixed K=4).

Max output tokens / callTop-1 Acc. (%)Correct / 113
80032.7437
1,024 (original)35.4040
2,04834.5139
4,09634.5139
Incomplete or invalid agent generations are counted as incorrect rather than excluded.

Increasing the generation limit beyond 1,024 does not improve observed Top-1 accuracy: both 2,048 and 4,096 tokens yield 34.51% (39/113), compared with 35.40% (40/113) at the original limit. Conversely, reducing the limit to 800 tokens lowers accuracy to 32.74% (37/113).

These results indicate that the 1,024-token cap is not an observed accuracy bottleneck within the evaluated range, while avoiding the claim that it is globally optimal. We have included this sensitivity analysis into the revised manuscript (Appendix M, Tables M.2–M.3).


Q1. Notation in Algorithm 1

The symbol ¬\neg in Line 7 denotes logical negation (NOT), indicating that the Doctor Agent's stopping criterion has not yet been satisfied. Thus, the verification loop continues until either the maximum number of rounds Tmax⁡T_{\max} is reached or the stopping condition becomes true. We also noticed an extra comma (,) in this line, which is a typo error. We have removed it and clarified the condition as while round < T_max and not STOP(L_Doctor) do in the revised manuscript.

Thanks for the responses. The rebuttal addressed some of my concerns, but the responses to W1 and W4 are not convincing. The VisCls is trained, but the VLM is not trained, so it is not a fair comparison. And the results in Table R16 (by the way, it should be R17) are statistically unreliable with only 113 samples. I maintain my score.

Reviewer_VtetOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RareVA, for evidence-based rare disease candidate prioritization from facial images under data scarcity, utilizing candidate-conditioned verification and adaptive reasoning strategies. The contributions are the formulation of disease prioritisation as candidate-conditioned verification, separates visual perception from reasoning through an intermediate phenotype representation and multi-agent collaboration. The approach is clinically meaningful and improves interpretability by explicitly linking predictions to phenotypic evidence and disease-phenotype knowledge.

Critical strengths

Addresses the clinically relevant challenge of verification rather than unrestricted diagnosis. Uses HPO-normalized phenotypes and a phenotype knowledge graph, improving transparency and traceability of diagnostic reasoning. Comprehensive ablation studies demonstrate the contribution of key system components. Evaluates generalization on the independent GMDB dataset. ACS experiments provide useful insights into the trade-off between verification depth, accuracy, and computational cost.

Critical weaknesses

The method primarily integrates existing components (DenseNet, Qwen3-VL, HPO retrieval, and Gemini-based agents), with the main novelty lying in inference-time orchestration and candidate-conditioned verification rather than new algorithmic advances. While we understand that the paper is about rare diseases, the primary test set is still very small. Some of the claims are not supported statistically. In addition, demographic robustness is not established.

Questions for authors

The DenseNet-169 candidate generator achieves only 43.36% Top-5 recall, which fundamentally limits downstream performance. Have you evaluated stronger foundation model backbones (e.g., CLIP- or DINO-based representations) for candidate retrieval?

Can you provide an error breakdown across candidate generation, candidate selection, Doctor verification, and Judge adjudication stages?

Verification accuracy drops from 35.40% at K=4 to 28.32% at K=5. What causes this degradation?

How does performance vary for diseases with rich, sparse, or absent knowledge-graph phenotype associations?

Is facial image alone sufficient for target disorders?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Vtet — Part 1/3: Summary and W1–W2 — Novelty and Statistical Support

Dear Reviewer Vtet,

Thank you for your positive feedback and for recognizing the clinical relevance of candidate-conditioned verification, HPO-grounded reasoning, and adaptive computation. To address your concerns, we have provided further clarification of methodological novelty (W1), additional statistical evidence and demographic subgroup analyses (W2+W3, Tables R6–R7), and new experiments on stronger visual backbones (Q1, Table R8), pipeline error attribution (Q2, Table R9), verification-budget sensitivity (Q3, Table R10), and KG annotation coverage (Q4, Table R11).

Notably, our stage-wise error analysis identifies candidate generation as the primary bottleneck, accounting for 64/74 errors (86.49%), while RareVA correctly prioritizes 39/49 (79.59%) cases when the ground-truth disease is included in the initial candidate set. We also provide additional evidence of statistical reliability and external generalization, while acknowledging the limitations of small demographic subgroups.

We have updated the revised manuscript to incorporate these analyses and clarifications. We hope they help address your concerns. Our detailed, point-by-point responses are organized into three parts, beginning with W1 below.


W1. Methodological Novelty

RareVA's methodological contribution lies in adaptive, candidate-conditioned verification, rather than new pretrained foundation models. Two mechanisms distinguish it from conventional multi-agent reasoning:

(1) Utility-driven Adaptive Candidate Selection (ACS). ACS learns case-specific verification budgets from downstream accuracy–cost utility, rather than relying on a globally fixed K or classifier-confidence heuristic (Section 3.3.1; Appendix E.2–E.3). It achieves 34.51% Top-1 accuracy with 17.15% fewer output tokens than fixed K=4 (35.40%; Section 4.7, Figure 6).

(2) Hierarchical Doctor–Judge verification. Each Doctor evaluates a specific disease hypothesis through iterative KG retrieval and structured phenotype–disease matching; the Judge adjudicates the resulting evidence (Sections 3.3.2–3.3.3; Appendices G–H). RareVA improves Top-1 accuracy from 29.20% for the strongest multi-round VLM baseline to 34.51%, while removing the Doctor hierarchy or Judge reduces accuracy to 29.20% (Section 4.2, Table 1; Section 4.8, Table 5).

These results support the contributions of learned verification allocation and hypothesis-specific evidence adjudication, beyond simply combining pretrained models and additional reasoning rounds.


W2. Limited Test Size and Statistical Support

We acknowledge the limited RDFace test size (n=113). To assess the reliability of the observed gains, we report paired bootstrap confidence intervals, external-cohort validation, and paired statistical testing (Sections 4.3–4.5; Appendix I), summarized in Table R6.

Table R6. Statistical evidence for RareVA's Top-1 improvements.

EvaluationComparisonGain (pp)95% CI of gain (pp)
RDFace (n=113)RareVA 34.51 vs. VisCls 25.66+8.85[2.65, 15.93]
RDFace, GT in Top-5 (n=49)RareVA 79.59 vs. VisCls 59.18+20.41[6.12, 34.69]
External GMDB (n=661)RareVA 12.25 vs. strongest VLM 10.74+1.51[0.61, 2.57]

Importantly, GMDB provides an independent external evaluation beyond the small RDFace test set, with 661 cases and a distinct disease distribution. We explicitly excluded 15 overlapping patients to prevent patient-level leakage (Section 4.4; Appendix B.3). Despite this cohort shift, RareVA maintains a positive improvement over the strongest VLM baseline (12.25% vs. 10.74%), supported by paired bootstrap confidence intervals and McNemar's test (p=0.006; Section 4.5, Table 4).

In addition, four stochastic runs yield 39, 39, 39, and 38 correct predictions on RDFace (Appendix I.1). Together, these results provide evidence of statistical reliability and cross-cohort generalization, while larger-scale validation remains desirable.

AuthorsOpenReview ↗

Response to Reviewer Vtet — Part 2/3: W3, Q1–Q2— Demographic Robustness, Visual Backbones, and Error Attribution

W3. Demographic Robustness

We evaluated subgroup performance using all RDFace test cases with available demographic annotations (dataset description in Section 4.1; demographic analysis in Appendix K.4, Table K.2). Because RDFace metadata are incomplete, the available sample sizes differ across attributes.

Table R7. Top-1 accuracy (%) across available RDFace demographic subgroups.

SubgroupnVisClsStrongest VLMRareVA
Sex
Male4916.3320.4126.53
Female4235.7140.4842.86
Age
0–31723.5317.6535.29
4–7812.5012.5012.50
8–11825.0025.0025.00
12+714.2928.5728.57
Geographic region
Africa20.000.000.00
Americas1833.3327.7833.33
Asia40.000.000.00
Europe922.2233.3344.44

RareVA matches or exceeds both baselines across all evaluated subgroups, with improvements over VisCls of +10.20 pp for males and +7.15 pp for females. As complementary evidence, RareVA also generalizes to the independent GMDB cohort (n=661), retaining an improvement over the strongest VLM baseline (12.25% vs. 10.74%; Section 4.5, Table 4) despite differences in data sources and disease composition.

These subgroup comparisons provide preliminary evidence across available demographic annotations. The independent GMDB results offer complementary evidence of cross-cohort generalization, although neither evaluation establishes demographic fairness.


Q1. Stronger Candidate-Generation Backbones

We have evaluated multiple visual backbones, including CLIP in Appendix C.1, Table C.1. We additionally tested DINOv3 (ViT-B/16) and report all backbones in Table R8 below. All backbones were fine-tuned on the same RDFace training split under a consistent supervised training and evaluation protocol.

Table R8. Candidate-generation accuracy (%) on RDFace.

BackboneTop-1Top-5Top-10
DenseNet-16925.6643.3650.44
DINOv3 (ViT-B/16)19.4632.7443.36
CLIP (ViT-B/32)17.7033.6340.71
Swin-T18.5829.2043.36
FaceNet17.7031.8643.36
ResNet-15210.6231.8644.25
VGG-1615.0430.0941.59

DenseNet-169 achieves the highest Top-5 recall (43.36%), exceeding DINOv3 by 10.62 pp and CLIP by 9.73 pp. Consistent with the original RDFace benchmark (Feng et al, 2026), we adopted DenseNet-169 as the visual candidate-generation backbone rather than selecting it retrospectively based on downstream RareVA performance. We then retain it consistently across downstream baselines to isolate the effect of candidate-conditioned verification.


Q2. Error Breakdown Across Pipeline Stages

We thank the reviewer for this important analysis. We traced all 113 RDFace cases through the visual candidate generator, ACS, Doctor verification, and Judge adjudication, assigning each error to its earliest decisive failure stage. The results are shown in Table R9.

Table R9. Mutually exclusive stage-wise outcomes on RDFace.

StageFailure criterionCases%
Candidate generationGT absent from initial Top-56456.64
ACS selectionGT in Top-5 but not selected43.54
Doctor verificationSelected GT not correctly verified32.65
Judge adjudicationGT supported, but another candidate selected32.65
Correct predictionFinal prediction matches GT3934.51

Candidate generation accounts for 64/74 errors (86.49%), identifying upstream candidate recall as the dominant bottleneck. Among the 49 GT-covered cases, RareVA correctly prioritizes 39 (79.59%), with the remaining 10 errors attributable to ACS (4), Doctor verification (3), and Judge adjudication (3).

These findings identify improving visual candidate coverage as the primary direction for further gains, while demonstrating the effectiveness of RareVA's downstream candidate-conditioned verification when the correct hypothesis is available. This analysis is consistent with the component ablations in Section 4.8 (Table 5).

AuthorsOpenReview ↗

Response to Reviewer Vtet — Part 3/3: Q3–Q5 — Verification Budget, KG Coverage, and Clinical Scope

Q3. Why Does Performance Decrease at K=5?

To investigate the decrease from 35.40% at K=4 to 28.32% at K=5 (Section 4.7, Figure 6), we compared predictions from both settings on the same 113 test cases in Table R10.

Table R10. Paired prediction transitions between K=4 and K=5.

Prediction outcomeK=4 correctK=4 incorrect
K=5 correct302
K=5 incorrect1071

Increasing K produces 10 regressions but only 2 recoveries, a net loss of eight correct predictions. In 7/10 regression cases, the Judge selects the newly introduced fifth candidate instead of the previously correct hypothesis.

Thus, additional verification can introduce competing disease hypotheses that displace correct predictions during adjudication. This is consistent with the phenotype-ambiguity analysis (Section 4.6, Figure 5) and motivates ACS's adaptive verification budget rather than always evaluating more candidates.


Q4. Performance Under Sparse KG Coverage

We evaluated performance across diseases with different KG annotation densities. The KG includes 4,259 disease–phenotype associations as mentioned in Appendix F. The 54 KG-covered test diseases were grouped by annotation count into High (top 30%), Medium (middle 40%), and Low (bottom 30%) coverage.

Table R11. Top-1 accuracy (%) by ground-truth disease KG annotation density.

KG coverage# ImagesVisClsRareVAGain (pp)
High2630.7738.46+7.69
Medium4721.2831.91+10.64
Low3729.7337.84+8.11
Absent (GT outside Top-5 in all 3 cases)30.000.00—

Note: All three cases without GT-disease KG annotations also have their GT outside the initial Top-5 candidates; these outcomes therefore cannot isolate downstream fallback performance.

RareVA improves over VisCls across High (+7.69 pp), Medium (+10.64 pp), and Low (+8.11 pp) coverage groups. Furthermore, Appendix G.5 reports 33.33% vs. 35.14% Top-1 accuracy for cases with partial versus complete candidate-level KG coverage. These findings indicate that RareVA's observed gains are not restricted to richly annotated diseases.


Q5. Is a Facial Image Alone Sufficient?

We appreciate the reviewer's concern regarding the sufficiency of facial images for rare-disease diagnosis. RareVA is designed for early-stage rare-disease candidate prioritization, rather than definitive clinical diagnosis, particularly when comprehensive clinical records or molecular evidence are not yet available. Facial dysmorphology provides valuable early diagnostic clues, as demonstrated by DeepGestalt (Gurovich et al., 2019) and GestaltMatcher (Hsieh et al., 2022), supporting the clinical relevance of image-based candidate prioritization.

Importantly, we do not assume that facial images alone are sufficient for definitive diagnosis. RareVA integrates visual candidate generation, phenotype extraction, and KG-grounded verification to prioritize disease hypotheses from facial evidence (Section 3; Appendices G–H). As illustrated by our failure analysis (Appendix J.2), some disorders lack sufficiently distinctive facial manifestations for reliable visual differentiation. Therefore, RareVA is intended to complement, not replace, clinical assessment and genetic testing.

Reviewer_iDxkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RareVA, an agent-based system for rare disease screening from facial images. Rather than predicting a disease directly from an image, RareVA first uses a visual classifier to select a few possible diseases. It then uses a vision-language model to describe facial features and compares each candidate with known facial features of rare diseases. The system also introduces an adaptive budget strategy (ACS), which uses more resources for difficult cases and fewer for easy ones. This helps reduce token costs while maintaining accuracy. Experiments on two datasets, RDFace and GMDB, show that RareVA outperforms many baseline methods. However, its performance depends heavily on the quality of these candidates.

Critical strengths
  1. It shifts from direct disease prediction to candidate‑by‑candidate verification, making model decisions easier to trace and understand.
  2. The ACS adaptive strategy lowers computing overhead, spending resources mainly on hard test samples without big accuracy drops.
Critical weaknesses
  • The entire RareVA pipeline is only effective when the actual disease is present in the Top-K candidate set output by the visual classifier VisCls. On the RDFace test set, the recall rate of the Top-5 candidates of VisCls is only 43.36%. If the actual disease is not in the candidate list, the downstream multi-agent verification will completely fail to correct this error.
  • The rate of false positives in phenotype extraction still remains as high as 45.14%. And the output of the phenotype extraction module contains noise, and errors will be passed on to the downstream inference stage. The authors should discuss how to solve this problem.
  • Around 34.5% of test samples trigger the fallback mechanism, since some candidate diseases lack sufficient facial‑phenotype entries in the HPO knowledge graph. With insufficient KG evidence, the Doctor agent falls back to risky direct image‑based judgement and raises error rates. This work struggles with rare diseases that have sparse annotated facial phenotypes. Therefore, the effectiveness of this work remains a cause for concern.
  • There is no experimental comparison with the existing RareAgents and DeepRare multi-agent baselines in the field.
Questions for authors
  1. Has the author tested for a higher candidate coverage? If the correct disease does not fall within the initial candidate set, can the system identify this situation and expand the search scope?
  2. How would the accuracy of RareVA change when the input contains a certain proportion of incorrect phenotypes?
5 comments
AuthorsOpenReview ↗

Response to Reviewer iDxk — Part 1/3: Summary and W1/Q1 — Limited Candidate Coverage and Search Expansion

Dear Reviewer iDxk,

Thank you for your thoughtful and constructive feedback. To address your concerns, we conducted additional experiments on candidate coverage and expansion detection (W1/Q1, Tables R1–R2), robustness to phenotype noise (W2/Q2, Table R3), KG coverage and fallback reliability (W3, Table R4), and comparisons with RareAgents and DeepRare (W4, Table R5). Notably, expanding the candidate pool improves RareVA's Top-1 accuracy from 34.51% to 41.59%, while the noise-injection experiments demonstrate consistent robustness across all evaluated perturbation settings. We also clarify the scope of KG fallback and provide additional comparisons with existing agentic rare-disease diagnostic systems.

We have updated the revised manuscript to incorporate these clarifications and additional analyses. We hope these results help address your concerns. . Our detailed, point-by-point responses are organized into three parts, beginning with W1/Q1 below.


W1/Q1. Limited Candidate Coverage and Search Expansion

(1) Higher candidate coverage. We expanded the initial candidate pool (MM) from �0��0� to �1��1�, keeping the verification architecture unchanged and retraining ACS using validation-only rollouts. As shown in Table R1, candidate recall increases from 43.36% to 50.44%, while RareVA with ACS (�2�)(�2�) improves from 34.51% to 41.59%. Notably, verifying all ten candidates achieves only 26.55%, compared with 39.82% at fixed �3��3�, indicating that broader candidate coverage benefits from selective rather than exhaustive verification.

Table R1. Effect of candidate pool size and verification budget on RDFace.

MethodM=5 (%)M=10 (%)
VisCls Top-M recall43.3650.44
Strongest VLM (multi-round KG-RAG)29.2030.09
Fixed K=333.6337.17
Fixed K=435.4039.82
Fixed K=528.3238.94
Fixed K=10—26.55
ACS (K≤5)34.5141.59
ACS (K≤10)—39.82

(2) Identifying inadequate candidate pools. We further examined whether RareVA's existing Judge score can indicate when candidate expansion is needed. As defined in Section 3.3.3 and Appendix H (Prompts 7–8 in Appendix A.1), the Judge aggregates Doctor MATCH/NO_MATCH decisions and evidence-grounded rationales, scoring candidate alignment, specificity/prior support, and consistency on a 0–1 scale. We use an exploratory threshold of τ=0.5\tau=0.5, the midpoint of the Judge's predefined [0,1][0,1] scoring rubric, to flag low-confidence candidate pools. This threshold provides a simple, interpretable operating point rather than a calibrated probability cutoff or an optimized decision boundary. With τ=0.5\tau=0.5, we flag cases with lower Judge scores for potential expansion, without modifying the original Doctor–Judge architecture.

Table R2. Judge-score-based candidate inadequacy detection on RDFace (τ=0.5).

Judge decisionGT rank 1–5GT rank 6–10GT rank 11–30GT rank 31–103
Flag for expansion672232
No expansion flag43120
Total4982432

As shown in the Table R2’s retrospective analysis, thresholding the existing Judge score flags 61/64 (95.31%) cases with GT outside Top-5, with 87.76% specificity (43/49) and 91.04% precision (61/67). Among the 32 cases with GT ranked 6–30, 29 (90.63%) are flagged for expansion (Appendix C.1, Table C.1), suggesting that existing verification feedback can help identify cases requiring a broader candidate search.

(3) Scope and limitation. These experiments demonstrate that expanding the proposal pool improves end-to-end performance and that the existing Judge score provides an informative candidate-inadequacy signal. The current framework does not yet execute automatic feedback-triggered candidate expansion, and the Table R2 evaluates expansion decisions retrospectively rather than measuring the accuracy of a dynamically expanded pipeline. We have clarified this distinction in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer iDxk — Part 2/3: W2/Q2 — Robustness to Noisy Phenotype Extraction

W2/Q2. Robustness to Noisy Phenotype Extraction

Noise mitigation in the original framework. RareVA incorporates three independent phenotype extractions per image and evaluates multiple fusion strategies to reduce variability in VLM-generated phenotype evidence (Section 3.2; Appendix D). The selected consensus-based strategy aggregates repeated extractions rather than relying on a single potentially noisy output.

Additional robustness analysis. To directly evaluate sensitivity to incorrect phenotype evidence, we conducted a controlled noise-injection study using anatomical regions derived from HPO ontology parent–child relationships. We evaluated three perturbation types: (1) random insertion, sampling facial HPO distractors; (2) cross-region insertion, sampling distractors outside the anchor phenotype's anatomical branch; and (3) same-region insertion, sampling non-equivalent distractors within the same branch. Each perturbation was evaluated at 5%, 15%, and 25% insertion budgets relative to the original predicted phenotype-set size. All methods received identical corrupted phenotype sets. We report conditional Top-1 accuracy on RDFace cases where the ground-truth disease appears in the VisCls Top-5 candidates, isolating downstream verification from candidate-generation failures. The results are shown in Table R3.

Table R3. Conditional Top-1 accuracy (%) under increasing phenotype noise on RDFace (GT ∈ Top-5).

Phenotype conditionNoise levelSingle VLM + PhenoStrongest VLMRareVA (Ours)
Original0%48.9867.3579.59
Random insertion5%44.9065.3179.59
15%44.9061.2277.55
25%44.9061.2275.51
Cross-region insertion5%44.9065.3179.59
15%42.8663.2777.55
25%42.8661.2277.55
Same-region insertion5%42.8663.2777.55
15%40.8257.1475.51
25%38.7855.1073.47

RareVA achieves the highest accuracy across all nine perturbation settings, retaining 73.47–79.59% conditional Top-1 accuracy. At the highest noise level (25%), its accuracy decreases by 2.04–6.12 percentage points (pp) relative to the original setting, compared with 6.12–12.24 pp for the strongest VLM baseline and 4.08–10.20 pp for Single VLM + Pheno. Notably, same-region insertion produces the largest degradation for RareVA (6.12 pp), compared with random insertion (4.08 pp) and cross-region insertion (2.04 pp), suggesting that anatomically related distractors are more challenging. Despite these perturbations, RareVA maintains a 14.29–18.37 pp advantage over the strongest VLM baseline at 25% noise, compared with 12.24 pp under the original condition.

Together, the multi-extraction consensus mechanisms reduce reliance on individual VLM extraction outputs, while the controlled noise-injection experiments show that RareVA maintains higher accuracy and a smaller worst-case degradation than both VLM baselines under additive spurious phenotype evidence. These findings support comparative robustness to phenotype insertions, although they do not establish robustness to missing or systematically mislabeled phenotypes.

AuthorsOpenReview ↗

Response to Reviewer iDxk — Part 3/3: W3–W4 — Incomplete KG Coverage and Fallback Reliability and Comparison with RareAgents and DeepRare

W3. Incomplete KG Coverage and Fallback Reliability

We thank the reviewer for raising the concern regarding incomplete disease–phenotype knowledge. We clarify below the scope of fallback reasoning and its impact on diagnostic performance.

(1) Clarification of fallback scope. As reported in Appendix G.5, 34.5% (39/113) of test cases contain at least one candidate lacking KG support in the initial candidate pool. Importantly, KG missingness is candidate-specific rather than case-wide. Across all initial candidates, only 7.43% (42/565) of candidate entries lack KG support, and no case has all five candidates without KG evidence (0/113). During verification, image-based fallback is invoked only for an executed Doctor whose iterative KG retrieval yields insufficient evidence (Appendix G.1–G.3), while other Doctors with sufficient KG evidence continue KG-grounded verification. Thus, the 34.5% case-level missingness statistic does not imply that 34.5% of cases rely exclusively on image-based reasoning.

(2) Robustness to incomplete KG coverage. Also as reported in Appendix G.5, RareVA achieves 33.3% Top-1 accuracy on cases with incomplete candidate-level KG coverage, compared with 35.1% on fully covered cases. We further stratified test cases by ground-truth disease KG annotation density, ranking diseases by the number of associated HPO phenotypes and dividing KG-covered diseases into High (top 30%), Medium (middle 40%), and Low (bottom 30%) coverage groups, as shown in Table R4 below.

Table R4. Top-1 accuracy across KG coverage groups.

KG coverage# ImagesVisCls (%)RareVA (%)Δ (pp)
High2630.7738.46+7.69
Medium4721.2831.91+10.64
Low3729.7337.84+8.11
Absent (GT outside Top-5 in all cases)30.000.000.00

Note: All three cases in the Absent group have their ground-truth diseases outside the initial Top-5 proposals. Their failures therefore arise at candidate generation and do not evaluate downstream fallback reliability.

RareVA improves over VisCls across high-, medium-, and low-coverage diseases (+7.69, +10.64, and +8.11 pp, respectively). These results indicate that RareVA's performance gains are not restricted to richly annotated diseases, while the contribution of fallback reasoning under entirely absent KG annotations cannot be isolated from the available cases.


W4. Comparison with RareAgents and DeepRare

We thank the reviewer for suggesting RareAgents and DeepRare. Both are relevant agent-based rare-disease diagnostic systems, but their evaluation protocols differ from RareVA's image-first, candidate-conditioned disease prioritization setting.

RareAgents (published results). RareAgents reports 55.89% Hit@1 and 68.67% Hit@3 on RareBench-Public using Llama-3.1-70B (Chen et al., AAAI 2026). Its evaluation relies on clinical text with patient-level symptoms extracted from EHRs, whereas RareVA operates on facial images without clinical records. These results are reported for completeness but are not directly comparable due to differences in input modality, benchmark, and diagnostic setting. Furthermore, although RareAgents provides a public repository, the available release lacks a complete executable multi-agent inference and evaluation pipeline for reproducing its results on RDFace.

DeepRare (adapted evaluation). DeepRare is designed for rare disease diagnosis using text-based clinical evidence and structured phenotypes, whereas RareVA performs candidate prioritization directly from facial images. To provide an empirical reference under our image-first setting, we additionally evaluated DeepRare on the same RDFace test cases, supplying only HPO phenotypes predicted by RareVA's visual phenotype extraction module, without ground-truth phenotypes or clinical records.

Table R5. Adapted evaluation on RDFace.

MethodInput evidenceCorrect / TotalTop-1 Acc. (%)
DeepRare (adapted)Predicted image-derived HPOs8/1137.08
RareVA (Ours)Facial image + predicted HPOs39/11334.51

Note: DeepRare originally operates on text-based clinical evidence, while RareVA is designed for image-first candidate prioritization. This adapted evaluation uses image-derived HPOs as DeepRare's input and does not reproduce its original clinical diagnostic setting.

Under this adapted protocol, RareVA achieves 34.51% Top-1 accuracy, compared with 7.08% for DeepRare. However, this difference should not be interpreted as a direct comparison of diagnostic capability, given the distinct input modalities, available evidence, and original task settings. Rather, the results provide a cross-paradigm reference for the image-first setting. We have incorporated both comparisons and clarified their respective evaluation protocols in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.