acceptodds
Under review as a conference paper at ICLR 2027

MIRROR: Iterative Neuro-Symbolic Reasoning for Human-Like Clinical Decision Making

Abstract

Effective clinical decision-making often unfolds as an iterative loop: attention is first allocated to a subset of patient symptoms and findings, symbolic medical reasoning is applied, and the results of reasoning guide subsequent clinical inquiries and inference. We present MIRROR (Multi-Iterative Reasoning with Rules and Observation Refinement), a trainable neuro-symbolic architecture that explicitly models this loopy perception–reasoning dynamic in medical diagnosis. MIRROR alternates between (i) an adaptive perception module that selectively attends to clinical features and patient state based on evolving diagnostic hypotheses, and (ii) a differentiable reasoning module that evaluates symbolic medical rules and clinical predicates in a learned embedding space. Across multiple cycles, predicate-level feedback from reasoning drives refined attention to further patient evidence, yielding an inspectable chain of diagnostic inference. Controlled experiments on three diagnostic benchmarks quantify diagnostic accuracy, path recovery, and evidence acquisition, while cycle-by-cycle traces expose when observation, implication, diagnosis, and knowledge-region routing change. MIRROR therefore provides an explicit computational model of iterative clinical decision making.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neuro-symbolic, perception-reasoning-loop, clinician-like decision making

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7wRCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MIRROR, a neuro-symbolic architecture that formulates clinical decision-making as an iterative perception–reasoning process. MIRROR employs sparse attention to determine the next clinical inquiry and executes symbolic Horn clauses through a Straight-Through Estimator (STE) combined with differentiable SoftMin and soft-OR operations. It maintains a signed state matrix to represent accumulated evidence and evolving diagnostic hypotheses across reasoning iterations, with an optional VQ-based knowledge-region module for conditioning rule reliabilities. Experiments on three medical datasets show that MIRROR achieves competitive diagnostic accuracy while providing explicit traces of the cognitive inquiry process underlying its decisions.

Critical strengths
  1. The full patient record ablation is highly compelling. Table 3 shows that providing all available evidence at t=0t=0 can actually degrade performance; for example, DiReCT drops from 50.46 to 38.43. This is a strong empirical observation. It indicates that the iterative perception-reasoning loop is not merely a conceptual addition, but a necessary mechanism for filtering noisy evidence and approximating the sequential nature of clinical inquiry.

  2. The integration of discrete inquiry actions with differentiable logic is also well executed. The paper addresses a genuine architectural challenge: how to propagate diagnosis-level supervision through discrete inquiry selection. The Straight-Through Estimator, or STE, in Eq. 7 provides a practical bridge between the diagnosis loss and the sparse attention-based inquiry scorer. Meanwhile, the SoftMin and soft-OR relaxations in Eqs. 10–12 make rule execution differentiable. As a result, symbolic forward chaining is not simply attached to a neural model as a post-hoc component; it is incorporated into the trainable computation graph.

Critical weaknesses
  1. The VQ-based “Knowledge Regions” module introduces substantial complexity, but its empirical benefit appears limited and may come at the cost of reasoning stability. The stated motivation is to simulate clinicians with different clinical experiences. However, Table 7 shows that the simplest setting, H=1H=1, which removes the VQ mechanism, achieves 71.69% accuracy, while the default setting H=2H=2 reaches 72.46%. This is an improvement of less than one percentage point. More importantly, increasing the number of regions to H=4H=4 reduces test accuracy to 71.11% and substantially destabilizes the reasoning trace, with State F1 dropping to 84.26 and a large standard deviation of ±6.18\pm 6.18. Given the additional architectural overhead introduced by the VQ-VAE codebook in Eqs. 14–15, as well as the added hyperparameter sensitivity, the current evidence does not clearly justify the necessity of this module.

  2. Training stability under STE is another significant concern. Although the end-to-end formulation is conceptually appealing, the DiReCT results suggest notable instability. For example, Table 3 reports a standard deviation of ±9.25\pm 9.25 for the “No implication” ablation on DiReCT, while the hyperparameter sensitivity analysis in Table 9 shows variations as large as ±6.85\pm 6.85. These results suggest that the surrogate gradients induced by discrete query selection may have high variance, making the model sensitive to initialization, rule-bank coverage, or other training details.

Questions for authors

1.Given that H=1H=1 and H=2H=2 yield nearly identical test accuracy in Table 7, and that H=4H=4 leads to a large drop in State F1 with high variance ±6.18\pm 6.18, what is the strict justification for the VQ module? At present, the VQ routing appears to introduce instability in the reasoning trace without clearly improving final diagnosis accuracy. Could the authors provide codebook utilization statistics, such as assignment perplexity or cluster occupancy, to show that the model learns distinct “doctor profiles” rather than using the codebook as a noisy regularizer?

  1. How do the authors mitigate the high variance observed in STE-based training? The reported standard deviations on DiReCT are substantial, including ±6.85\pm 6.85 in the hyperparameter sweeps and ±9.25\pm 9.25 in the Table 3 ablations. Have the authors considered alternatives such as REINFORCE with a learned baseline, Gumbel-Softmax relaxation, or a curriculum over query length to stabilize discrete action learning?

  2. The rule bank appears to be strictly frozen. How does the model behave when a patient presents symptoms that trigger no rules, or when activated rules are mutually conflicting? Since CtC_t is updated through rule inference in Eq. 13, missing rules could leave the state matrix EE close to a zero-information state, which may cause the attention mechanism to fail when selecting the next useful inquiry. How does the model handle out-of-distribution cases or rule-coverage failures at inference time?

3 comments

We warmly thank the reviewer for the positive assessment and the constructive questions.


Weakness 1 / Question 1

Q: Given the small difference between H=1H=1 and H=2H=2 and the instability at H=4H=4, what justifies the VQ knowledge-region module?

A: The VQ knowledge-region module is motivated by behavioral modeling of bounded, heterogeneous human knowledge rather than solely maximizing aggregate classification accuracy on unlabelled encounters. Its empirical behavior is explained directly by clinician-profile identifiability without doctor IDs:

  • (1) Why real-data accuracy gains are modest: In theory, each clinician ii relies on an individualized rule-weight vector wi∈[0,1]Rw_i \in [0,1]^R. However, none of the three public datasets records clinician identifiers (doctor IDs), so MIRROR can only learn HH population-level latent rule-use profiles WK=[w(1),…,w(H)]W_{\mathrm{K}} = [w^{(1)}, \dots, w^{(H)}] without identity supervision. When patient features alone do not reveal which physician treated the case, outcome accuracy gap is small between H=2H=2 from H=1H=1. Our design aims to account for factors that affect human decision making, so the small accuracy gains from knowledge regions do not mean that knowledge limitations are unimportant.
  • (2) Why H=4H=4 destabilizes State F1F_1: Without clinician-ID supervision to anchor clusters, setting HH too large over-partitions finite trajectories across redundant codewords. This increases cross-seed routing variance (destabilizing intermediate State F1F_1), or causes extra codes to latch onto disease categories rather than reasoning styles. Thus, we set the default of H=2H=2.

Controlled synthetic results: To verify that the VQ module learns rule profiles rather than acting as a noisy regularizer, we simulated clinicians drawn from H⋆H^\star ground-truth rule profiles and trained MIRROR without doctor IDs: When case context carries signal about clinician type, the VQ module recovers the latent clinician profiles without ID supervision (ARI 0.720.72 at H⋆=2H^\star=2, 0.740.74 at H⋆=3H^\star=3) and boosts accuracy by +4.5+4.5 and +6.2+6.2 points over H=1H=1, closely approaching an ID-oracle. (Please see revised draft, Section 4.3)

Q: Could the authors provide codebook utilization statistics, such as assignment perplexity or cluster occupancy, to show that the model learns distinct profiles rather than collapsing?

A: Thanks for your suggestion. We add three metrics: assignment perplexity, active-code count, and diagnosis NMI. Under H=2H=2, perplexity is the effective number of codes used (11 means complete collapse to one code; 22 means balanced use of both codes); Active-code count tests whether either code is unused; Diagnosis NMI measures whether routing merely reproduces the disease label (lower is better for ruling out trivial disease-based routing). Results across five seeds (see revised draft, Section 4.3 and Table 4):

DatasetAssignment perplexityActive codesDiagnosis NMI
MZ-41.97±0.031.97 \pm 0.032/22/20.16±0.100.16 \pm 0.10
DDXPlus1.47±0.271.47 \pm 0.272/22/20.00±0.000.00 \pm 0.00
DiReCT1.87±0.091.87 \pm 0.092/22/20.24±0.040.24 \pm 0.04

Both profiles are active on every dataset and seed. MZ-4 and DiReCT use them nearly evenly; DDXPlus is more imbalanced but does not collapse to one code. The low diagnosis NMI (0.000.00–0.240.24) further shows that the profiles are not simply copies of disease labels. Together, these results show that the compact H=2H=2 codebook is used non-trivially.

Weakness 2 / Question 2

Q: How do the authors mitigate the high variance observed in STE-based training? The reported standard deviations on DiReCT are substantial...

A: MZ-4 and DDXPlus also use STE, yet their main-result standard deviations are small (0.290.29 and 0.800.80 percentage points, respectively). This suggests that STE does not invariably cause unstable training. We therefore consider DiReCT's small sample size and severe class sparsity (364364 training notes over 6262 diagnoses) likely major contributors to its variability. Its test set contains only 7272 cases, so a single case changes accuracy by 1.391.39 percentage points.

On MZ-4 and DiReCT, inquiry receives two complementary training signals: diagnosis loss through STE and direct supervision from the differentiable process loss. These signals can be used jointly. DDXPlus has no process supervision (λproc=0\lambda_{\mathrm{proc}}=0) and uses the diagnosis-to-inquiry STE signal alone.

Q: Have the authors considered alternatives such as REINFORCE with a learned baseline, Gumbel-Softmax relaxation, or a curriculum over query length to stabilize discrete action learning?

A: We appreciate your suggestion and we implement all three discrete-action learning alternatives across five seeds: (1) Gumbel-Softmax (straight-through), (2) Plackett–Luce REINFORCE (for top-kk sampling without replacement (k=1k=1 in the current paper), using an action-preceding learned baseline), and (3) Query-Budget Curriculum (linearly annealing query horizon from 11 to TT):

The comparison is shown below (see revised draft, Appendix D, Table 14). Alternatives use five seeds.

DatasetMain modelGumbel-STREINFORCEBudget curriculum
MZ-472.46 ± 0.2971.36 ± 0.4370.67 ± 1.2971.07 ± 0.38
DDXPlus99.20 ± 0.8095.84 ± 2.6083.84 ± 7.2884.12 ± 8.22
DiReCT50.46 ± 2.8937.22 ± 6.6217.78 ± 4.3342.22 ± 14.08

Overall, no alternative consistently improves both accuracy and stability across datasets.

Question 3

Q: How does the model behave when a patient presents symptoms that trigger no rules?

A: We discreate the work flow. The state is never zero-information, because the chief complaint initializes E(0)E^{(0)} with observed signed facts. Attention queries come from occupied slots Qt=Ot−1∪It−1Q_t=O_{t-1}\cup I_{t-1}, so the model can always rank unread predicates from the observed facts alone. If no diagnostic rule fires, the logits are all zero, the prediction is uniform, and confidence stopping never triggers, so the model keeps asking until the budget is exhausted.

Q: How does the model handle mutually conflicting rules?

A: MIRROR handles conflicting rules through signed aggregation. For each predicate or diagnosis class, it first combines all rules supporting the positive conclusion and all rules supporting the negative conclusion using two separate weighted soft-OR operations. It then subtracts the negative support from the positive support to compute the implication value vkv_k or diagnostic logit ℓc\ell_c. For an empty rule set, we define softOR⁡(∅)=0\operatorname{softOR}(\varnothing)=0. Conflicting rules therefore offset one another and reduce the magnitude of the final score. (Please refer to Eqs. 12 and 16.)

Q: How robust is the model empirically to rule-coverage failures, conflicts, or rule-bank perturbations?

A: Thanks for your question. We add some statistic results to further explain the questions (see revised draft, Appendix D, “Rule coverage, conflict, and perturbation,” and Table 15).

  • (1) Unfired & conflicting rule statistics: Zero diagnostic rules fire in only 0.6–0.9%0.6\text{–}0.9\% of MZ-4 cases, 0%0\% of DDXPlus cases, and 0–4.2%0\text{–}4.2\% of DiReCT cases. On MZ-4, positive and negative rules for the same diagnosis co-fire in 60–61%60\text{–}61\% of test cases, yet accuracy on these conflicting cases (71.7–72.6%71.7\text{–}72.6\%) matches overall accuracy (72.46%72.46\%).
  • (2) Rule deletion and noise injection: We andomly deleting 10% / 30% / 50% of rules and injecting 10% random noisy rules. Mean diagnosis accuracy is shown below. Perturbations use five seeds:
DatasetFull modelDelete 10%Delete 30%Delete 50%Inject 10% noisy rules
MZ-472.4672.4671.7171.7168.4668.4668.1268.1272.0672.06
DDXPlus99.2099.2096.9296.9295.6095.6094.7694.7693.6493.64
DiReCT50.4650.4637.5037.5033.0633.0627.5027.5036.6736.67

Deleting more rules generally lowers accuracy. Injecting 10%10\% noisy rules changes MZ-4 accuracy only slightly, but lowers DDXPlus to 93.6493.64 and DiReCT to 36.6736.67. Thus, robustness to rule noise varies across datasets, with DiReCT showing the largest degradation.

Future extension to OOD rule learning: While the rule bank is frozen in this study as we focus on behavior modeling, our MIRROR is easy to learn the rule structure, which is an advantage of our nerual symbolic design. Representing predicates in a shared continuous anchor space provides a natural foundation for differentiable Horn-clause structure learning, which we list as future work.

Reviewer_8WYXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MIRROR, a trainable neuro-symbolic framework that models human-like clinical decision-making through iterative perception and reasoning. Its key contributions include adaptive attention for evidence selection, differentiable symbolic rule-based reasoning, and dynamic feedback mechanisms that refine diagnostic hypotheses and enable interpretable reasoning trajectories. The framework also learns knowledge-region profiles to capture variations in rule usage. Validation on three medical datasets demonstrates competitive diagnostic accuracy, alongside promising recovery of clinician-relevant evidence. Ablation studies, cognitive-path evaluations, and case studies further support the effectiveness of iterative reasoning, process supervision, and transparent diagnostic inference.

Critical strengths
  1. An interpretable reasoning process. MIRROR maintains an explicit state over named clinical predicates and records observations, inferred findings, and rule activations at each step, making its decision process easier to inspect.
  2. A unified end-to-end framework. The method combines adaptive evidence acquisition, differentiable rule-based reasoning, and state-dependent rule weighting within a single trainable architecture.
  3. Evaluation across diverse clinical datasets. The experiments cover three datasets with different evidence structures and supervision signals, and include baseline comparisons, ablation studies, and case studies to examine both predictive performance and intermediate reasoning behavior.
Critical weaknesses
  1. The proposed MIRROR framework appears to be primarily an integration of several existing techniques. Specifically, neuro-symbolic reasoning and forward chaining have already been explored in prior work such as NSFR, while iterative symptom inquiry and diagnosis have been investigated in DxFormer. Although combining these components into a unified perception–reasoning loop is potentially useful, the paper does not sufficiently clarify the key novelty beyond integrating existing mechanisms for clinical diagnosis.
  2. The claim of mirroring human decision-making and interpretability is somewhat overclaimed. In particular, the cognitive path recovery results in Table 2 are relatively weak on DiReCT. The author explains that some inferred predicates are absent from the physician-annotated reasoning trees. Howerver, it remains unclear whether these additional predicates represent clinically-meaningful reasoning steps, since MIRROR does not always outperform the strongest baseline.
  3. Insufficient ablation studies on reasoning-to-perception feedback. The No implication updates ablation in Table 3 does not clarify whether the gains come from inferred features improving diagnosis or from these features guiding subsequent inquiries. Since the latter is a key contribution, a more direct ablation would be to retain the implication modules while blocking inferred predicates to the attention modules. Furthermore, several ablations in Table 3 modify the information flow only at evaluation time without retraining the models. The observed performance degradation may partly result from a mismatch between training and evaluation conditions.
Questions for authors
  1. Could the author clarify the specific version and model size of Qwen used in Table 1? These details appear to be missing.
  2. Could the author clarify whether the iterative state update in Equation 13 may repeatedly accumulate support from the same evidence?
3 comments

We thank the reviewer for the thoughtful feedback and constructive suggestions, and would be happy to discuss any remaining questions.


Weakness 1

Q: What is the novelty beyond integrating NSFR and DxFormer?

A: First, from motivation, MIRROR is not intended primarily as a state-of-the-art black-box classifier; its goal is to model how a human clinician reaches a diagnosis by embedding two explicit cognitive constraints into the architecture: (i) limited knowledge, approximated via rule-reliability profiles, and (ii) limited information processing per unit time, operationalized by inspecting at most kk findings per round (k=1k=1 in the current paper) within budget TT and confidence-based stopping.

Second, from architecture, the novelty of MIRROR lies not in isolated forward chaining or sequential inquiry, but in closing the perception–reasoning loop over a single and elegant design. We have discussed differentiable forward chaining (NSFR) and sequential symptom inquiry (DxFormer). We explain the distinction clearer as follows,

  • NSFR (Static Symbolic Reasoning) executes differentiable forward chaining over a fixed, fully observed input and cannot actively decide what missing clinical evidence to acquire next.
  • DxFormer (Black-Box Sequential Inquiry) selects symptom queries from an entangled latent vector without named symbolic predicates, explicit polarity (+1/−1/0+1/-1/0), or inspectable logical rules.
  • MIRROR (Closed Symbolic Perception–Reasoning Loop): Both observed answers and rule-inferred conclusions are written into and read from the same signed state E(t)E^{(t)}. Consequently, multi-hop symbolic deductions directly enter the attention query set (Qt=Ot−1∪It−1Q_t = O_{t-1} \cup I_{t-1}) to reshape subsequent evidence acquisition, while keeping every intermediate step transparent.

Q: What empirical comparisons isolate this closed-loop contribution from the rule bank and process supervision?

A: To test this contribution, we added matched baselines and retrained ablations that isolate each factor (see Table 5):

  • Same rule bank, no learning (Hard Horn execution): Deterministic Horn execution achieves 67.25/20.00/30.56% on MZ-4 / DDXPlus / DiReCT (DDXPlus ties on all cases under unweighted OR).
  • Direct feedback isolation (No-Feedback ablation): Explicitly blocking inferred predicates from the attention module directly tests reasoning-to-perception coupling (detailed in Weakness 3 below).

Weakness 2

Q: The claim of mirroring human decision-making and interpretability is somewhat overclaimed.

A: MIRROR is designed as a computational behavioral model of clinical decision-making under cognitive constraints. Because no public clinical benchmark records ground-truth internal cognition, our objective is to generate explicit, cognitively bounded diagnostic paths and evaluate their agreement with the partial behavioral or structural annotations available across three complementary regimes:

  • DDXPlus (Outcome-only supervision): Contains no clinician trajectories; it tests whether MIRROR can learn selective sequential inquiry and latent intermediate deductions purely from diagnostic outcomes.
  • MZ-4 (Weak inquiry-behavior reference): Derived from real doctor–patient consultations; we use recorded implicit symptoms as a weak reference for observable information-seeking behavior.
  • DiReCT (Structural clinical reference): Provides physician-annotated diagnostic knowledge graphs and entailment trees, offering a structural reference of clinical premises and intermediate hypotheses rather than a temporal log of internal cognition.

Q: Cognitive path recovery results in Table 2 are relatively weak on DiReCT ... it remains unclear whether these additional inferred predicates represent clinically meaningful reasoning steps.

A: Separating queried observations (OstopO_{\mathrm{stop}}) from rule-inferred hypotheses (IstopI_{\mathrm{stop}}) clarifies both why full-state precision is low on DiReCT and what those extra predicates represent:

  • Queried findings align closely with physicians: When evaluating recorded inquiries OstopO_{\mathrm{stop}} alone against the physician reference C⋆C^\star, DiReCT path F1F_1 is 79.6279.62 (precision 82.0282.02, recall 77.3977.39). Thus, MIRROR's perception module does retrieve physician-relevant clinical evidence.
  • Why full-state precision drops: Full state C=Ostop∪IstopC = O_{\mathrm{stop}} \cup I_{\mathrm{stop}} includes all predicates activated by forward-chaining rules. Our structural audit shows that the extra nodes in Istop∖C⋆I_{\mathrm{stop}} \setminus C^\star predominantly correspond to competing differential diagnoses and related intermediate nodes in the clinical knowledge graph triggered by shared symptoms—whereas the physician's post-hoc entailment tree C⋆C^\star annotates only the final confirmed diagnosis path and omits ruled-out differentials.

Weakness 3

Q: A more direct ablation would be to retain the implication modules while blocking inferred predicates to the attention modules.

A: Thank you for this suggestion. We implemented and retrained this No-Feedback variant from scratch across five seeds (See revised draft, Section 4.4 and Table 5): symbolic implication is retained in full for state evolution and final diagnosis, but inferred predicates are masked from the inquiry attention module (restricting query rows strictly to observed findings Qt=Ot−1Q_t = O_{t-1} and zeroing inferred inputs to attention).

The complete ablation results are shown below (test diagnosis accuracy, mean ±\pm sample SD):

DatasetFull MIRRORChief-onlyFull-recordNo-implicationNo-feedbackNo-process
MZ-472.46±0.2972.46 \pm 0.2963.36±0.3963.36 \pm 0.3971.96±0.7571.96 \pm 0.7571.00±0.8471.00 \pm 0.8470.83±0.6970.83 \pm 0.6963.71±1.1763.71 \pm 1.17
DDXPlus99.20±0.8099.20 \pm 0.8066.56±1.5166.56 \pm 1.5199.12±0.5899.12 \pm 0.5894.20±5.2094.20 \pm 5.2093.64±3.1893.64 \pm 3.1899.20±0.8099.20 \pm 0.80
DiReCT50.46±2.8950.46 \pm 2.899.17±1.589.17 \pm 1.5838.33±2.8838.33 \pm 2.8821.39±3.8821.39 \pm 3.8838.33±4.6738.33 \pm 4.6717.22±2.5217.22 \pm 2.52

On DiReCT, No-Feedback reaches 38.33±4.6738.33\pm4.67, compared with 21.39±3.8821.39\pm3.88 for No-Implication. Keeping implication therefore improves diagnostic representation even when inferred predicates cannot guide inquiry. However, No-Feedback remains below full MIRROR (50.46±2.8950.46\pm2.89), supporting an additional contribution from reasoning-to-inquiry feedback. The No-Feedback versus No-Implication comparison is most pronounced on DiReCT; their means are close on MZ-4 and DDXPlus. This is consistent with the datasets’ different evidence structures. DiReCT’s diagnostic graphs connect clinical premises through intermediate hypotheses, so implication can enrich the diagnostic state even when inferred findings cannot guide subsequent inquiry. In contrast, MZ-4 and DDXPlus provide structured findings that more directly support diagnosis.

Q: Several ablations in Table 3 modify the information flow only at evaluation time without retraining the models.

A: Thank you for pointing it out. In the revision, every ablation variant is retrained from scratch under its own information-flow setting across five seeds. (See revised draft, Section 4.4 and Table 5.)

  • Chief-only: Trained and evaluated solely on initial chief complaints C0C_0.
  • Full-record: All available patient facts written at t=0t=0 without sequential inquiry, keeping rule bank and depth LL fixed.
  • No-implication (L=0L=0): Trained without symbolic forward-chaining updates.
  • No-process (λproc=0\lambda_{\mathrm{proc}}=0): Trained with outcome cross-entropy only.

The table above reports these retrained variants together with the new No-Feedback variant. The chief-only variant is below the full loop on every dataset, indicating that inquiry provides useful evidence beyond the initial facts. Full-record accuracy is close to the full loop on MZ-4 and DDXPlus, but substantially lower on DiReCT (38.3338.33 versus 50.4650.46). Removing implication has a smaller effect on MZ-4 (72.4672.46 to 71.0071.00), lowers DDXPlus from 99.2099.20 to 94.2094.20, and causes the largest drop on DiReCT (50.4650.46 to 21.3921.39). This is consistent with the different reference structures: MZ-4 findings are named slots in the patient record, whereas on DiReCT, implication supports inference over intermediate hypotheses from clinical premises. Removing process supervision lowers MZ-4 from 72.4672.46 to 63.7163.71 and DiReCT from 50.4650.46 to 17.2217.22, highlighting the value of process annotations. DDXPlus is unchanged because its full model already uses outcome supervision alone.

Question 1

Q: What Qwen version and model size were used?

A: We used Qwen2.5-7B-Instruct (7B parameters, 4-bit MLX quantization: mlx-community/Qwen2.5-7B-Instruct-4bit) for zero-shot full-record classification across all three datasets. (See revised draft, Appendix C.3, “Qwen configuration.”)

  • Prompting: Each prompt presented the candidate diagnosis list in a fixed order followed by the clinical record, with the system instruction: "You are a medical text classifier. Output exactly one complete label from the candidate labels. Do not explain and do not add punctuation or any other text."
  • Decoding: Deterministic greedy decoding (temperature = 0.0), with max new tokens set to 2424, 2424, and 3232 for MZ-4, DDXPlus, and DiReCT, respectively.

Question 2

Q: Could the author clarify whether the iterative state update in Equation 13 may repeatedly accumulate support from the same evidence?

A: Yes. Because the interpolation s←(1−∣v∣)s+vs\leftarrow(1-|v|)s+v is applied after every hop and every round, a constant support vv applied nn times yields s=1−(1−v)ns=1-(1-v)^n, which increases monotonically even if no new evidence arrives. This explains the rise of class probability during rounds with unknown replies. From a descriptive behavioral-modeling perspective, this raises a possible connection to belief reinforcement and confirmation-related biases. We therefore treat it as a possible behavioral interpretation.

  • Quantitative analysis: we retrain a variant: an idempotent update that assigns sk=vks_k=v_k for unread kk, retaining graded partial support while removing interpolation-based accumulation, so repeated identical support yields the same value. The table reports mean accuracy (%), with retrained variants using five seeds (see revised draft, Appendix D, Table 13):
DatasetOriginal modelIdempotent update
MZ-472.4671.88
DDXPlus99.2099.00
DiReCT50.4630.56

On MZ-4 and DDXPlus, the idempotent variant achieves nearly identical accuracy to the original model. On DiReCT, accuracy drops from 50.4650.46 to 30.5630.56. The result suggests that persistent accumulation contributes to diagnosis on DiReCT.

Reviewer_TR2bOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes MIRROR, an iterative neuro-symbolic model for diagnosis. It selects a patient finding, updates its internal state, applies a fixed set of rules, and uses the resulting state to guide the next inquiry. The model also learns different rule-weight profiles. Experiments evaluate diagnostic accuracy and overlap with reference findings on three datasets.

Critical strengths

The model explicitly records queried findings, inferred findings, and rule activations, rather than generating an explanation afterward. This makes it possible to inspect how the diagnosis changes and where an error might occur. The results show some promise among the rule-based methods.

Critical weaknesses
  • I am not convinced that the experiments demonstrate recovery of human reasoning. The paper repeatedly claims to reconstruct human cognitive processes and individualized knowledge regions. However, DDXPlus is synthetic and has no clinician trajectory, while the MZ-4 process target is the stored order of implicit symptoms, which is not necessarily the physician's actual reasoning order. The learned knowledge regions are also not validated against individual clinicians. Moreover, the "cognitive path" metric measures overlap between final sets of predicates, without evaluating their order, polarity, or reasoning connections. Recovering similar findings is different from recovering how a physician reached a diagnosis. These claims should be narrowed or supported by more direct evaluation.

  • Differentiable forward chaining already exists in methods such as NSFR (Shindo et al, 2021), and DxFormer already addresses sequential symptom inquiry. The contribution seems to be their integration with a signed state and adaptive rule weighting, rather than iterative reasoning itself. Although many baselines are included, they do not clearly isolate this contribution: MIRROR receives a constructed rule bank and, on two datasets, process supervision. Comparisons with methods using the same knowledge and supervision would be more informative. The graph-guided sequential baseline from the original DiReCT paper would also be relevant.

  • In Table 3, the full-record and no-implication variants are changes made at evaluation time, rather than separately trained models. Their lower performance could partly reflect a mismatch with training, rather than showing that these alternatives are inherently worse. Please retrain these variants under their respective settings. A fixed or random inquiry policy with the same query budget would also help establish whether the learned evidence-selection policy is useful.

  • I am concerned that repeated reasoning can strengthen a conclusion without additional supporting evidence. Under Equation 10, a rule can receive nonzero support even when one required premise is unknown or contradicted. Equation 13 can then repeatedly strengthen the inferred finding from that same support. In Figure 5, the correct-diagnosis probability continues increasing over several rounds where the queries return unknown. This does not necessarily mean the diagnosis is wrong, but it raises the question of whether the model is acquiring useful evidence or repeatedly reinforcing its existing assumptions.

  • The DiReCT inquiry setup seems inconsistent with the case study. Appendix B.1 says that final-diagnosis leaves are excluded from the inquiry set. However, Figure 5 includes queries for ischemic stroke and hemorrhagic stroke, and Appendix E lists these nodes as directly supporting the corresponding final diagnoses. What does querying a diagnosis mean, and where does its answer come from?

  • The experimental scope and efficiency claims need more support. DDXPlus is restricted to five conditions, with several baselines already reaching near-perfect performance. DiReCT has only 72 test cases for a 62-class diagnosis task, making class coverage and uncertainty important. Random Forest also outperforms MIRROR on the reported diagnostic accuracy measures. This does not eliminate the value of interpretability, but the paper should clearly establish the trade-off.

Questions for authors
  • Important details, including the restricted dataset settings and the meaning of process supervision, should appear in the main text. The exact Qwen model and MIRROR training settings also need to be specified.
9 comments

We thank the reviewer for the careful reading and useful feedback. We address each concern and question below.


Weakness 1

Q: I am not convinced that the experiments demonstrate recovery of human reasoning...

A: MIRROR is designed as a computational behavioral model of clinical decision-making under cognitive constraints. Because no public clinical benchmark records ground-truth internal cognition, our objective is to generate explicit, cognitively bounded diagnostic paths and evaluate their agreement with the partial behavioral or structural annotations available across three complementary regimes:

  • DDXPlus (Outcome-only supervision): Contains no clinician trajectories; it tests whether MIRROR can learn selective sequential inquiry and latent intermediate deductions purely from diagnostic outcomes.
  • MZ-4 (Weak inquiry-behavior reference): Derived from real doctor–patient consultations; we use recorded implicit symptoms as a weak reference for observable information-seeking behavior.
  • DiReCT (Structural clinical reference): Provides physician-annotated diagnostic knowledge graphs and entailment trees, offering a structural reference of clinical premises and intermediate hypotheses rather than a temporal log of internal cognition.

Q: How can the knowledge regions be interpreted without clinician identifiers?

A: Because none of the three datasets provides clinician identifiers (doctor IDs), we cannot validate learned individual profiles wiw_i against individual physicians; instead, we formulate knowledge regions as population-level profiles WK=[w(1),…,w(H)]W_{\mathrm{K}}=[w^{(1)},\dots, w^{(H)}] learned without identity supervision. MIRROR fits WKW_{\mathrm{K}} end-to-end only by diagnostic and process losses. The three datasets don't provide knowledge region ground truth for comparison, but explicitly representing heterogeneous knowledge use remains essential to behavioral modeling. Our design aims to account for factors that affect human decision making, so the small accuracy gains from knowledge regions do not mean that knowledge limitations are unimportant.

Controlled verification (please refer to revised Section 4.3 and Appendix D.1, Tables 4, 17–18): To test profile identifiability directly, we added a synthetic clinician study with known ground-truth clinician types. When the case context carries clinician-type information, MIRROR recovers the latent profiles without access to clinician IDs, achieving ARI values of 0.720.72 for H⋆=2H^\star=2 and 0.740.74 for H⋆=3H^\star=3 (Adjusted Rand Index: higher is better, with 11 indicating perfect agreement and 00 chance-level agreement), while improving accuracy by +4.5+4.5 and +6.2+6.2 percentage points over H=1H=1, respectively. We also add real-data codebook utilization diagnostics. At H=2H=2, assignment perplexity is 1.971.97 on MZ-4, 1.471.47 on DDXPlus, and 1.871.87 on DiReCT, showing that both regions are used on all three datasets. (Perplexity is the effective number of codes used, where 11 indicates complete collapse to one code and 22 indicates balanced use of both codes.)

Q: Does the cognitive-path metric evaluate order, polarity, or reasoning connections rather than only final-set overlap?

A: We originally used set overlap because multiple inquiry orders can be clinically valid when no unique ground-truth sequence exists, but we agree that set overlap alone does not establish sequential or structural fidelity. To strengthen process evaluation, we compare the learned inquiry policy with random inquiry as suggested, keeping the trained rule and diagnosis modules fixed and matching query counts for each case and round. The reported results show that MIRROR's learned inquiry improves diagnostic accuracy by about 8-30 percentage points on three dataset. The details are shown in Weakness 3 below, or please see revised Table 3.

Weakness 2

Q: What is the novelty relative to NSFR and DxFormer?

A: First, from motivation, MIRROR is not intended primarily as a state-of-the-art black-box classifier; its goal is to model how a human clinician reaches a diagnosis by embedding two explicit cognitive constraints into the architecture: (i) limited knowledge, approximated via rule-reliability profiles, and (ii) limited information processing per unit time, operationalized by inspecting at most kk findings per round (k=1k=1 in the current paper) within budget TT and confidence-based stopping.

Second, from architecture, the novelty of MIRROR lies not in isolated forward chaining or sequential inquiry, but in closing the perception–reasoning loop over a single and elegant design. We have discussed differentiable forward chaining (NSFR) and sequential symptom inquiry (DxFormer). We explain the distinction clearer as follows,

  • NSFR (Static Symbolic Reasoning) executes differentiable forward chaining over a fixed, fully observed input and cannot actively decide what missing clinical evidence to acquire next.
  • DxFormer (Black-Box Sequential Inquiry) selects symptom queries from an entangled latent vector without named symbolic predicates, explicit polarity (+1/−1/0+1/-1/0), or inspectable logical rules.
  • MIRROR (Closed Symbolic Perception–Reasoning Loop): Both observed answers and rule-inferred conclusions are written into and read from the same signed state E(t)E^{(t)}. Consequently, multi-hop symbolic deductions directly enter the attention query set (Qt=Ot−1∪It−1Q_t = O_{t-1} \cup I_{t-1}) to reshape subsequent evidence acquisition, while keeping every intermediate step transparent.

Q: Comparisons with methods using the same knowledge and supervision would be more informative. The graph-guided sequential baseline from the original DiReCT paper would also be relevant.

A: For more baselines, we compare Same rules, no learning: deterministic hard Horn execution. Diagnosis accuracy is 67.25/20.00/30.56% on MZ-4/DDXPlus/DiReCT. DDXPlus ties on every case.

Graph-guided baseline from DiReCT: We added the graph-guided sequential baseline from the original DiReCT paper, using the same Qwen model and the same 72-case test split used in our study.

  • With G : Uses the diagnostic graph structure.
  • With K: Additionally receives clinical diagnostic conditions and thresholds.
MethodCategory AccCategory F1Diagnosis AccDiagnosis F1
with G70.8362.9838.8931.46
with K70.8362.9851.3941.16
MIRROR (ours)67.59±6.4256.02±4.5250.46±2.8932.29±1.29

The graph-guided LLM baseline achieves 38.89% diagnosis accuracy under With G and 51.39% under With K. MIRROR achieves 50.46%. These results show that our explicit, trainable perception–reasoning framework achieves diagnosis accuracy comparable to an LLM augmented with external diagnostic knowledge.

Weakness 3

Q: In Table 3, the full-record and no-implication variants are changes made at evaluation time ... Please retrain these variants under their respective settings.

A: Thank you for pointing it out. In the revision, every ablation variant is retrained from scratch under its own information-flow setting across five seeds (see revised draft, Section 4.4 and Table 5).

  • Chief-only: Trained and evaluated solely on initial chief complaints C0C_0.
  • Full-record: All available patient facts written at t=0t=0 without sequential inquiry, keeping rule bank and depth LL fixed.
  • No-implication (L=0L=0): Trained without symbolic forward-chaining updates.
  • No-process (λproc=0\lambda_{\mathrm{proc}}=0): Trained with outcome cross-entropy only.

Test diagnosis accuracy (%), mean ±\pm sample standard deviation:

DatasetFull MIRRORChief-onlyFull-recordNo-implicationNo-process
MZ-472.46±0.2972.46 \pm 0.2963.36±0.3963.36 \pm 0.3971.96±0.7571.96 \pm 0.7571.00±0.8471.00 \pm 0.8463.71±1.1763.71 \pm 1.17
DDXPlus99.20±0.8099.20 \pm 0.8066.56±1.5166.56 \pm 1.5199.12±0.5899.12 \pm 0.5894.20±5.2094.20 \pm 5.2099.20±0.8099.20 \pm 0.80
DiReCT50.46±2.8950.46 \pm 2.899.17±1.589.17 \pm 1.5838.33±2.8838.33 \pm 2.8821.39±3.8821.39 \pm 3.8817.22±2.5217.22 \pm 2.52

The chief-only variant is below the full loop on every dataset, indicating that inquiry provides useful evidence beyond the initial facts. Full-record accuracy is close to the full loop on MZ-4 and DDXPlus, but substantially lower on DiReCT. Removing implication has a smaller effect on MZ-4, lowers DDXPlus from 99.2099.20 to 94.2094.20, and causes the largest drop on DiReCT (50.4650.46 to 21.3921.39). This is consistent with the different reference structures: MZ-4 findings are named slots in the patient record, whereas on DiReCT, implication supports inference over intermediate hypotheses from clinical premises. Removing process supervision lowers MZ-4 from 72.4672.46 to 63.7163.71 and DiReCT from 50.4650.46 to 17.2217.22, highlighting the value of process annotations. DDXPlus is unchanged because its full model already uses outcome supervision alone.

Q: A fixed or random inquiry policy with the same query budget would also help establish whether the learned evidence-selection policy is useful.

A: Following your suggestion, we added inquiry policy controls that keep the trained rule and diagnosis modules fixed and match the exact number of queried predicates per case and per round:

  • Random legal inquiry: Uniformly samples unread predicates.
  • Fixed frequency-ordered inquiry: Queries unread predicates in descending order of training-set prevalence (plus a reference-order teacher control).

Mean test accuracy (%) is shown below (see revised draft Table 3).

DatasetLearned inquiryRandom inquiryFrequency-ordered inquiry
MZ-472.4664.5668.72
DDXPlus99.2076.3984.44
DiReCT50.4619.2629.44

The learned policy outperforms both random inquiry and frequency policy.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Response

We thank all reviewers for their constructive feedback. Reviewers noted that MIRROR records queried findings, inferred hypotheses, and active rules explicitly rather than explaining post hoc (TR2b, 8WYX, 7wRC), and appreciated the motivation for sequential inquiry (7wRC). Below is a summary of our main revisions (all changes in the revised draft are highlighted in blue).

  1. Clarified Goal and Motivation: Behavioral Modeling Under Cognitive Limits. MIRROR is a computational model of clinical decision-making under cognitive constraints, not a SOTA classifier. We model two explicit human limits: limited knowledge and limited per-round capacity. Unlike black-box classifiers or free-text LLM outputs, every intermediate step in MIRROR can be inspected and controlled. We view this process-level modeling as a foundation for human-centric AI systems that can support, remind, or teach human decision makers.

  2. Knowledge Regions Learning: None of the datasets includes doctor IDs, MIRROR therefore learns HH population-level rule profiles instead of individual ones. Because our goal is to account for factors that shape human decisions, small accuracy gains on ID-free data don't mean knowledge limits are unimportant. Two new checks verify that the VQ module captures meaningful, non-collapsed knowledge profiles (Section 4.3, Table 4; Appendix D.1, Tables 17–18):

  • (i) Synthetic clinician study: When case context carries signal about clinician type, the VQ module recovers latent clinician profiles without ID supervision (ARI 0.72–0.74) and improves accuracy by +4.5 to +6.2 points over H=1H=1.
  • (ii) Codebook utilization: At H=2H=2, both profiles stay active on all datasets (perplexity 1.47–1.97) with low diagnosis NMI (0.00–0.24).
  1. New Experiments and Clarified Setup:
  • Retrained ablations: Retrained all ablations from scratch and added a No-Feedback variant to isolate reasoning-to-inquiry coupling (Section 4.4, Table 5).
  • Baselines & inquiry controls: Added budget-matched random/frequency inquiry (Section 4.2, Table 3), deterministic Hard Horn execution, and the DiReCT graph-guided LLM baseline (With G / With K).
  • Mechanism & stability checks: Analyzed SoftMin accumulation (Table 13), discrete-action training alternatives (Table 14), rule perturbations (Table 15), and single-thread CPU latency (<100 ms/case, Table 16) in Appendix D.
  • Experimental details: Specified dataset splits, the DiReCT no-leaf protocol, Qwen2.5-7B-Instruct settings, and all hyperparameters (Section 4.1; Appendices B.1, C.3).

We hope these revisions address all concerns and thank the reviewers for helping improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.