Beyond Answer Consistency: Evaluating the Stability of Explicit Clinical Reasoning Across Repeated LLM Generations

Final-answer consistency does not necessarily imply stability in the explicit reasoning produced by a large language model (LLM). This distinction is particularly important in clinical diagnosis, where repeated generations may reach the same conclusion while relying on substantially different evidence and interpretations. We study clinical reasoning stability by representing generated diagnostic reasoning as Clinical Reasoning Graphs (CRGs) and comparing their explicit support structures using a symmetric structural distance, CA-CRED. To distinguish prompt-associated changes from ordinary generation variability, we introduce a leave-one-case-out calibration procedure and a case-level calibrated Structural Reasoning Divergence score (cSiRD). Using 120 clinical cases and 720 formal generations from Qwen3-8B, we find substantial structural variability even under repeated generation of the same case. Importantly, the apparent increase in structural dispersion under Structured Reflection largely disappears after masking diagnostic conclusion identity: the Baseline–Structured within-condition gap decreases from 0.0661 to approximately 0.0020. Likewise, the strong association between conclusion-aware cSiRD and answer changes (AUC = 0.888) is markedly reduced under support-only calibration (AUC = 0.599), indicating that final diagnosis identity accounts for a substantial portion of the original separation. Nevertheless, semantically consistent diagnostic answers can still coexist with highly divergent support structures. Finally, in an exploratory multi-level analysis, support-only similarity follows a consistent hierarchy across cases, with same-case similarity (0.7280) exceeding same-diagnosis similarity (0.7005), which in turn exceeds different-diagnosis similarity (0.6952). Together, these results suggest that answer consistency, conclusion variation, and support-structure stability capture distinct aspects of observable clinical reasoning behavior.