SLAM: A Self-Hostable Multi-Backend Reconciler for Real-Time Medical ASR

Medical automatic speech recognition (ASR) must recover safety-critical content while also applying formatting and editing operations to a clinical document. We study multi-backend ASR reconciliation, in which an LLM combines hypotheses from heterogeneous ASR systems into a single corrected and formatted output. We focus on a real-time per-chunk configuration that avoids the additional cost and state required by revisable rolling-window reconciliation. To make this configuration practical to deploy, we investigate replacing its proprietary frontier-model reconciler with a self-hostable open-weight student. We train students from the teacher's reconciliations using real clinician speech, synthetic speech, and their combination. Synthetic examples are language model generated, rendered with text-to-speech, and acoustically augmented to induce disagreement between the production ASR backends. We evaluate transcription and content fidelity on 226 held-out real clinical encounters and more complex formatting behavior on a separate 61-case targeted suite. Our best 27B student approaches the teacher, with no statistically significant difference in either word error rate (WER) or keyword error rate (KER). Synthetic supervision is competitive with real supervision at the evaluated training budget, while blended supervision matches or exceeds real-only supervision on the behavioral suite at every model size. Acoustic augmentation improves KER at every evaluated model size, with the largest observed gain for the smallest student. Self-hosting reduces measured reconciler cost by approximately $5$x in our deployment comparison, and a 35B mixture-of-experts student reduces median reconciliation latency from 1,100,ms to 466,ms. These results show that the frontier-model reconciler in real-time multi-backend clinical ASR can be replaced by a faster, lower-cost, self-hostable model, with much of the required supervision generated synthetically.