Neural Evidence Transport: Reusing Brain Decoders Across Speech Tokenizers

Speech tokenizers define different vocabularies for the same audio, preventing a brain decoder trained on one vocabulary from directly serving another. We introduce Neural Evidence Transport to reuse a participant’s decoder with a frozen target model. The method divides source probabilities by reference token frequencies and averages the resulting ratios through a soft bridge estimated from ordinary audio. This transfers support relative to baseline, avoiding repeated frequency weighting when updating the target prior. We prove that transport preserves neutral evidence and derive an exact identity for the loss of chi-square contrast. Fusion weights depend on the available target-token context, are selected on development participants, and remain fixed for confirmation. On magnetoencephalography (MEG) data from 11 held-out LibriBrain100 participants, transport reduces target-token negative log-likelihood relative to the prior and single-weight fusion in both HuBERT–Wav2Vec2 directions. It outperforms matched target-space decoders for Wav2Vec2 and retains 97% of Ridge’s improvement over the prior for HuBERT. In an independent electroencephalography (EEG) study with 14 held-out participants, transport achieves 60.6% attended-speaker accuracy over 12 seconds (95% confidence interval: 56.4–65.6%), retaining 81% of the target-space Ridge gain over chance. Timing and source-label controls link these gains to aligned neural observations and vocabulary correspondence. These results demonstrate decoder reuse without an additional participant-specific target-space neural fit.