- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper introduces a synthetic data generation scheme for intracortical speech data that can be used to calibrate attempted speech decoders on new days with little real data. The data is generated conditioned on phoneme sequences without alignment, and a day-specific code/embedding allows for modelling of cross-session variability in the firing rate statistics. The authors evaluate the method's performance on finetuning a decoder to future days after pretraining, and find that the synthetic data helps improve performance when added on top of real data. They also include some ablations on the data generator itself.
Critical strengths
- Speech decoding is an important problem and so is few-shot calibration, so the use of synthetic data to reduce real data collection requirements is a very important problem.
Critical weaknesses
- To start with, knowing that spike band powers (SBPs) are very important for good speech decoding performance, it seems like a poor decision to not also generate SBPs here with a separate head allowing for a different objective that can work with powers rather than counts.
- The phoneme error rates, partially as a consequence of the above (but not entirely), are very high here. It's unclear exactly why this might be but the choice of amount of pretraining data (number of sessions), the specific decoder used, etc. For example in Ryoo et al. (2025) the PERs are close to or below 30% even for the spike only GRU baseline and for the POSSM model they go as low as 27%; with SBPs they go below 20%.The GRU baseline is not very far behind. It's not clear to me how there can be such a huge gap, even when moving from a full-data regime to a split pretrain + finetune session split.
- There is only one decoder evaluated and it isn't the standard, best-performing non-ensemble baseline -- the GRU used by Willett et al. 2023 that is also used in practice -- whose performance has been carefully benchmarked and allows us to compare results with a well-tuned reference. Also, only one dataset seems to be used with no results on another, e.g., the 2025 brain-to-text dataset.
- Quite importantly, there doesn't seem to be an analysis of the generated data itself: the only comparisons are made on the basis of phoneme error rates of the decoder. For a data generation paper I'd also expect to be able to see that by several metrics the generated data is realistic.
- The method, in the end, seems overly complex for marginal gain and especially when it does not allow modelling of SBPs, which are known to be very useful for speech decoding. Thus it does not make much sense from a practical use standpoint.
- A lot of useful analyses could be done such as at finetuning time, looking at scaling curves for the number of synthetic trials and how that affects performance, but this is not done, and there are just two arbitrary "medium" and "long" regimes with not much day-specific analyses and tests of the quality of data for later days that are further from the training sessions and so on.
Questions for authors
- Why not as many sentences as all real FT? Why not add synthetic on top of all real FT to see if better?
- Have the authors attempted to look at how realistic the data is and whether it can be quantified in comparison to actual data?
- How can SBPs be integrated into this architecture?
- Have the authors tried alternative decoder architecture to see if the benefits hold? Perhaps even the baseline GRU where the numbers I think were better and we have established baseline performance?
17 comments
Response to Reviewer zTqq (2/3)
W4: Lack of generated data quality analysis
We have explicitly evaluated conditional fidelity in our experiments. To assess the quality of generated data independently of the training process, we use an independently initialized Conformer-CTC evaluator trained only on real target-day sentences and never exposed to generated samples or generator optimization to decode generated trials under different conditioning conditions. Table 3 (Ablation on conditional fidelity) directly demonstrates generated data realism:
- Generated with correct conditions: PER 50.10±0.31 (baseline)
- Generated with shuffled phonemes: PER 89.37±0.63 (+39.27 points)
- Generated with mismatched day: PER 63.70±0.45 (+13.60 points)
This decomposition proves that generated counts preserve both phonetic content and day-specific neural characteristics. Shuffling phonemes destroys linguistic structure, yielding near-random performance; mismatching the day code eliminates cross-day adaptation. Additionally, Appendix Table A8 reports further ablations on unconditional generation, confirming that our conditioning mechanisms are necessary for realistic synthesis.
The Ins/Del/Sub edit decomposition in Table 3 further quantifies how conditioning errors propagate to decoder performance, providing a direct quality metric beyond end-task PER.
We also directly compare spike-count statistics between generated and real recordings. At high firing rates, the generated counts have a per-channel median Fano factor of 0.867±0.003, close to 0.872 in the real recordings (Figure A1, Appendix C). The distance between the generated and empirical Fano-rate curves is 0.012 (Table 4). These statistics assess generated spike-count variability directly, without relying on decoder PER.
W5: Method complexity
We respectfully disagree with the complexity assessment. Our method comprises three components: (1) a birth-death count bridge with analytic reverse rates, (2) a pooled Fano-rate profile with day-dependent adjustment, and (3) an alignment-free phoneme conditioner. Each addresses a specific design requirement (discrete counts, sub-Poisson variability, sequence generation).
Importantly, the auxiliary training objectives (semantic loss, day-contrastive loss, invariance loss) are discarded after training; at deployment, only the decoder remain. The deployed system adds no inference-time complexity compared to standard calibration pipelines.
W6: Lack of scaling curves and day-specific analyses
We have explicitly evaluated performance across calibration budgets in Table 6 (substitution gap vs. calibration budget), which reports results for . The substitution gap remains close to one PER percentage point across the evaluated budgets (+0.94, +0.86, and +1.11, respectively), with no monotonic trend as the budget increases. Furthermore, Tables 1 and 2 report day-by-day performance across Days 10-14 (medium horizon) and Days 15-21 (long horizon), providing the requested day-specific analyses.
Response to Reviewer zTqq (3/3)
Questions
Q1: Why not as many sentences as all real FT? Why not add synthetic on top of all real FT to see if better?
Our goal is to reduce calibration burden from several hundred sentences (hours of recording time) to 50-100 sentences. Using as many sentences as All-real FT defeats this purpose, it represents the full-calibration baseline we aim to approach with minimal real data.
Adding synthetic data on top of All-real FT provides negligible benefit. When the decoder already has several hundred target-day sentences, the real data sufficiently covers the phonetic and neural variability space. Synthetic augmentation is most valuable precisely when real calibration data is limited, which is the scenario BURST targets.
Q2: Have the authors attempted to look at how realistic the data is and whether it can be quantified in comparison to actual data?
Yes. As addressed in W4 above, Table 3 (Ablation on conditional fidelity) directly quantifies generated data realism through conditional ablations. Shuffling phonemes increases PER by +39.27 points, and mismatching the day code increases PER by +13.60 points, demonstrating that generated counts preserve both phonetic content and day-specific neural characteristics. Appendix Table A8 provides further unconditional generation ablations.
Q3: How can SBPs be integrated into this architecture?
As detailed in W1 above, we integrate SBP generation through a parallel Gaussian flow head sharing the same phoneme conditioner and day code. This adds approximately 15% parameters and improves PER by .008 on the long-horizon protocol, though the benefit primarily reflects decoder baseline improvement rather than enhanced cross-day transfer capability.
Q4: Have the authors tried alternative decoder architecture to see if the benefits hold? Perhaps even the baseline GRU where the numbers I think were better and we have established baseline performance?
As addressed in W3, the core contribution is the generation framework, which is decoder-agnostic. The benefits derive from adaptive target-day count generation rather than decoder-specific optimization. Conformer-CTC is a standard architecture for sequence decoding (Gulati et al., 2020; Feghhi et al., 2025, NeurIPS).
We have additionally evaluated BURST with the GRU decoder architecture used by Willett et al. (2023) on the long-horizon protocol:
Table Q4: GRU decoder on the long-horizon protocol, , PER (mean across Days 15-21).
| Method | Avg. PER | vs. NR-alone |
|---|---|---|
| NR-alone (100 real, GRU) | .378 | - |
| NR-alone (100 real, Conformer-CTC) | .361 | - |
| BURST (NR+NG, GRU) | .342 | -.036 |
| BURST (NR+NG, Conformer-CTC) | .337 | -.024 |
The GRU decoder achieves slightly worse absolute PER compared to Conformer-CTC (.378 versus .361 for NR-alone). However, BURST provides virtually identical benefit across both architectures: This confirms that the generation framework generalizes across decoder choices and that the cross-day adaptation benefits derive from adaptive target-day count generation rather than decoder-specific tuning.
Summary
We have clarified the experimental protocol (W2), pointed to existing analyses (W4, W6), provided SBP experiments (W1), and addressed decoder generalization (W3, Q4). The reported performance already matches state-of-the-art baselines, and the generation framework addresses the practical deployment bottleneck of calibration burden. In light of these clarifications and additional experiments, we would appreciate if the reviewer could consider raising the score.
Response to Reviewer zTqq (1/3)
We thank the reviewer for the detailed feedback. However, we respectfully note that several raised concerns are directly addressed in our manuscript. We clarify these points below.
W1: Lack of SBP generation
Our core contribution is cross-day spike-count generation addressing sub-Poisson variability that varies across days. The degree of sub-Poisson suppression changes daily due to electrode impedance, micromotion, and neural state shifts, this is the primary cross-day transfer challenge. SBP features, while beneficial for absolute decoding performance, provide limited additional value for cross-day adaptation because they do not capture the day-specific count dispersion structure.
Lightweight SBP extension. Our framework readily accommodates SBP generation through a parallel Gaussian flow head. The SBP head shares the same phoneme conditioner and day code as the count bridge, but operates on log-transformed continuous SBP targets using standard flow matching. This adds approximately 15% parameters without requiring additional pilot data. The two heads are trained jointly with a weighted multi-task loss.
Experimental validation on the long-horizon protocol. We evaluated count-only versus count+SBP generation on the long-horizon protocol (Days 1-12 pretraining, Days 15-21 testing):
Table W1: SBP generation on the long-horizon protocol, N=100, PER (mean across Days 15-21).
Adding SBP generation improves PER by .008 (.337 → .329), but this gain is modest compared to the count-based adaptation benefit (.361 → .337, Δ=.024). The SBP contribution primarily reflects the decoder's improved absolute performance with spectral features, not enhanced cross-day transfer capability. Our count-bridge already captures the day-specific variability structure; SBP provides incremental benefit through complementary frequency-domain information.
W2: High PER compared to Ryoo et al. (2025)
We respectfully clarify that our reported performance already aligns with the reviewer's referenced benchmarks. Table 2 explicitly reports the long-horizon protocol: pretraining on Days 1-12, testing on Days 15-21. The All-real FT (all sentences, lower bound) baseline achieves average PER of approximately 31%, which is consistent with the 30% suggested by reviewer (Ryoo et al. (2025)).
Table 1 reports the medium-horizon protocol (pretraining on Days 1-7, testing on Days 10-14), where All-real FT achieves approximately 42% PER due to reduced pretraining data. Our testing protocol follows realistic deployment scenarios: we train only on historical data and test on future target days, never allowing the model to peek at target-day neural distributions during pretraining. Training on all available days (including future ones) and testing on held-out sentences from those same days would artificially inflate performance by leaking distributional information.
Our performance numbers match state-of-the-art baselines under comparable cross-day transfer conditions.
W3: Single decoder architecture and dataset
Our decoder is a Conformer-CTC architecture, a standard choice for sequence decoding tasks. Conformer-CTC is well established in speech recognition (Kim et al., NeurIPS). The architecture achieves performance consistent with the GRU baseline used by Willett et al. (2023), confirming that our generation method generalizes across decoder choices.
Our core contribution is the generation framework, not decoder optimization. W2 already clarified that our performance matches state-of-the-art baselines under comparable cross-day transfer conditions.
Regarding the Brain-to-Text '25 dataset mentioned by the reviewer, the public release accompanying Card et al. (2024) provides concatenated threshold-crossing and SBP features from 256 electrodes (512 features per 20-ms bin). The official data documentation states that these features are z-scored using the preceding 20 trials; unnormalized integer threshold-crossing counts are not directly provided. BURST's birth-death bridge operates on non-negative integer counts and models their rate-dependent dispersion. The normalized features therefore cannot be used directly to train this count bridge or assess the original count Fano-rate relationship. The SBP extension described in W1 retains the integer-count branch and does not remove this requirement. Consequently, the public 2025 release is not directly suitable for evaluating BURST's count-generation formulation.