- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies a representational mismatch in brain visual decoding: features that are good for retrieval need not be compatible with a frozen pretrained generator. BHA first aligns a brain encoder to a fused target over multiple visual foundation models (Stage 1), then freezes the backbone and attaches task-specific readouts (Stage 2): teacher-specific Group MoE readouts whose matching scores are composed with the shared score via learnable gating for retrieval, and separate semantic/spatial adapters that condition a frozen diffusion generator in its native feature spaces for reconstruction. Two linear propositions motivate the design: complementary teachers reduce alignment estimation error, while adapting targets toward brain-predictability induces reconstruction discrepancy.
Validation on public datasets (THINGS-EEG1/EEG2, THINGS-MEG) shows within-subject retrieval gains (THINGS-EEG2 CSLS-2 Top-1: 90.95% to 93.90%) that persist with less training data, plus representation analyses showing that experts contribute opposing matching signals despite overlapping geometry. Notably, the gains do not transfer across subjects: under LOSO the gated readout underperforms the pretrained backbone (20.20 vs 22.50 Raw Top-1).
The retrieval-side score composition and the accompanying analyses are a real conceptual contribution, while the reconstruction branch is closer to existing fused-alignment pipelines; the honesty about negative results is a strength, but the method is best described as a well-motivated recombination rather than a fundamentally new mechanism.
Critical strengths
-
The retrieval-side design is a genuine conceptual advance over single-space alignment. BHA is, to my knowledge, the first brain decoding work that retains teacher-specific prediction spaces and composes their matching scores with the shared score. The supporting empirical finding experts assign opposing signed contributions to the same image regions despite near-identical embedding geometry (Fig. 4) is a non-obvious result that substantiates the claim that shared geometry does not imply interchangeable matching functions, and it is supported by both occlusion attribution and CLIP-Discept concept analysis.
-
The two propositions precisely delimit when complementary teachers help (estimation error) and when adaptation hurts (generator compatibility); the authors expose negative results that weaken their own method (LOSO degradation of the gated readout: 20.20 vs 22.50 Raw Top-1; adaptation lowering CLIP identification 0.754 to 0.706), which makes the positive claims more credible than the norm in this area.
Critical weaknesses
-
The shared stem + teacher-specific readouts + router/gating structure follows SAK (ICLR 2025); the fused-target alignment with a frozen SDXL/IP-Adapter generator follows MBP (2026) and ENIGMA (2026); the reconstruction-side semantic/spatial adapters are a repackaging of the same conditioning pipeline with native spaces instead of a learned fusion prior. The retrieval score composition and joint brain-visual adaptation are new, but they are incremental rather than mechanism-level innovations, and the reconstruction branch's delta over MBP is an engineering choice whose benefit is not isolated (no head-to-head against an MBP-style fusion prior under identical teachers and backbone).
-
The empirical gains are within-subject only and modest in magnitude. The main improvement is CSLS-2 Top-1 from 90.95% to 93.90% on THINGS-EEG2; under LOSO the proposed gating is worse than the frozen pretrained backbone, so the core mechanism does not transfer across subjects, which limits the significance for a field where cross-subject generalization is a stated goal. Reconstruction gains are inconsistent ("richer pretraining does not improve every reconstruction metric"), and the two propositions hold only for linear models, so their connection to the nonlinear pipeline is motivational rather than guaranteed.
Questions for authors
- Since the gated readout underperforms the pretrained backbone under LOSO (Table 3), is the within-subject gain partly attributable to subject-specific overfitting in the gating/routing parameters? Would per-subject calibration-free gating, or sharing readouts across subjects, recover LOSO performance?
- The reconstruction branch differs from MBP mainly by replacing the learned fusion prior with separate semantic and spatial adapters to native generator spaces. Can the authors provide an isolation experiment in same teachers, same backbone, same training budget, whicn can compare adapter-based native-space conditioning against an MBP-style fusion prior?
2 comments
Thank you for reconsidering your score and for the constructive questions. The LOSO explanation and the fusion-prior control are now in Section 5.4 and Table A.19.
Thanks for the reply. All my issues have been resolved, so I'll raise my rating.