acceptodds
Withdrawn from ICLR 2027

RESHARE: REGION-STRUCTURED LEARNING FOR SHARED NEURAL VISUAL DECODING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural visual decoding holds promise for brain-computer interfaces, but collecting neural recordings paired with visual stimuli is time-consuming. Many highperforming EEG and MEG decoding methods rely on participant-specific models, while existing shared decoders remain challenged by differences in neural response distributions across participants. These differences hinder the reuse of existing recordings when only limited training data are available from a new participant. We propose ReShare, a region-structured framework for shared visual decoding from EEG and MEG. ReShare uses EEG and MEG sensor layouts to define regional groups for spatial and temporal encoding. It then calibrates the encoded responses using learned, participant-specific scaling and offsets within each region. A network shared across participants integrates these calibrated responses and aligns its outputs with intermediate visual features from a frozen CLIP ViT-L/14 encoder. Fine-tuning the pretrained decoder with 10% of a new EEG participant’s training pairs achieves 78.0% Top-1 accuracy. This exceeds the 75.0% achieved by a prior state-of-the-art participant-specific model trained on the same participant’s full training set. With one shared model per dataset, ReShare also achieves state-of-the-art 200-way brain-to-image retrieval of unseen images from participants included in training. Top-1 accuracies reach 95.70% on THINGS-EEG2 and 61.88% on THINGS-MEG. These results show that region-structured shared learning supports accurate visual decoding while reducing the paired training data needed to adapt to a new participant.

Primary area:
applications to neuroscience & cognitive science
Keywords:
multimodal representation learning, cross-modal alignment, contrastive learning, neural visual decoding, cross-subject learning, EEG, MEG

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_158TOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary
  1. This paper proposes ReShare, a region-structured framework for shared visual decoding from EEG and MEG signals across multiple participants.
  2. The framework combines region-specific neural tokenization, participant-specific feature calibration, and shared latent integration to address inter-subject variability. It further supports efficient adaptation to unseen participants through pretrained decoder reuse.
  3. Experiments on THINGS-EEG2 and THINGS-MEG demonstrate strong zero-shot image retrieval performance, achieving 95.70% and 61.88% Top-1 accuracy, respectively. Additional experiments investigate cross-subject generalization, limited-data adaptation, neural representation consistency, and component effectiveness.
Critical strengths
  1. The paper addresses the important challenge of inter-subject variability in neural visual decoding. Its integration of regional representations and participant-specific calibration provides a reasonable mechanism for transferring knowledge across participants and reducing the demand for individual training data.
  2. ReShare substantially outperforms existing methods on both EEG and MEG benchmarks. Particularly encouraging is its limited-data adaptation performance, achieving 73.15% average Top-1 accuracy using only 10% of each unseen EEG participant's training pairs, compared with 14.15% when training from scratch. The extensive ablations further strengthen the empirical evaluation.
Critical weaknesses
  1. The proposed framework primarily combines established techniques, including regional tokenization, FiLM-based calibration, attention mechanisms, and MoE routing. More importantly, the non-regional baseline already achieves 93.90% Top-1 accuracy, while the complete framework reaches 95.70%. This relatively small improvement raises concerns about whether the proposed region-structured mechanisms are responsible for the substantial gains over existing methods.
  2. The main comparisons involve methods with different visual backbones, training configurations, and computational budgets, making it difficult to isolate architectural improvements. Furthermore, performance decreases substantially under leave-one-subject-out evaluation, particularly for MEG, while limited-data adaptation is evaluated only on EEG. These limitations weaken the evidence for robust cross-participant generalization.
Questions for authors
  1. Given that the non-regional baseline already achieves 93.90% Top-1 accuracy, can the authors clarify which proposed mechanisms fundamentally distinguish ReShare from existing shared decoders and justify its methodological novelty?
  2. Can the authors provide comparisons under matched visual representations, training budgets, and model capacities to determine whether the reported improvements originate from the proposed regional architecture rather than differences in experimental configurations?
  3. Can the authors explain the substantial performance degradation in leave-one-subject-out evaluation, particularly on MEG, and provide evidence that the proposed limited-data adaptation strategy generalizes beyond the EEG experiments?
Reviewer_XYGJOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ReShare is a framework for shared visual decoding from EEG and MEG, with separate models trained for each modality. It combines regional sensor tokenization, participant-specific regional FiLM calibration, and a shared latent integration network. Its predictions are aligned with intermediate patch features from a frozen CLIP ViT-L/14 encoder using contrastive and regression objectives.

Critical strengths

ReShare reports participant-averaged neural-to-image Top-1 accuracies of 95.70% on THINGS-EEG2 and 61.88% on THINGS-MEG. Evaluation across both modalities, together with component ablations and cross-participant matching, provides useful evidence that the framework is effective under the evaluated conditions.

Critical weaknesses

1.The architectural contribution and scientific insight are not sufficiently established. In Table 3, the non-regional baseline B0 already achieves 93.90% Top-1, compared with 95.70% for the complete model. Regional tokenization and MoE contribute only 0.15 and 0.05 percentage points, respectively, in the progressive ablation. Although Appendix D.2 provides stronger evidence for regional FiLM, the necessity of several added components remains unclear, particularly without seed variability and computational costs for each variant. Intermediate visual alignment is adopted from prior work, so its benefits should be distinguished from the novel architectural contribution.

Furthermore, the calibration interventions establish that the trained model uses participant-specific parameters, but do not establish why the particular topographic partition is important. Comparisons with size-matched random partitions would help distinguish meaningful spatial structure from generic grouping or regularization. This limits both the demonstrated methodological novelty and the scientific explanation of the gains.

2.The comparisons do not sufficiently isolate the source of the performance improvements. As acknowledged in Section 4.1, ReShare uses joint training across participants, whereas most reference results in Table 1 come from independently trained participant-specific models. The headline gaps therefore combine differences in training-data coverage with differences in architecture and training objectives. The jointly trained controls are useful, but retain different visual targets, input processing, and optimization configurations.

Consequently, the comparisons demonstrate the effectiveness of the overall system more convincingly than the advantage of the proposed regional design itself. Stronger controls with matched supervision, input information, and training budgets are needed to support the architectural claims.

Questions for authors

1.Which differences in input channels, time windows, visual targets, optimization budgets, and checkpoint selection remain between ReShare and the jointly trained baselines? Can the authors provide a more tightly matched comparison that isolates regional organization and calibration?

2.Why is the prediction from a mixed neural input regressed toward the unmixed visual target in Equation (7), while the contrastive loss supervises both mixture constituents? Appendix D.3 compares mixed and unmixed regression inputs, but does not directly test mixed visual targets. Also, what is the intended sample-weighting interpretation of the unnormalized transposed soft targets in the reverse terms of Equations (6)/(16)?

Reviewer_g17sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies cross-participant EEG/MEG visual decoding. The authors propose ReShare, which uses sensor topography to partition EEG or MEG into regions. Each region uses a linear spatial projection shared across participants, yielding 256 regional tokens in total that retain the full temporal trajectory. Experiments on multiple datasets prove the effectiveness of the method.

Critical strengths
  1. The experiments are relatively comprehensive, covering EEG and MEG visual retrieval, few-shot adaptation, component ablations, and cross-participant neural matching.
  2. The proposed region-wise FiLM is a simple and parameter-efficient design.
Critical weaknesses
  1. The main contribution is not sufficiently validated. In Table 3, B0 already achieves 93.90% Top-1 accuracy, exceeding the previous SOTA of 82.60%, while the full ReShare only improves it to 95.70%. Moreover, NoFiLM in Table 12 performs even worse than B0 (91.85% vs. 93.90%). These results suggest that the gains may mainly come from shared training rather than regional structure.
  2. Limited methodological novelty. The framework mainly combines existing techniques, including sensor grouping, FiLM, attention, and CLIP-based alignment. Similar ideas have been explored in EEGiT, RAHS, and ENIGMA.
  3. Insufficient explanation of regional calibration. Table 16 shows almost no change in cross-participant matching before and after FiLM (9.96% vs. 9.97%). However, Table 17 shows a large accuracy drop when using another participant's FiLM (95.70% to 81.69%). Does FiLM actually reduce cross-participant differences, or does it mainly encode participant-specific information?
Questions for authors

1.Why does the NoFiLM variant (91.85%) perform worse than B0 (93.90%)? How do the authors explain this result, given that regional structure is claimed to provide a useful inductive bias?

  1. Would the method achieve similar performance using randomly assigned sensor regions? A comparison with random partitions under the same architecture and training settings would help verify whether the improvements truly come from sensor topography.

  2. Does regional FiLM reduce cross-participant distribution differences or mainly encode participant identity? Could the authors provide additional evidence or comparisons with standard calibration methods?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.