acceptodds
Under review as a conference paper at ICLR 2027

PrimateAlign: Unifying Zero-Shot Cross-Species Visual Decoding and Neural Translation

Abstract

Recent neural decoding advances have successfully translated brain activity into visual representations. Despite neuroscientific evidence for profound similarities in primate visual processing, computational decoding models remain single-species. To address this limitation, we propose PrimateAlign, the first unified framework for cross-species visual decoding. Our approach projects human electroencephalography (EEG) and macaque multi-unit activity (MUA) into a shared multimodal latent space. By combining global semantic matching with a novel composite loss formulation for dense spatial alignment, the framework explicitly preserves both high-level conceptual meaning and fine-grained visual details. This shared representational space creates a computational bridge between distinct biological signals. In doing so, it unlocks a fundamentally novel capability of bidirectional cross-species neural translation on unseen stimuli. Extensive experiments demonstrate that PrimateAlign achieves state-of-the-art image retrieval performance in both modalities, reaching 85.9% (+5.9%) on THINGS-EEG2 and 96.4% (+27.1%) on THINGS-TVSD.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neural Signal, EEG, MUA, Multi-Unit Activity, Electroencephalography, Cross-Sepcies, Visual Decoding, Representation Learning, THINGS

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_RqgyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  1. This paper proposes PrimateAlign, a unified framework for cross-species visual decoding and neural translation between human EEG and macaque MUA signals.
  2. The framework employs modality-specific neural encoders and shared visual representations, combining global semantic alignment with dense token-level matching through MaxSim and soft-target contrastive learning. It further introduces cross-species latent alignment and conditional decoders for bidirectional neural signal translation.
  3. Experiments on THINGS-EEG2 and THINGS-TVSD demonstrate improved zero-shot image retrieval, achieving 85.9% and 96.4% Top-1 accuracy, respectively. Additional experiments evaluate cross-species neural retrieval and translation, supported by component-wise ablations.
Critical strengths
  1. The paper addresses the underexplored problem of aligning human EEG and macaque MUA within a unified representation space. Extending neural decoding beyond individual species and enabling bidirectional neural translation introduces a potentially valuable research direction for computational neuroscience.
  2. PrimateAlign demonstrates substantial improvements in visual retrieval, particularly on macaque MUA, while achieving promising cross-species retrieval accuracy on unseen stimuli. The ablation studies further support the effectiveness of dense alignment and explicit cross-species coupling.
Critical weaknesses
  1. The cross-species experiments primarily compare against relatively simple PCA-based baselines, which may not adequately establish the advantages of the proposed framework. Furthermore, the modest signal correlations, particularly for human-to-macaque translation, leave uncertainty about whether the model captures meaningful neural dynamics beyond shared stimulus semantics.
  2. The framework largely integrates established components, including Mamba encoders, MaxSim matching, soft-target contrastive learning, and latent alignment. Although the ablations demonstrate component effectiveness, they do not sufficiently isolate the contributions of individual cross-species objectives or distinguish architectural gains from those arising from joint training and additional supervision.
Questions for authors
  1. Can the authors compare PrimateAlign against stronger cross-modal alignment methods trained with equivalent visual supervision to demonstrate that the observed improvements originate from the proposed cross-species learning mechanism?
  2. How do the authors interpret the relatively low Pearson correlation in human-to-macaque translation? Can they provide additional evidence that the reconstructed signals preserve meaningful neural dynamics rather than primarily stimulus-dependent statistical patterns?
  3. Can the authors separately ablate cross-species latent alignment and translation supervision to quantify their individual contributions to neural retrieval, signal translation, and within-species decoding?
Reviewer_WfwVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PrimateAlign, a unified framework for cross-species visual decoding and neural translation. It aligns human EEG and macaque MUA in a shared visual-semantic space through global semantic matching, dense spatial alignment, and cross-species latent constraints. Experiments on THINGS-EEG2 and THINGS-TVSD demonstrate strong performance in zero-shot image retrieval, cross-species neural retrieval, and bidirectional neural signal translation.

Critical strengths
  1. The paper explores an interesting and challenging problem of cross-species neural decoding, providing a new direction for learning shared representations from heterogeneous neural signals.
  2. The proposed framework supports multiple tasks, including visual retrieval, cross-species neural retrieval, and bidirectional neural signal translation. The reported results are promising, with notable improvements over existing baselines.
  3. The paper is generally well written and easy to follow. The experiments are relatively comprehensive, including comparisons with existing methods, ablation studies, and analyses across multiple random seeds.
Critical weaknesses
  1. Limited methodological novelty. The proposed framework mainly combines existing techniques, including modality-specific encoders, CLIP-based alignment, MaxSim, semantic soft targets, and reconstruction losses. The key methodological differences from existing multimodal alignment frameworks remain limited.
  2. Insufficient experimental analysis. Although the authors report promising performance improvements, the paper lacks biological interpretation, such as analyses of brain-region activation patterns. This makes it difficult to understand why cross-species alignment improves human EEG decoding. Moreover, it remains unclear whether the improvements come from cross-species alignment itself or simply from the use of additional training data.
  3. The benefits of cross-species joint training are not sufficiently proved. In Table 5, joint training improves human EEG retrieval by only 1.7 percentage points compared with human-only training, while macaque retrieval decreases from 98.1% to 96.4%. How to explain these results?Whether cross-species training consistently benefits the decoding tasks?
  4. The experiments are restricted to two datasets involving human EEG and macaque MUA responses to shared visual stimuli.Evaluation on additional datasets and decoding tasks would be necessary to further demonstrate the generalizability of the proposed method and its conclusions.
Questions for authors

See the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.