acceptodds
Under review as a conference paper at ICLR 2027

BIGEM: BIDIRECTIONAL EEG–IMAGE GENERATIVE MODELING FOR PROBING VISUAL REPRESENTATIONAL GEOMETRY

Abstract

Understanding the correspondence between visual stimuli and brain activity requires models that characterize the representations linking them. However, the mappings between these two modalities are typically modeled independently. We introduce BiGEM, a bidirectional generative framework that connects EEG-to-image decoding and image-to-EEG generation through a shared visual latent space. On THINGS-EEG2, BiGEM achieves 49.3% Top-1 accuracy in 200-way visual retrieval from EEG and a Pearson correlation of 0.456 between generated and measured mean EEG responses. Bidirectional analysis reveals that stimulus identity is preserved most strongly in the shared latent representation, while joint refinement improves reciprocal consistency without improving direct decoding. The learned EEG representation aligns most strongly with intermediate stages of the visual hierarchy, with early EEG responses associated with spatial phase, texture, and edge orientation. Controlled perturbations further dissociate two complementary properties of generated neural responses: perturbation severity determines response magnitude, whereas the identity of perturbation families remains recoverable from spatiotemporal response organization after severity control. This study provides a unified framework for modeling and studying the shared representations linking visual and neural signals.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Visual Encoding and Decoding, Neural Representation Learning, Bidirectional Generation, Rectified Flow, Cycle Consistency

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WpMqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

BiGEM connects EEG-to-image reconstruction and image-to-EEG generation through a shared OpenCLIP visual representation. A subject-specific EEG encoder is trained using blur-aware visual alignment, a distilled adapter conditions frozen SDXL, and a subject-specific rectified-flow model generates PCA coefficients that a fixed decoder maps to EEG. The paper evaluates frozen bidirectional compositions and a separate latent-cycle refinement stage, distinguishing returned-latent identity from rendered-output fidelity. It also analyzes learned representations and measured EEG and probes generated responses using nine visual perturbation families. On THINGS-EEG2, it reports 49.30% Top-1 in 200-way retrieval from measured EEG averaged over 80 repetitions and a correlation of approximately 0.456 between generated and measured mean responses. The main analytical claims concern information preservation across transformations and the separation of perturbation-response magnitude from spatiotemporal organization.

Critical strengths

1. A potentially useful analytical framework with informative controls. The study goes beyond optimizing a reconstruction score by separating direct generation, frozen compositions, and joint refinement. Table A2 includes a deterministic predictor, mismatched conditioning, and an ATM-S readout not used to train the EEG generator. These controls provide evidence of stimulus-information retention and reveal meaningful trade-offs. The authors explicitly distinguish the high shared-latent readout from independent evidence of EEG realism.

2. Careful treatment of several statistical and interpretive issues. The appendix describes subject-level inference, multiplicity correction, matched generation noise, image-grouped cross-validation for magnitude prediction, and exclusion of the query subject from the strict severity-adjustment and template-fitting procedure. It also distinguishes primary and secondary analyses and acknowledges that generated perturbation responses do not establish causal effects in humans. These are substantive strengths, although they do not resolve the concerns below.

Critical weaknesses

1. The headline retrieval result has an unresolved candidate-side dependency on test EEG, so its interpretation and comparability are not established.

Appendix A.2.6 (p. 16) states that candidate scalars are obtained from fixed modulation networks applied to features of real test-EEG means, that modulation is applied twice, and that retained amplitude modulation produces an EEG-assisted gallery rather than plain cosine retrieval. The unresolved issue is whether candidate image j is scaled using its paired test response. If so, constructing the gallery requires candidate-side neural information and its image correspondence, beyond the current EEG query and an image-only candidate set. Using a fixed gallery for every query does not remove this dependency. Candidate-specific scaling can change rankings; by contrast, a common positive scalar applied to all candidates for one query cannot. The paper needs an indexed scoring equation or executable scoring routine that resolves this distinction, including the normalization order and the source of each scalar.

The decisive control is to evaluate the same trained checkpoint and the same 80-repeat EEG queries against the same 200 images, but with candidate representations constructed without their paired test EEG. This is not answered by Table A3b's comparison of nonblurred, fixed-medium, and adaptive training targets: that comparison does not isolate a change to the evaluation rule for a fixed encoder. Table 1 should then compare methods under explicitly matched information access, with paired subject-level differences. The variability reported in Table A4 is useful, but does not resolve this protocol issue. My concern is not that 49.30% is intrinsically implausible, and I cannot determine the magnitude or direction of any scoring effect without this control. The current text establishes a test-response dependency requiring clarification, not that test EEG entered gradient training or that the results were fabricated.

The approximately 97% readouts must also remain separate from measured-EEG retrieval. They start from a known image and assess whether its identity can be recovered from generated EEG in a representation that also supplies the generator's semantic supervision (A.2.4). Appendix A.2.6 explicitly states that the 97.15% returned-latent result uses the original normalized image gallery, so I do not attribute that result to candidate modulation. Nevertheless, it is not independent evidence of realistic neural responses. The authors recognize this limitation; the 13.55% ATM-S result in Table A2 is a more independent identity check, although the difference between the two readouts does not by itself quantify artificial inflation.

Finally, the comparisons need a traceable evaluation specification. Tables 2-3 point to A.2.6 for sources, but that section does not provide a baseline-by-baseline account of quoted versus rerun values and matched preprocessing, repetition/sample aggregation, and metric settings. Table 3 is explicitly contextual and cannot substitute for a controlled comparison. The source of the frozen modulation-network weights and the exact retrieval implementation are particularly important here. This assessment is limited to the PDF; it does not assert that no separate implementation package exists.

2. The distinctive contribution of bidirectional refinement and stochastic perturbation probing is not isolated from simpler explanations.

Attribution of joint-refinement gains. Appendix A.2.5 and Table A4 compare independent, 1,000-update warm-up, and 2,000-update cycle-active endpoints. Training duration and cycle activation covary, as the authors acknowledge on p. 31. The observed gains establish improvement after the stated refinement procedure, but do not isolate the cycle objective. Direct image-to-EEG correlation already rises from 0.4555 to 0.4747 during warm-up and is 0.4702 at the cycle-active endpoint; direct EEG retrieval changes from 49.30% to 48.85%. These values are not evidence of significant deterioration, but an equal-update, cycle-disabled continuation from the same initialization is needed to attribute the gains. The absence of a direct-decoding improvement is not itself grounds for rejection; the problem is identifying what the added mechanism contributes.

Independent support for realistic stochastic responses. Table A2 shows higher mean-response correlation for the deterministic predictor than for the flow (0.5967 versus 0.4555). Its ATM-S accuracy is 12.60% versus 13.55% for the flow, with an uncorrected paired p=0.218750. This does not establish equivalence, and the mismatched-condition control provides useful evidence of stimulus dependence. However, a distinctive benefit of stochastic generation needs evidence about response distributions, not simply the ability to produce different samples. The near-unit dispersion ratio measures mean pairwise distances in a retained, standardized PCA subspace; it does not establish correct conditional covariance, temporal/spectral structure, or variability in discarded dimensions. A relevant control is the existing deterministic predictor augmented with residual variability estimated only from training data, assessed with independent variability diagnostics.

Added value and neural correspondence of the probing results. The magnitude/pattern distinction may also arise in simpler image-conditioned mappings. As a mathematical illustration, a linear predictor f(z)=Wz+b produces a response change W(z_pert-z): magnitude may track visual displacement while response patterns retain displacement-direction information. This is an alternative explanation, not an empirical finding about BiGEM. Applying the same perturbation analysis to the existing deterministic predictor would help establish what the bidirectional generative framework adds. Similarly, the intermediate-layer RSA profile should be compared with that of the blur-aware visual target itself, since EEG representations are explicitly trained toward that target.

The measured-EEG correspondence in Figure A19 (p. 48) further limits the broader interpretation: all nine confidence intervals for global same-feature alignment advantage include zero, and significant positive restricted preference is found for edge orientation, but not spatial phase or texture. This is not proof of absent correspondence, but broad feature-specific agreement between generated perturbation patterns and measured EEG is not established. The paper appropriately avoids a human-causal claim; even under its descriptive framing, these mixed results should be central to the interpretation. The strict residual-pattern analysis also holds out subjects while averaging the same image set for queries and templates (A.4.8); it should be distinguished from the separate image-held-out magnitude analysis and from generalization of the perturbation patterns to new stimuli.

Overall, the concern is not merely that existing modules are combined. The potentially valuable contribution lies in the framework and its empirical findings, but the present controls do not sufficiently distinguish those findings from evaluation-specific effects, additional optimization, or simpler conditional predictors. I therefore lean toward rejection in the current form. An auditable image-only-gallery evaluation and focused controls of the existing claims could materially change this assessment.

Questions for authors

1. Retrieval information boundary and fixed-checkpoint control. Does candidate j use its paired real test EEG to obtain either modulation scalar? Please provide the fully indexed scoring equation or code, the two modulation steps, and the normalization/broadcasting axes. Using the same trained encoder and test queries, what are Top-1, Top-5, and MRR with an image-only gallery and no candidate-side test-EEG calibration? Please include paired subject-level changes and explicit image IDs in the ranking outputs.

2. Equal-update cycle control. Are direct-only continuations available with the same initialization, update count, optimizer schedule, and sampling as the cycle-active models? How do cycle and independent readouts compare under this matched control? The existing 1,000-update warm-up endpoint does not isolate the effect at 2,000 updates.

3. Stochastic fidelity and probing-specific value. Can the existing deterministic predictor, with and without training-derived residual variability, be evaluated using the same independent response-distribution diagnostics and perturbation-family analysis? Which principal findings distinguish the flow-based framework from these simpler alternatives?

4. Interpretation of neural correspondence. How should the broad representational conclusion be qualified in light of Figure A19's global intervals and restricted results? Does the intermediate-layer RSA pattern differ from that of the visual alignment target? Please distinguish cross-subject identification on the same stimuli from the separate image-held-out magnitude prediction result.

5. Reproducibility and comparison provenance. Please identify the scoring implementation, modulation-network provenance, adapter loss definitions and numerical weights, and the configurations that produced the principal results. Which baseline values were rerun versus quoted, and how were information access, preprocessing, repeat/sample averaging, and metric computation matched? These are clarification requests about the reported experiments, not a request for a new dataset or clinical study.

6 comments

We thank you for your detailed review and constructive suggestions on evaluation, experimental controls, and reproducibility. Your comments helped us identify and correct the candidate-side EEG dependency in retrieval and clarify the evidence supporting our scientific conclusions. Below, we address your questions point by point and describe the corresponding corrections, additional analyses, and manuscript revisions.

Response to Q1: Retrieval information boundary and fixed-checkpoint control

We confirm that the original 49.30% Top-1 retrieval accuracy was affected by test-time data leakage, as candidate image representations were calibrated using their paired real test EEG. We corrected the evaluation protocol by removing candidate-side EEG calibration while keeping the trained model weights, test EEG queries, and 200-image candidate set unchanged. Retrieval now uses an image-only gallery and cosine similarity between EEG and image embeddings.

Across ten subjects, the corrected image-only evaluation yields 48.90% Top-1, 78.55% Top-5 and 0.6222 MRR. Relative to the superseded scoring rule, the paired changes are −0.40 percentage points, 0.00 percentage points and −0.002819, respectively.

None of these differences was statistically significant after BH-FDR correction (Top-1: q = 0.281; Top-5: q = 1.000; MRR: q = 0.123). Table R5 provides the paired subject-level changes.

Table R5. Corrected image-only retrieval and paired changes. Δ = corrected − original; pp denotes percentage points.

SubjectImage-only Top-1 (%)Δ Top-1 (pp)Δ Top-5 (pp)Δ MRR
sub-0143.50−1.500.00−0.008429
sub-0248.50−1.000.00−0.005699
sub-0349.000.000.00−0.001168
sub-0447.50−1.00−0.50−0.006182
sub-0540.500.000.00−0.000654
sub-0655.00+1.000.00+0.004322
sub-0747.500.000.00−0.001257
sub-0856.50−0.500.00−0.004811
sub-0943.500.000.00+0.000997
sub-1057.50−1.00+0.50−0.005306
Mean48.90−0.400.00−0.002819

We further examined all 2,000 paired queries. Sixteen queries changed Top-1 correctness after correction: four changed to correct and twelve changed to incorrect, giving a net loss of eight correct queries (0.40 pp). Table R6 lists all corresponding stimulus image IDs.

Table R6. Stimulus IDs for queries whose Top-1 correctness changed between galleries.

SubjectChanged to correctChanged to incorrect
sub-01crumb_12scordon_bleu_08s, egg_12s, nightstand_05s, vise_11s
sub-02—duffel_bag_03s, tomato_sauce_02s
sub-03tiara_07scobra_01b
sub-04—croissant_06s, dagger_06s
sub-05——
sub-06minivan_17s, snowshoe_04s—
sub-07——
sub-08—bok_choy_03s
sub-09——
sub-10—ladle_13s, wheat_05n
Total4 images12 images

These results demonstrate that removing candidate-side EEG calibration produces only a small observed performance difference, with no statistically significant change after multiple-comparison correction. Although this does not establish statistical equivalence, it indicates that the strong retrieval performance is largely retained without candidate-side test-EEG information.

Manuscript revisions. We have corrected the retrieval evaluation in Sections 4.1–4.2 and replaced the previous headline result with the corrected image-only result in the Abstract, main text and Table 1. Appendix A.2.6 documents the original and corrected scoring protocols and retains the previous EEG-assisted metrics alongside the corrected metrics for historical documentation and comparison. All retrieval readouts affected by the original scoring rule, including those in the alignment ablations and stage-specific evaluations, have been re-evaluated under the corrected protocol. The corresponding retrieval metrics, statistical comparisons, and descriptions have been updated accordingly. Other experimental results that do not depend on this scoring rule remain unchanged.

Response to Q2: Equal-Update Cycle Control

We thank you for raising this important concern. We clarify our training protocol and provide a new equal-update control across ten subjects to isolate the contribution of cycle consistency losses.

Training protocol. Our Independent (0) condition consists of two separately pretrained unidirectional generators composed without additional joint training. The Warm-up (1000) stage performs 1,000 joint optimization updates using only direct-task objectives, without cycle losses. The original Cycle (2000) stage continues from this checkpoint for another 1,000 updates, introducing both image-start and EEG-start cycle consistency losses.

Equal-update control. To disentangle the effect of cycle losses from additional optimization, we introduce an extended Warm-up (2000) control. Both Warm-up (2000) and Cycle (2000) start from the same complete Warm-up (1000) checkpoint and receive exactly 1,000 additional updates, sharing identical initialization, optimizer states, learning-rate schedules, and pre-specified sampling inputs. The only intended difference is the inclusion of cycle losses.

Results. Relative to Warm-up (2000), Cycle (2000) has higher mean image-cycle cosine (0.2992→0.3114) and EEG-cycle Pearson correlation (0.4050→0.4396), and lower EEG-cycle MSE (0.0968→0.0719; Table R7). These matched-update comparisons support a contribution of the cycle losses beyond continued direct-task optimization.

Table R7. Matched-update cycle control (mean ± SD, n=10).

MetricHistorical Independent (0)Warm-up (1000)Warm-up (2000)Cycle (2000)
Image-cycle cosine ↑0.2707 ± 0.01500.2887 ± 0.01540.2992 ± 0.01620.3114 ± 0.0174
EEG-cycle Pearson ↑0.4014 ± 0.05700.4035 ± 0.05510.4050 ± 0.05470.4396 ± 0.0630
EEG-cycle MSE ↓0.1021 ± 0.05230.0977 ± 0.05060.0968 ± 0.05180.0719 ± 0.0332
Direct EEG→Visual Top-1 (%) ↑48.90 ± 5.7748.85 ± 5.4548.50 ± 4.8948.45 ± 4.69

Values are mean ± SD across ten subjects. The matched-update comparison is Cycle (2000) versus Warm-up (2000). Historical Independent (0) values are provided for reference and are excluded from this comparison.

Independent readout control. We further evaluate all four stages using the same frozen independently pretrained EEG reader (Table R8). Compared with the equal-update Warm-up (2000) control, Cycle (2000) increases mean image-cycle cosine similarity from 0.2770 to 0.3010, Cycle-A latent Top-1 from 96.05% to 97.25%, and Cycle-B latent Top-1 from 23.40% to 25.30%. These higher mean scores provide additional evidence of improved cycle consistency under a fixed, independently trained readout.

Table R8. Cycle consistency under a fixed independent EEG reader (10 subjects).

MetricIndependent (0)Warm-up (1000)Warm-up (2000)Cycle (2000)
Image-cycle cosine ↑0.2707 ± 0.01500.2756 ± 0.01530.2770 ± 0.01600.3010 ± 0.0160
Cycle-A latent Top-1 (%) ↑95.65 ± 1.9796.85 ±1.9796.05 ± 1.7497.25 ± 1.53
Cycle-B latent Top-1 (%) ↑21.65 ± 4.3923.10 ± 4.5323.40 ± 5.0725.30 ± 5.57

Overall, the matched-update experiments support improved mean bidirectional consistency with cycle losses beyond continued direct-task optimization. Section 4.3 and Appendix C.2 (Table A4) now report the updated matched-update results and the direct-task trade-off. The fixed independent-reader control is provided in Table R8 of this response.

Response to Q4: Neural Correspondence and Representational Interpretation

Neural correspondence (Figure A19). We agree that our findings should not be interpreted as establishing broad feature-specific correspondence between generated and measured EEG. Figure A19 shows that all nine global same-feature alignment advantages have 95% confidence intervals including zero. The restricted analysis provides significant positive evidence for edge orientation (Holm-adjusted p=0.00293), but not spatial phase or texture.

Importantly, our study distinguishes three complementary levels of evidence: visual-feature associations independently observed in measured EEG (Fig. 3), perturbation-family information recoverable from generated EEG after severity control (Figs. 7 and A18b), and direct correspondence between generated and measured response patterns (Fig. A19). The first two findings do not imply global feature-wise agreement in the third.

We therefore limit our conclusion to representational structure observed within each domain, with partial and feature-specific evidence of correspondence across generated and measured EEG, rather than claiming comprehensive neural correspondence. This interpretation is now explicit in Section 4.5 and the Conclusion.

2. Intermediate-layer RSA versus visual alignment targets (Tables R15–R16). To directly examine whether the intermediate-layer preference is inherited from visual supervision, we conducted a matched RSA analysis using a newly trained ViT-H/14-aligned EEG encoder.

As shown in Table R15, the EEG representation peaks at ResNet-50 (He et al., 2016) Layer3 (ρ=0.3361), whereas its medium-blur ViT-H/14 (Cherti et al., 2023) target peaks at Layer4/AvgPool (ρ=0.2804). EEG exhibits a stronger intermediate-layer preference (ΔH=0.1538). EEG–Layer3 correspondence remains positive after controlling the visual target RDM (partial ρ=0.2955) and after additionally controlling nine visual-descriptor RDMs (partial ρ=0.2362). These results indicate that the EEG representation's hierarchical profile is not simply identical to its visual supervision target.

Table R15. Layer-wise RSA for EEG representations and their visual alignment target.

ResNet-50 LayerEEG Representation RSAVisual Target RSA
Conv10.0936240.139149
Layer10.2250110.126977
Layer20.3103810.175631
Layer30.3361460.227842
Layer40.1896730.280388
AvgPool0.1896730.280388

We further examined whether this preference depends on blur-aware training. Table R16 compares matched no-blur, fixed-medium, and adaptive-blur EEG encoders. All three peak at Layer3, with nearly identical Layer3 RSA values (0.3383, 0.3362, and 0.3361) and similar intermediate-layer preferences (H=0.1454, 0.1487, and 0.1488). Thus, the intermediate-layer preference is not unique to adaptive blur-aware alignment. We do not attribute this preference specifically to adaptive blur or claim that the observed geometry establishes a distinct neural mechanism.

Table R16. Intermediate-layer RSA across blur conditions.

MetricNo-blurFixed-mediumAdaptive
Layer3 RSA ↑0.3383320.3362000.336146
Intermediate preference H ↑0.1454380.1487110.148768

3. Distinct Generalization Settings

We clarify that these analyses evaluate two distinct generalization settings.

Perturbation-family identification (Appendix A.4.8; Fig. A18b) uses leave-one-subject-out evaluation. Templates are constructed from the other nine subjects, while the query subject is excluded from severity-model fitting. Both templates and queries aggregate responses over the same 200 stimulus images. Accordingly, this result supports cross-subject identification of perturbation families over a shared stimulus set; it does not test perturbation-family identification on held-out images.

Magnitude prediction (Appendix A.4.6) uses fivefold image-grouped cross-validation, holding out entire image identities together with all associated perturbation conditions. The held-out R² is 0.9022 using severity predictors and 0.9172 after adding family identity and family-by-displacement interactions. These results support response-magnitude prediction for images held out from the regression fit, but do not establish image-held-out perturbation-family identification.

Section 4.5 now explicitly distinguishes these two evaluation settings and their conclusions.

References

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. CVPR, 770–778.

Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., & Jitsev, J. (2023). Reproducible Scaling Laws for Contrastive Language-Image Learning. CVPR, 2818–2829.

Reviewer_P7YkOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper models both directions of the EEG-image relationship, decoding and encoding, in a single shared latent space, then audits where stimulus information survives the round trip. A frozen OpenCLIP ViT-H/14 defines the space; a subject-specific EEG encoder aligns into it with blur-aware targets, a visual adapter feeds it to frozen SDXL for image generation, and a rectified-flow sampler with a PCA decoder generates EEG from it. Frozen image->EEG->image and EEG->image->EEG compositions are evaluated before and after a latent-space cycle refinement stage.

On THINGS-EEG2, retrieval reaches 49.30% Top-1 (next best 47.45) and image-conditioned EEG generation reaches r=0.4555. The clearest result is the cycle decomposition: the returned latent keeps 97.15% Top-1 while the rendered image keeps only 8.90%, so the generators, not the alignment, are the bottleneck. Joint refinement improves cycle consistency (+15.2%, q=0.0081) but not direct retrieval (49.30 to 48.85, n.s.). Supporting analyses map the learned EEG representation onto intermediate ResNet-50 stages and use nine perturbation families to separate response magnitude, driven by severity, from spatiotemporal organization, which keeps perturbation identity recoverable after severity control.

Critical strengths
  1. The frozen-cycle decomposition localizes the information bottleneck in the modality generators rather than the alignment, a concrete and checkable claim. The negative result that cycle refinement improves round-trip consistency without improving direct retrieval is reported with effect sizes and q values, which makes the positive claims more trustworthy.

  2. The perturbation study separates response magnitude, explained almost fully by perturbation severity, from spatiotemporal organization, which retains family identity well above chance after severity control with leave-one-subject-out validation. Severity matching that fails Holm correction is reported as a null, not smoothed over.

Critical weaknesses
  1. Nearly every functional module is imported. The blur-aware alignment is UBP's and alone accounts for most of the retrieval gain (37.95 without it), image generation is frozen SDXL, and EEG generation is rectified flow over a PCA basis. What is new is the composition and the audit. The framing sometimes reads as if bidirectional generation itself were the contribution, and the reader has to reconstruct which parts come from which citation.

  2. The encoding direction is weak next to dedicated encoders (r=0.4555 vs. above 0.6 on the same dataset in recent work), everything rests on one dataset and EEG only, and the rendered cycle accuracy of 8.90% shows the full pipeline is far from a usable generative product.

Questions for authors
  1. Why does the encoding branch operate on 1024 PCA coefficients rather than the signal space used by dedicated encoders? A same-protocol comparison would clarify whether the gap reflects the representation or the method.

  2. Does the bottleneck claim replicate on a second dataset or on MEG? It currently stands on one dataset with one generator pair.

5 comments

We thank you for recognizing the value of our frozen-cycle decomposition, the careful distinction between latent identity and rendered-output fidelity, and the controlled perturbation analysis. We appreciate your constructive suggestions regarding methodological attribution, EEG representation, and generalization.

1. Scientific Contribution and Methodological Novelty (Critical Weakness 1)

We adopt established methods and pretrained models, including UBP, CLIP, and SDXL, as foundational components of BiGEM. We have revised the manuscript to distinguish inherited components, architectural adaptations, and original scientific findings. However, BiGEM is not merely an integration of existing models, but introduces architectural adaptations and a bidirectional framework for investigating representational consistency and information preservation.

Specifically, BiGEM extends UBP's retrieval-oriented framework (Wu et al., 2025) through a 63-channel dual-branch EEG encoder, decoder-aware image generation, conditional EEG synthesis, and bidirectional cycle objectives. BiGEM also demonstrates competitive performance against existing bidirectional approaches (Tables 1–2).

To the best of our knowledge, BiGEM is the first bidirectional EEG–image generative framework to explicitly distinguish latent-level stimulus identity retention from rendered-image fidelity through stage-wise analysis. This analysis characterizes information preservation and reciprocal representational consistency, revealing a substantial gap between the information retained in shared representations and that recovered in final image outputs (Sec. 4.3; Table 5).

References

Wu, H., Li, Q., Zhang, C., He, Z., & Ying, X. (2025). Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior. CVPR, 2246–2257.

2. Encoding Performance and Generative Limitations (Critical Weakness 2)

We agree that Pearson correlation is an important measure of EEG waveform fidelity. However, correlation depends on the evaluation protocol, including temporal windows, repetition averaging, and training objectives (loss functions). Our original experiments demonstrate that higher mean-response correlations are achievable: the deterministic mean-response predictor reaches r = 0.5967 (Sec. 4.2; Appendix B.1, Table A2), while our additional mean-target PCA regression achieves r = 0.6612 (Table R3). In contrast, our conditional Flow is designed to generate stochastic single-trial EEG responses rather than solely optimize mean-waveform prediction. To further examine the effect of evaluation windows and stimulus-information preservation, we evaluated the cycle-active BiGEM model (Table R2).

Table R2. Evaluation-window comparison for cycle-active BiGEM(n=10).

ModelPearson ↑ (0–200 ms)Pearson ↑ (0–500 ms)Pearson ↑ (0–1000 ms)ATM-S Top-1 ↑
Cycle-active BiGEM0.57860.54370.452413.5%

The results show that Pearson correlation varies substantially with the evaluation window. Meanwhile, generated EEG retains stimulus-discriminative information, achieving 13.5% Top-1 accuracy in 200-way identification (chance: 0.5%).

These findings highlight the importance of evaluating waveform fidelity and stimulus-specific information. Our objective extends beyond maximizing Pearson correlation to generating informative EEG responses that preserve visual representations and support reciprocal neural–visual transformations.

3. Why PCA Rather Than Direct Signal-Space Generation? (Question 1)

We thank you for this valuable suggestion. We chose PCA as a simple and efficient way to compress EEG signals for conditional flow-based EEG generation. The flow model generates compact PCA coefficients conditioned on image representations, and a fixed PCA decoder maps these coefficients back to EEG waveforms. This design reduces dimensionality and trainable parameter count while maintaining high reconstruction fidelity and mean-response prediction performance close to that of matched signal-space models.

Table R3. PCA and signal-space EEG generation under matched objectives(n=10).

RepresentationMethodPearson r ↑MSE ↓ATM-S Top-1 ↑Params (M)
PCAOracle Reconstruction0.9326———
PCAMean-target Flow0.5014 ± 0.08130.035045 ± 0.0083882.30% ± 1.096.44
Signal SpaceMean-target Flow0.5190 ± 0.07580.032725 ± 0.0060392.85% ± 1.3621.54
PCADeterministic0.6612 ± 0.05310.021111 ± 0.0044481.85% ± 1.235.52
Signal SpaceDeterministic0.6639 ± 0.05310.020978 ± 0.0043872.40% ± 1.3713.08
—Train-only Mean ERP0.6600———
PCAOriginal BiGEM Flow0.4555 ± 0.07970.0437513.55% ± 3.176.44

All newly trained models in Table R3 use MSE as the sole training loss, with all other experimental settings held constant apart from the model configuration. The original BiGEM Flow is included as a reference.

Our same-protocol experiments show that PCA substantially reduces model size while maintaining mean-response prediction performance close to direct signal-space modeling (Table R3). This pattern holds for both deterministic regression and mean-target Flow, supporting PCA as an efficient representation for conditional EEG generation. Under the full training objective, the PCA-based BiGEM Flow also achieves 13.55% ATM-S Top-1, showing that generated EEG retains stimulus-discriminative information (Table R3). The additional training objectives strengthen representational consistency with a modest reduction in waveform correlation, as documented by the loss ablations (Appendix B.3; Table A3b(c)). Together, these findings support our choice of PCA as a compact representation that reduces computational demands and supports the intended balance between waveform fidelity and stimulus-information preservation. This efficiency also extends to the latent representation. Compared with ViBE’s TSC-VAE (Xu et al., 2026), PCA uses substantially fewer latent elements (1,024 vs. 4,032) while achieving a reconstruction correlation close to the reported TSC-VAE result (0.9326 vs. 0.9412; Appendix A.2.4; Table R3).

References

Xu, G., Lai, Z.-R., Tang, Y., Song, Y., Zhou, S., Zhou, G., Wang, B., Zhu, J., & Long, J. (2026). ViBE: Visual-to-M/EEG Brain Encoding via Spatio-Temporal VAE and Distribution-Aligned Projection. arXiv preprint arXiv:2604.26218.

Li, D., Wei, C., Li, S., Zou, J., & Liu, Q. (2024). Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion. Advances in Neural Information Processing Systems, 37, 102822–102864. doi: 10.52202/079017-3266.

4. Generalization Beyond THINGS-EEG2 (Question 2)

We thank you for this valuable suggestion. To examine the generalizability of our information-bottleneck finding, we conducted additional experiments on the Natural Scenes Dataset (NSD; Allen et al., 2022) using fMRI, evaluating BiGEM-NSD and NeuroFlow (Mai et al., 2026). BiGEM-NSD follows the same bidirectional framework and stage-wise information-retention analysis as BiGEM, with modality-specific adaptations for the NSD fMRI data. Both models show a substantial gap between pre-render and post-render stimulus identity preservation (Table R4).

Table R4. Stimulus identification before and after rendering on THINGS-EEG2 and NSD.

DatasetModelPre-render Top-1Post-render Top-1
THINGS-EEG2BiGEM97.15%8.90%
NSDBiGEM-NSD99.54%24.54%
NSDNeuroFlow98.32%19.97%

These results provide cross-dataset and cross-model evidence that high latent-level identity preservation does not guarantee faithful stimulus recovery after image generation. The observed bottleneck is therefore not restricted to the original THINGS-EEG2 (Gifford et al., 2022) experiment or a single generator pair. More broadly, visual information captured by EEG and fMRI is shaped by neural processing pathways and affected by measurement noise. Differences in how visual features are transformed, attenuated, or obscured may contribute to varying degrees of information loss and reconstruction bottlenecks across modalities.

References

Allen, E. J., St-Yves, G., Wu, Y., Breedlove, J. L., Prince, J. S., Dowdle, L. T., Nau, M., Caron, B., Pestilli, F., Charest, I., Hutchinson, J. B., Naselaris, T., & Kay, K. (2022). A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25(1), 116–126. doi: 10.1038/s41593-021-00962-x.

Gifford, A. T., Dwivedi, K., Roig, G., & Cichy, R. M. (2022). A large and rich EEG dataset for modeling human visual object recognition. NeuroImage, 264, 119754. doi: 10.1016/j.neuroimage.2022.119754.

Mai, W., Nan, M., Zhu, Y., Cao, J., Zhang, R., Dai, Y., Song, C., Luo, A., & Wu, J. (2026). NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity. CVPR, 12966–12976.

Reviewer_TyCpOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper introduces BiGEM, a bidirectional framework that connects EEG-to-image reconstruction and image-to-EEG generation by sharing the visual latent space, aiming to study how visual information is retained, transformed, and organized between neural signals and computational representations. The model encodes EEG into the frozen visual space, drives image generation through an adapter, and generates EEG using conditional rectified flow. The main contribution of this paper is to combine bidirectional generation with information retention, geometric representation, and controlled perturbation analysis.
Critical strengths
  • The paper distinguishes information preservation in the shared latent space from fidelity in the generated outputs. This provides a useful diagnostic perspective on where stimulus information is lost during reciprocal translation.
  • The perturbation analysis goes beyond response magnitude by examining spatiotemporal organization and controlling for effective perturbation severity. These controls strengthen the interpretation of the model’s responses to different perturbation families.
Critical weaknesses
  • The motivation of this article is not clear enough. In related research, it has been pointed out that existing methods have achieved bidirectional modeling. However, this article still regards "independent modeling in both directions" as its main motivation. Clearly identifying the problems that existing methods fail to solve can better demonstrate the innovation of this article.
  • In the model architecture of this paper, I did not observe any innovative design by the author. It is mainly based on published methods. Moreover, in the crucial ablation experiments in Table 4 of the article, it is an ablation of the key design of the published article UBP. This further raises doubts about the innovation of the article.
Questions for authors
  • At the same time, the literature review for the baseline articles was not comprehensive enough. There were a series of one-way EEG-to-image visual decoding methods, and excellent retrieval results were achieved.
  • The perturbation analysis in the article is based on the image2EEG pathway. The authors have not demonstrated that the current bidirectional architecture is more stable than a single-directional image2EEG pathway.
2 comments

We appreciate your concrete questions about motivation, novelty, baselines, and one-way controls; they clarify where stronger evidence is needed.

1. Scientific Motivation and Gaps in Prior Bidirectional Work (Critical Weakness 1)

We revised the Introduction and related work accordingly.

Prior bidirectional studies have addressed different goals. MB2C (Wei et al., 2024) uses cycle consistency to improve EEG–visual alignment and decoding, but does not establish whether such gains extend to neural encoding or output fidelity. D²-FOSA (Yu et al., 2026) uses bidirectional latent diffusion for EEG-to-image reconstruction without evaluating generated EEG fidelity. NeuroFlow (Mai et al., 2026) jointly models fMRI-based encoding and decoding through reversible flow, but does not examine EEG-specific spatiotemporal feature organization.

We introduce BiGEM to integrate visual encoding and decoding through a shared latent space and investigate cross-directional representational geometry and information preservation.

References

Wei, Y., Cao, L., Li, H., & Dong, Y. (2024). MB2C: Multimodal Bidirectional Cycle Consistency for Learning Robust Visual Neural Representations. ACM Multimedia, 8992–9000. doi: 10.1145/3664647.3681292.

Yu, C., Shen, S., Li, X., & Li, Y. (2026). D²-FOSA: Dual-Diffusion Guided EEG-to-Image Reconstruction with Frequency-Oriented Semantic Alignment. CVPR, 26698–26710.

Mai, W., Nan, M., Zhu, Y., Cao, J., Zhang, R., Dai, Y., Song, C., Luo, A., & Wu, J. (2026). NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity. CVPR, 12966–12976.

2. Novelty and UBP Attribution (Critical Weakness 2)

We adopt UBP’s adaptive blur-aware alignment method (Wu et al., 2025) within our bidirectional generative framework to improve the alignment between EEG and image representations. Our main contribution is a bidirectional framework that connects EEG-to-image reconstruction and image-to-EEG generation through a shared visual representation space. Among the evaluated bidirectional baselines, BiGEM achieves the highest visual-retrieval accuracy and the strongest paired image-reconstruction performance (Tables 1–2). Our distinctive contribution is to jointly investigate information preservation and representational organization within this bidirectional framework. Frozen reciprocal transformations separate stimulus identity retained in the shared latent space from fidelity after image rendering, revealing where information is preserved or lost across generation stages (Sec. 4.3; Table 5). Controlled visual perturbations further examine how changes in the shared visual representation propagate into generated EEG, distinguishing perturbation-response magnitude from feature-specific spatiotemporal organization (Sec. 4.5).

Compared with UBP, BiGEM uses a 63-channel dual-branch EEG encoder and introduces a decoder-aware visual adapter to recover visual features suitable for image generation from the blur-aware shared representation. The framework further incorporates image-conditioned rectified-flow EEG generation and bidirectional cycle objectives to support reciprocal generation and consistency analysis.

References

Wu, H., Li, Q., Zhang, C., He, Z., & Ying, X. (2025). Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior. CVPR, 2246–2257.

Liu, X., Gong, C., & Liu, Q. (2023). Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. ICLR.

3. Related Work and Baselines (Question 1)

We have expanded our literature review to include NeuroBridge (AAAI 2026; W. Zhang et al., 2026), Autoregressive Visual Decoding from EEG Signals (ICLR 2026; Dai et al., 2026), SAMGA (2026 preprint; Jiang et al., 2026) These works are discussed in Section 2; baseline sources and comparison settings are summarized in Appendix F.3.

References

Zhang, W., Wang, S., Su, Y., Li, X., Zhang, C., & Zhong, S. (2026). NeuroBridge: Bio-Inspired Self-Supervised EEG-to-Image Decoding via Cognitive Priors and Bidirectional Semantic Alignment. Proceedings of AAAI, 40(21), 18028–18036. doi: 10.1609/aaai.v40i21.38863.

Dai, S., Xiao, H., Yu, S., & Ye, Q. (2026). Autoregressive Visual Decoding from EEG Signals. ICLR.

Yu, C., Shen, S., Li, X., & Li, Y. (2026). D²-FOSA: Dual-Diffusion Guided EEG-to-Image Reconstruction with Frequency-Oriented Semantic Alignment. CVPR, 26698–26710.

Jiang, L., She, Q., Xu, J., Xu, H., Wu, D., & Kuang, Z. (2026). Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval. arXiv preprint arXiv:2604.17782.

4. Unidirectional Control: Shared Perturbation Organization, Feature Specificity, and Information Preservation (Question 2)

We thank the reviewer for raising this important question. To determine which perturbation findings genuinely benefit from bidirectional modeling, we evaluated a ten-subject unidirectional Image-to-EEG Flow that removes semantic cycle supervision while retaining the original velocity-matching and waveform objectives. We compare this control with the original BiGEM Full Flow reported in our response to Reviewer WpMq (Tables R8–R13).

Table R1. Unidirectional versus cycle-constrained BiGEM Flow (10 subjects).

MetricUnidirectional FlowBiGEM Full Flow
Shared perturbation organization
Positive dose-response slopes90/9090/90
Shared-map Spearman ρ0.93520.8920
Shared occipital peak80–120 ms80–120 ms
Feature-specific perturbation identification (%)
Full-window Top-127.7837.78
Early-window Top-131.1152.22
Relative-map Top-155.5671.11
Severity-controlled Full Top-118.8928.89
Severity-controlled Early Top-126.6741.11
Perturbation-response predictability
Severity-only predictive R²0.84000.9022
Severity + Family + Interaction R²0.86340.9172
Family incremental ΔR²0.013540.0128
Interaction incremental ΔR²0.009900.0023
Cross-modal identity recovery
Shared-latent Top-1 (%)26.0097.20

The BiGEM column reproduces the original Flow results from the WpMq Q3 experiments; the unidirectional column reports the new control. Between-model differences are descriptive because paired statistics for the fixed historical BiGEM checkpoint are not available.

  1. The unidirectional model reproduces shared and feature-specific perturbation organization. The unidirectional Flow reproduces positive dose-response slopes across all 90 subject–feature combinations and the shared occipital peak at 80–120 ms, consistent with BiGEM (Tables R1 and R12; Sec. 4.5). All five perturbation-family identification readouts exceed chance (BH q ≤ 0.0008). Shared, feature-specific, and severity-controlled feature-specific maps are also reproducible across independent image splits (full-window split-half ρ = 0.9957, 0.7935, and 0.7676, respectively). Thus, these qualitative phenomena are not exclusive to bidirectional modeling.

  2. BiGEM shows higher descriptive feature-family discriminability. BiGEM achieves higher accuracy across all five identification readouts (Tables R1 and R11), particularly for early-window identification (52.22% vs. 31.11%) and relative-map identification (71.11% vs. 55.56%). This pattern persists after severity control in both the full window (28.89% vs. 18.89%) and early window (41.11% vs. 26.67%). These results distinguish the reproducibility of feature-specific patterns from their discriminability; the between-model differences remain descriptive because paired significance has not been established.

  3. BiGEM preserves more cross-modal stimulus information. BiGEM achieves higher held-out predictability of EEG response magnitude (R² = 0.9172 vs. 0.8634) and higher shared-latent stimulus identification (97.20% vs. 26.00%; Tables R1, R9, and R13). These findings are consistent with the role of semantic supervision in preserving stimulus-discriminative information along the Image→EEG→Visual return pathway. The shared-latent readout is aligned with the training objective and does not independently establish physiological fidelity.

Under the evaluated settings, BiGEM outperforms the unidirectional control in perturbation-family discriminability, response-magnitude predictability, and cross-modal stimulus recovery (Table R1). These results support the advantage of bidirectional semantic constraints in preserving recoverable visual information across EEG and image representations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.