- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
BiGEM connects EEG-to-image reconstruction and image-to-EEG generation through a shared OpenCLIP visual representation. A subject-specific EEG encoder is trained using blur-aware visual alignment, a distilled adapter conditions frozen SDXL, and a subject-specific rectified-flow model generates PCA coefficients that a fixed decoder maps to EEG. The paper evaluates frozen bidirectional compositions and a separate latent-cycle refinement stage, distinguishing returned-latent identity from rendered-output fidelity. It also analyzes learned representations and measured EEG and probes generated responses using nine visual perturbation families. On THINGS-EEG2, it reports 49.30% Top-1 in 200-way retrieval from measured EEG averaged over 80 repetitions and a correlation of approximately 0.456 between generated and measured mean responses. The main analytical claims concern information preservation across transformations and the separation of perturbation-response magnitude from spatiotemporal organization.
Critical strengths
1. A potentially useful analytical framework with informative controls. The study goes beyond optimizing a reconstruction score by separating direct generation, frozen compositions, and joint refinement. Table A2 includes a deterministic predictor, mismatched conditioning, and an ATM-S readout not used to train the EEG generator. These controls provide evidence of stimulus-information retention and reveal meaningful trade-offs. The authors explicitly distinguish the high shared-latent readout from independent evidence of EEG realism.
2. Careful treatment of several statistical and interpretive issues. The appendix describes subject-level inference, multiplicity correction, matched generation noise, image-grouped cross-validation for magnitude prediction, and exclusion of the query subject from the strict severity-adjustment and template-fitting procedure. It also distinguishes primary and secondary analyses and acknowledges that generated perturbation responses do not establish causal effects in humans. These are substantive strengths, although they do not resolve the concerns below.
Critical weaknesses
1. The headline retrieval result has an unresolved candidate-side dependency on test EEG, so its interpretation and comparability are not established.
Appendix A.2.6 (p. 16) states that candidate scalars are obtained from fixed modulation networks applied to features of real test-EEG means, that modulation is applied twice, and that retained amplitude modulation produces an EEG-assisted gallery rather than plain cosine retrieval. The unresolved issue is whether candidate image j is scaled using its paired test response. If so, constructing the gallery requires candidate-side neural information and its image correspondence, beyond the current EEG query and an image-only candidate set. Using a fixed gallery for every query does not remove this dependency. Candidate-specific scaling can change rankings; by contrast, a common positive scalar applied to all candidates for one query cannot. The paper needs an indexed scoring equation or executable scoring routine that resolves this distinction, including the normalization order and the source of each scalar.
The decisive control is to evaluate the same trained checkpoint and the same 80-repeat EEG queries against the same 200 images, but with candidate representations constructed without their paired test EEG. This is not answered by Table A3b's comparison of nonblurred, fixed-medium, and adaptive training targets: that comparison does not isolate a change to the evaluation rule for a fixed encoder. Table 1 should then compare methods under explicitly matched information access, with paired subject-level differences. The variability reported in Table A4 is useful, but does not resolve this protocol issue. My concern is not that 49.30% is intrinsically implausible, and I cannot determine the magnitude or direction of any scoring effect without this control. The current text establishes a test-response dependency requiring clarification, not that test EEG entered gradient training or that the results were fabricated.
The approximately 97% readouts must also remain separate from measured-EEG retrieval. They start from a known image and assess whether its identity can be recovered from generated EEG in a representation that also supplies the generator's semantic supervision (A.2.4). Appendix A.2.6 explicitly states that the 97.15% returned-latent result uses the original normalized image gallery, so I do not attribute that result to candidate modulation. Nevertheless, it is not independent evidence of realistic neural responses. The authors recognize this limitation; the 13.55% ATM-S result in Table A2 is a more independent identity check, although the difference between the two readouts does not by itself quantify artificial inflation.
Finally, the comparisons need a traceable evaluation specification. Tables 2-3 point to A.2.6 for sources, but that section does not provide a baseline-by-baseline account of quoted versus rerun values and matched preprocessing, repetition/sample aggregation, and metric settings. Table 3 is explicitly contextual and cannot substitute for a controlled comparison. The source of the frozen modulation-network weights and the exact retrieval implementation are particularly important here. This assessment is limited to the PDF; it does not assert that no separate implementation package exists.
2. The distinctive contribution of bidirectional refinement and stochastic perturbation probing is not isolated from simpler explanations.
Attribution of joint-refinement gains. Appendix A.2.5 and Table A4 compare independent, 1,000-update warm-up, and 2,000-update cycle-active endpoints. Training duration and cycle activation covary, as the authors acknowledge on p. 31. The observed gains establish improvement after the stated refinement procedure, but do not isolate the cycle objective. Direct image-to-EEG correlation already rises from 0.4555 to 0.4747 during warm-up and is 0.4702 at the cycle-active endpoint; direct EEG retrieval changes from 49.30% to 48.85%. These values are not evidence of significant deterioration, but an equal-update, cycle-disabled continuation from the same initialization is needed to attribute the gains. The absence of a direct-decoding improvement is not itself grounds for rejection; the problem is identifying what the added mechanism contributes.
Independent support for realistic stochastic responses. Table A2 shows higher mean-response correlation for the deterministic predictor than for the flow (0.5967 versus 0.4555). Its ATM-S accuracy is 12.60% versus 13.55% for the flow, with an uncorrected paired p=0.218750. This does not establish equivalence, and the mismatched-condition control provides useful evidence of stimulus dependence. However, a distinctive benefit of stochastic generation needs evidence about response distributions, not simply the ability to produce different samples. The near-unit dispersion ratio measures mean pairwise distances in a retained, standardized PCA subspace; it does not establish correct conditional covariance, temporal/spectral structure, or variability in discarded dimensions. A relevant control is the existing deterministic predictor augmented with residual variability estimated only from training data, assessed with independent variability diagnostics.
Added value and neural correspondence of the probing results. The magnitude/pattern distinction may also arise in simpler image-conditioned mappings. As a mathematical illustration, a linear predictor f(z)=Wz+b produces a response change W(z_pert-z): magnitude may track visual displacement while response patterns retain displacement-direction information. This is an alternative explanation, not an empirical finding about BiGEM. Applying the same perturbation analysis to the existing deterministic predictor would help establish what the bidirectional generative framework adds. Similarly, the intermediate-layer RSA profile should be compared with that of the blur-aware visual target itself, since EEG representations are explicitly trained toward that target.
The measured-EEG correspondence in Figure A19 (p. 48) further limits the broader interpretation: all nine confidence intervals for global same-feature alignment advantage include zero, and significant positive restricted preference is found for edge orientation, but not spatial phase or texture. This is not proof of absent correspondence, but broad feature-specific agreement between generated perturbation patterns and measured EEG is not established. The paper appropriately avoids a human-causal claim; even under its descriptive framing, these mixed results should be central to the interpretation. The strict residual-pattern analysis also holds out subjects while averaging the same image set for queries and templates (A.4.8); it should be distinguished from the separate image-held-out magnitude analysis and from generalization of the perturbation patterns to new stimuli.
Overall, the concern is not merely that existing modules are combined. The potentially valuable contribution lies in the framework and its empirical findings, but the present controls do not sufficiently distinguish those findings from evaluation-specific effects, additional optimization, or simpler conditional predictors. I therefore lean toward rejection in the current form. An auditable image-only-gallery evaluation and focused controls of the existing claims could materially change this assessment.
Questions for authors
1. Retrieval information boundary and fixed-checkpoint control. Does candidate j use its paired real test EEG to obtain either modulation scalar? Please provide the fully indexed scoring equation or code, the two modulation steps, and the normalization/broadcasting axes. Using the same trained encoder and test queries, what are Top-1, Top-5, and MRR with an image-only gallery and no candidate-side test-EEG calibration? Please include paired subject-level changes and explicit image IDs in the ranking outputs.
2. Equal-update cycle control. Are direct-only continuations available with the same initialization, update count, optimizer schedule, and sampling as the cycle-active models? How do cycle and independent readouts compare under this matched control? The existing 1,000-update warm-up endpoint does not isolate the effect at 2,000 updates.
3. Stochastic fidelity and probing-specific value. Can the existing deterministic predictor, with and without training-derived residual variability, be evaluated using the same independent response-distribution diagnostics and perturbation-family analysis? Which principal findings distinguish the flow-based framework from these simpler alternatives?
4. Interpretation of neural correspondence. How should the broad representational conclusion be qualified in light of Figure A19's global intervals and restricted results? Does the intermediate-layer RSA pattern differ from that of the visual alignment target? Please distinguish cross-subject identification on the same stimuli from the separate image-held-out magnitude prediction result.
5. Reproducibility and comparison provenance. Please identify the scoring implementation, modulation-network provenance, adapter loss definitions and numerical weights, and the configurations that produced the principal results. Which baseline values were rerun versus quoted, and how were information access, preprocessing, repeat/sample averaging, and metric computation matched? These are clarification requests about the reported experiments, not a request for a new dataset or clinical study.
6 comments
Response to Q2: Equal-Update Cycle Control
We thank you for raising this important concern. We clarify our training protocol and provide a new equal-update control across ten subjects to isolate the contribution of cycle consistency losses.
Training protocol. Our Independent (0) condition consists of two separately pretrained unidirectional generators composed without additional joint training. The Warm-up (1000) stage performs 1,000 joint optimization updates using only direct-task objectives, without cycle losses. The original Cycle (2000) stage continues from this checkpoint for another 1,000 updates, introducing both image-start and EEG-start cycle consistency losses.
Equal-update control. To disentangle the effect of cycle losses from additional optimization, we introduce an extended Warm-up (2000) control. Both Warm-up (2000) and Cycle (2000) start from the same complete Warm-up (1000) checkpoint and receive exactly 1,000 additional updates, sharing identical initialization, optimizer states, learning-rate schedules, and pre-specified sampling inputs. The only intended difference is the inclusion of cycle losses.
Results. Relative to Warm-up (2000), Cycle (2000) has higher mean image-cycle cosine (0.2992→0.3114) and EEG-cycle Pearson correlation (0.4050→0.4396), and lower EEG-cycle MSE (0.0968→0.0719; Table R7). These matched-update comparisons support a contribution of the cycle losses beyond continued direct-task optimization.
Table R7. Matched-update cycle control (mean ± SD, n=10).
| Metric | Historical Independent (0) | Warm-up (1000) | Warm-up (2000) | Cycle (2000) |
|---|---|---|---|---|
| Image-cycle cosine ↑ | 0.2707 ± 0.0150 | 0.2887 ± 0.0154 | 0.2992 ± 0.0162 | 0.3114 ± 0.0174 |
| EEG-cycle Pearson ↑ | 0.4014 ± 0.0570 | 0.4035 ± 0.0551 | 0.4050 ± 0.0547 | 0.4396 ± 0.0630 |
| EEG-cycle MSE ↓ | 0.1021 ± 0.0523 | 0.0977 ± 0.0506 | 0.0968 ± 0.0518 | 0.0719 ± 0.0332 |
| Direct EEG→Visual Top-1 (%) ↑ | 48.90 ± 5.77 | 48.85 ± 5.45 | 48.50 ± 4.89 | 48.45 ± 4.69 |
Values are mean ± SD across ten subjects. The matched-update comparison is Cycle (2000) versus Warm-up (2000). Historical Independent (0) values are provided for reference and are excluded from this comparison.
Independent readout control. We further evaluate all four stages using the same frozen independently pretrained EEG reader (Table R8). Compared with the equal-update Warm-up (2000) control, Cycle (2000) increases mean image-cycle cosine similarity from 0.2770 to 0.3010, Cycle-A latent Top-1 from 96.05% to 97.25%, and Cycle-B latent Top-1 from 23.40% to 25.30%. These higher mean scores provide additional evidence of improved cycle consistency under a fixed, independently trained readout.
Table R8. Cycle consistency under a fixed independent EEG reader (10 subjects).
| Metric | Independent (0) | Warm-up (1000) | Warm-up (2000) | Cycle (2000) |
|---|---|---|---|---|
| Image-cycle cosine ↑ | 0.2707 ± 0.0150 | 0.2756 ± 0.0153 | 0.2770 ± 0.0160 | 0.3010 ± 0.0160 |
| Cycle-A latent Top-1 (%) ↑ | 95.65 ± 1.97 | 96.85 ±1.97 | 96.05 ± 1.74 | 97.25 ± 1.53 |
| Cycle-B latent Top-1 (%) ↑ | 21.65 ± 4.39 | 23.10 ± 4.53 | 23.40 ± 5.07 | 25.30 ± 5.57 |
Overall, the matched-update experiments support improved mean bidirectional consistency with cycle losses beyond continued direct-task optimization. Section 4.3 and Appendix C.2 (Table A4) now report the updated matched-update results and the direct-task trade-off. The fixed independent-reader control is provided in Table R8 of this response.
Response to Q4: Neural Correspondence and Representational Interpretation
Neural correspondence (Figure A19). We agree that our findings should not be interpreted as establishing broad feature-specific correspondence between generated and measured EEG. Figure A19 shows that all nine global same-feature alignment advantages have 95% confidence intervals including zero. The restricted analysis provides significant positive evidence for edge orientation (Holm-adjusted p=0.00293), but not spatial phase or texture.
Importantly, our study distinguishes three complementary levels of evidence: visual-feature associations independently observed in measured EEG (Fig. 3), perturbation-family information recoverable from generated EEG after severity control (Figs. 7 and A18b), and direct correspondence between generated and measured response patterns (Fig. A19). The first two findings do not imply global feature-wise agreement in the third.
We therefore limit our conclusion to representational structure observed within each domain, with partial and feature-specific evidence of correspondence across generated and measured EEG, rather than claiming comprehensive neural correspondence. This interpretation is now explicit in Section 4.5 and the Conclusion.
2. Intermediate-layer RSA versus visual alignment targets (Tables R15–R16). To directly examine whether the intermediate-layer preference is inherited from visual supervision, we conducted a matched RSA analysis using a newly trained ViT-H/14-aligned EEG encoder.
As shown in Table R15, the EEG representation peaks at ResNet-50 (He et al., 2016) Layer3 (ρ=0.3361), whereas its medium-blur ViT-H/14 (Cherti et al., 2023) target peaks at Layer4/AvgPool (ρ=0.2804). EEG exhibits a stronger intermediate-layer preference (ΔH=0.1538). EEG–Layer3 correspondence remains positive after controlling the visual target RDM (partial ρ=0.2955) and after additionally controlling nine visual-descriptor RDMs (partial ρ=0.2362). These results indicate that the EEG representation's hierarchical profile is not simply identical to its visual supervision target.
Table R15. Layer-wise RSA for EEG representations and their visual alignment target.
| ResNet-50 Layer | EEG Representation RSA | Visual Target RSA |
|---|---|---|
| Conv1 | 0.093624 | 0.139149 |
| Layer1 | 0.225011 | 0.126977 |
| Layer2 | 0.310381 | 0.175631 |
| Layer3 | 0.336146 | 0.227842 |
| Layer4 | 0.189673 | 0.280388 |
| AvgPool | 0.189673 | 0.280388 |
We further examined whether this preference depends on blur-aware training. Table R16 compares matched no-blur, fixed-medium, and adaptive-blur EEG encoders. All three peak at Layer3, with nearly identical Layer3 RSA values (0.3383, 0.3362, and 0.3361) and similar intermediate-layer preferences (H=0.1454, 0.1487, and 0.1488). Thus, the intermediate-layer preference is not unique to adaptive blur-aware alignment. We do not attribute this preference specifically to adaptive blur or claim that the observed geometry establishes a distinct neural mechanism.
Table R16. Intermediate-layer RSA across blur conditions.
| Metric | No-blur | Fixed-medium | Adaptive |
|---|---|---|---|
| Layer3 RSA ↑ | 0.338332 | 0.336200 | 0.336146 |
| Intermediate preference H ↑ | 0.145438 | 0.148711 | 0.148768 |
3. Distinct Generalization Settings
We clarify that these analyses evaluate two distinct generalization settings.
Perturbation-family identification (Appendix A.4.8; Fig. A18b) uses leave-one-subject-out evaluation. Templates are constructed from the other nine subjects, while the query subject is excluded from severity-model fitting. Both templates and queries aggregate responses over the same 200 stimulus images. Accordingly, this result supports cross-subject identification of perturbation families over a shared stimulus set; it does not test perturbation-family identification on held-out images.
Magnitude prediction (Appendix A.4.6) uses fivefold image-grouped cross-validation, holding out entire image identities together with all associated perturbation conditions. The held-out R² is 0.9022 using severity predictors and 0.9172 after adding family identity and family-by-displacement interactions. These results support response-magnitude prediction for images held out from the regression fit, but do not establish image-held-out perturbation-family identification.
Section 4.5 now explicitly distinguishes these two evaluation settings and their conclusions.
References
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. CVPR, 770–778.
Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., & Jitsev, J. (2023). Reproducible Scaling Laws for Contrastive Language-Image Learning. CVPR, 2818–2829.
We thank you for your detailed review and constructive suggestions on evaluation, experimental controls, and reproducibility. Your comments helped us identify and correct the candidate-side EEG dependency in retrieval and clarify the evidence supporting our scientific conclusions. Below, we address your questions point by point and describe the corresponding corrections, additional analyses, and manuscript revisions.
Response to Q1: Retrieval information boundary and fixed-checkpoint control
We confirm that the original 49.30% Top-1 retrieval accuracy was affected by test-time data leakage, as candidate image representations were calibrated using their paired real test EEG. We corrected the evaluation protocol by removing candidate-side EEG calibration while keeping the trained model weights, test EEG queries, and 200-image candidate set unchanged. Retrieval now uses an image-only gallery and cosine similarity between EEG and image embeddings.
Across ten subjects, the corrected image-only evaluation yields 48.90% Top-1, 78.55% Top-5 and 0.6222 MRR. Relative to the superseded scoring rule, the paired changes are −0.40 percentage points, 0.00 percentage points and −0.002819, respectively.
None of these differences was statistically significant after BH-FDR correction (Top-1: q = 0.281; Top-5: q = 1.000; MRR: q = 0.123). Table R5 provides the paired subject-level changes.
Table R5. Corrected image-only retrieval and paired changes. Δ = corrected − original; pp denotes percentage points.
We further examined all 2,000 paired queries. Sixteen queries changed Top-1 correctness after correction: four changed to correct and twelve changed to incorrect, giving a net loss of eight correct queries (0.40 pp). Table R6 lists all corresponding stimulus image IDs.
Table R6. Stimulus IDs for queries whose Top-1 correctness changed between galleries.
These results demonstrate that removing candidate-side EEG calibration produces only a small observed performance difference, with no statistically significant change after multiple-comparison correction. Although this does not establish statistical equivalence, it indicates that the strong retrieval performance is largely retained without candidate-side test-EEG information.
Manuscript revisions. We have corrected the retrieval evaluation in Sections 4.1–4.2 and replaced the previous headline result with the corrected image-only result in the Abstract, main text and Table 1. Appendix A.2.6 documents the original and corrected scoring protocols and retains the previous EEG-assisted metrics alongside the corrected metrics for historical documentation and comparison. All retrieval readouts affected by the original scoring rule, including those in the alignment ablations and stage-specific evaluations, have been re-evaluated under the corrected protocol. The corresponding retrieval metrics, statistical comparisons, and descriptions have been updated accordingly. Other experimental results that do not depend on this scoring rule remain unchanged.