acceptodds
Withdrawn from ICLR 2027

Uni-Neur2Img: Unified Neural Signal-Guided Image Generation, Editing, and Stylization via Diffusion Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Non-invasive Brain–Computer Interfaces (BCIs) provide a viable pathway for decoding visual perception, yet EEG-to-image translation remains constrained by low signal-to-noise ratio, temporal non-stationarity, and substantial trial- and subject-level variability. Treating EEG as a clean text-like prompt can expose the diffusion backbone to both signal and noise, motivating explicit control over neural conditioning. We propose Uni-Neur2Img, a unified Diffusion Transformer architecture and conditioning interface for generation, editing, and stylization, with separately fine-tuned task-specific weights. A dual-pathway Neural Adapter maps EEG to pooled and token conditioning spaces without discrete EEG-to-text decoding. The Neural Signal Injection Module (NSIM) combines adaptive LoRA, using EEG- and timestep-dependent gates and a rank schedule, with quality-aware cross-modal attention. A separate subject-adversarial objective regularizes EEG representations. A frozen T5/CLIP text teacher provides semantic supervision only in EEG+Text training and is omitted in EEG-only training and inference. We introduce EEG-Style, comprising 20,000 paired trials from five participants viewing 4,000 images across 1,000 styles, with a style-disjoint split. Experiments evaluate generation on EEGCVPR40, editing on LoongX, and stylization on EEG-Style. Results are reported as means over three random seeds. In pooled stylization experiments involving five participants, EEG-only conditioning outperforms text-only conditioning on five of seven metrics with the tested prompts, reducing Style Loss by 68.75% and Color Loss by 14.86%. Joint EEG+Text conditioning surpasses both individual modalities on six of seven metrics, with DINO as the exception. These findings support visually evoked EEG as style guidance and show complementary gains from EEG and text under the evaluated conditions.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG-Guided Visual Generation, Diffusion Transformers, Multimodal Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7xsnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a unified Diffusion Transformer (DiT) framework (called Uni-Neur2Img) built upon the FLUX.1-Kontext backbone to handle three neural decoding tasks: EEG-guided image generation, image editing, and image stylization. To facilitate neural conditioning without discrete EEG-to-text decoding, the authors introduce a dual-pathway Neural Adapter and a Neural Signal Injection Module (NSIM) that combines adaptive LoRA (with timestep- and EEG-dependent gates and rank schedules) with quality-aware cross-modal attention. Furthermore, the paper introduces a new dataset called EEG-Style that comprises 20000 paired EEG-image trials from 5 participants viewing images across 1000 styles with a style-disjoint split.

Critical strengths
  1. Investigating visually evoked EEG as a direct style guide for image stylization is an underexplored direction in Brain–Computer Interface (BCI) research.
  2. Employing state-of-the-art flow-matching and DiT architectures (FLUX.1-Kontext) yields higher visual fidelity compared to older GAN- or U-Net-based BCI decoders.
  3. It evaluates across three distinct tasks (generation, editing, stylization) via multiple quantitative metrics across repeated random seeds.
Critical weaknesses
  1. Section 3 states that “The four images sharing a style formed a block, with a 2 s rest between blocks and extended breaks... Style-disjoint splitting alone does not rule out all session or stimulus confounds.” In EEG research, block-design methodologies are notoriously prone to low-frequency physiological drifts, sustained attention/fatigue states, and block-level event-related potentials (ERPs) instead of true fine-grained stylistic neural representations. If the model exploits block-level baselines or slow temporal non-stationarities, the reported gains in "style decoding" may reflect experimental artifacts rather than genuine neural decoding of artistic style. Hence control experiments like shuffled/permuted block label tests (e.g., training the model on randomly scrambled block assignments to verify whether performance collapses) should be conducted. Feature attribution or frequency-band probing should be performed to show that the model relies on actual stimulus-evoked neural responses rather than slow session drifts. Moreover, the authors should re-evaluate with cross-session splits where style blocks are interleaved instead of grouped.

  2. The entire empirical evaluation on the newly introduced EEG-Style dataset relies on a cohort of only 5 participants. Furthermore, the Leave-One-Subject-Out (LOSO) ablation (Table 4) reveals that while CLIP-I is preserved, other metrics degrade significantly when generalizing to unseen subjects. This indicates that semantic alignment alone is insufficient for robust zero-shot stylization across subjects. Given the massive inter-subject variance in EEG signals (caused by skull thickness, electrode positioning, and individual neuroanatomy), 5 participants are insufficient to substantiate claims of generalizability or real-world BCI applicability. Hence it is necessary to expand data collection to a substantially larger participant pool (ideally 20+ subjects) to validate cross-subject robustness.

  3. In the stylization and editing tasks, the model receives both the source image via the VAE/backbone and the target-evoked EEG. Because the source image provides the exact content structure, and FLUX-Kontext is a powerful in-context editing model, it remains ambiguous how much heavy lifting the EEG signal is actually performing versus the backbone's strong image-to-image priors. The ablation table (Table 4) shows that removing NSIM or the text teacher still yields competitive performance, raising questions about whether EEG acts as a primary style guide or only a weak regularizer. Hence a "random EEG" or "shuffled EEG" ablation test during inference for stylization should be conducted to quantify the exact performance drop when conditioning on uncorrelated brain signals. Moreover, clearer quantitative disentanglement should be provided to show how much variance is explained by EEG conditioning versus source-image conditioning alone.

  4. NSIM combines three distinct mechanisms: adaptive LoRA with EEG/timestep gates, a timestep-dependent rank schedule, and a quality-aware cross-modal attention bias based on spectral/amplitude heuristics. However, Table 4 indicates that individual components have mixed or offsetting effects (e.g., timestep-aware routing shows mixed effects; quality-aware conditioning helps semantic similarity but affects other metrics). The entire architecture is over-engineering with marginal ablation gains. Worse still, all the components are not new, and the combination of these components yields only incremental novelty to methodology.

Questions for authors

see Weaknesses

Reviewer_3tgEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

They introduce:

  • Uni-Neur2Img is a framework that conditions frozen FLUX diffusion transformers on EEG signals for image generation, manipulation and stylization. Uni-Neur2Img integrates an EEG encoder, a two-path adapter to map EEG features to token and pooled conditionings of the backbone, and a Neural Signal Injection Module comprising gated LoRA branches and quality-aware attention. Uni-Neur2Img employs a unified architecture across tasks, but with task-specific parameterization. It is trained with flow matching and where appropriate, subject-adversarial learning, with a frozen T5/CLIP text teacher providing alignment targets in the EEG+Text setting.
  • EEG-Style: A stylization dataset comprising 20,000 trials by five subjects from the same 4,000 stylized images, grouped by 1,000 styles. Training and testing styles are disjoint. On the multi-subject stylization task, Uni-Neur2Img conditioned on EEG-only achieves improvement over the text-only config tested by the authors in five of seven metrics, and a config conditioned on EEG+Text outperforms either modality alone in six of seven metrics.
Critical strengths
  • EEG-Style is a promising direction for the study of visual appearance signals in visually evoked EEG. It is also a nice design choice to assign each style group as a whole to either training or testing, so no target image or style appears in both. The dataset can be useful for further research on EEG-based image stylization when made available with appropriate documentation.

  • It performs a variety of experiments on three tasks, including ablations, multiple random seeds, and leave-one-subject-out evaluation. I also appreciate the authors clarifying the scope of claims made: i.e., that the paper investigates image-evoked response and does not make a claim about decoding imagined imagery, or that the quality scalar used is heuristic.

Critical weaknesses
  1. In EEG-Style, the model receives the source image and is trained toward a paired stylized target. The gains could therefore come from a learned source-to-stylized mapping that makes little use of the EEG. The current experiments do not rule this out.

Two controls are needed:

  • The trained model evaluated with mismatched EEG (swapped across style groups, same source image and seed).
  • A model trained on the source image alone, without informative EEG.

The text comparison is also weak. The exact prompts and their use across methods are not sufficiently specified. IP-Adapter (Text Only) reaches a Style Loss of 0.0854, below the authors' own text-only model (0.2045 / 0.2935) and their single-subject EEG-only model (0.2681). However, the pooled EEG-only model achieves a lower Style Loss of 0.0639. The comparison therefore supports the pooled EEG-only result on this metric, but does not establish a general advantage of EEG over text without clearer specification of the text baselines.

The same problem appears in the other tasks:

  • Editing: Text-only beats EEG-only on all metrics (Table 7). The best setting, EEG+Speech, contains the spoken instruction, but no clearly identified quantitative speech-only result is reported. Appendix D.2 discusses a Speech-Only baseline, although the accompanying figure labels are inconsistent.
  • Generation: EEGCVPR40 has known block-design confounds (Li et al., 2021), and the gains over GWIT also include a different, stronger backbone (FLUX).
  1. NSIM combines established mechanisms, including gated LoRA branches, timestep-dependent rank scheduling, and quality-aware attention. The contribution therefore depends on showing what this combination adds. Table 4 shows that the full configuration improves several metrics, but the individual mechanisms have mixed effects, and timestep-aware gating produces limited gains over the no-NSIM baseline. The single-to-dual-pathway comparison shows a substantial improvement, yet the paper does not specify which pathway the single-pathway variant retains. A comparison with and without NSIM, keeping the full dual-pathway adapter and text supervision fixed, would help establish NSIM's contribution.
Questions for authors
  • Can you evaluate the comparison between matched and shuffled EEG on EEG-Style using the same source image and random seed, as well as a baseline trained on the source image only, without EEG. If matched EEG performs clearly better than both, I will increase my rating.
  • Can you explain the discrepancy between the results on the two text-only rows in Table 3, which share five values and confidence intervals but have different Style Loss and Color Loss. Please verify the correct values and adjust the percentage drops if necessary.
  • How do you create text prompts for Uni-Neur2Img and IP-Adapter? Do you use the text descriptions from Qwen directly? What is the input of IP-Adapter baseline?
  • Can you provide the quantitative results of the speech-only case in Uni-Neur2Img.
  • Will you release the EEG-Style? What is the intended license? Do you have the rights to share the source and target images from Midjourney?
Reviewer_qimjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The author presents Uni-Neur2Img, a shared EEG-conditioning architecture for image generation, editing, and stylization, using separately fine-tuned task-specific weights. A dual-pathway Neural Adapter maps EEG into pooled and token-level interfaces of frozen FLUX backbones. The Neural Signal Injection Module combines condition- and timestep-dependent low-rank adaptation with quality-aware attention, while text-teacher supervision is used only in the joint EEG+Text setting. The authors introduce EEG-Style, containing 20,000 trials from five participants viewing 4,000 images across 1,000 style groups, and hold out complete style groups across participants. Experiments report improvements over selected baselines and complementary benefits from EEG and text under the evaluated conditions.

Critical strengths
  1. EEG-Style and the associated stylization task are potentially valuable contributions. Holding out complete style groups across all participants prevents a test target or style group from entering training through another participant’s recordings. The study also appropriately identifies its scope as paired stylization guided by visually evoked EEG rather than decoding imagined images, emotions, or unconstrained aesthetic intentions. This provides a useful setting for investigating appearance-related information in neural recordings.
Critical weaknesses
  1. The submission is incomplete and not self-contained: the uploaded PDF ends immediately after the References (and carries an ICLR 2026 running header throughout), omitting all referenced appendices (Appendices A–H) as well as Table 7 cited in the editing section.

  2. The experimental comparisons on LoongX and EEG-Style also suffer from unaddressed confounds and unexplained settings. On LoongX, the methodology defines encoders solely for multichannel EEG and optional text prompts, yet the editing table reports results for Neural Signals (which in LoongX includes fNIRS, PPG, and head motion) and Neural+Speech (which includes speech audio) without explaining how non-EEG biosignals or speech audio are encoded. On EEG-Style, the central claim that EEG Only outperforms Text Only cannot be properly evaluated because the text prompts supplied to Text Only are never specified, the Text Only baseline inexplicably changes its Style and Color loss values between the single-subject and multi-subject blocks even though it uses no EEG, and there is no shuffled or null EEG control to verify that the high-rank LoRA adapters are decoding trial-specific EEG signals rather than simply learning an unconditional Midjourney style prior from the training pairs.

Questions for authors
  1. Why does the Text Only baseline in the stylization table have different Style Loss and Color Loss values between the single-subject and multi-subject blocks when it takes no EEG input, and what exact text prompts were supplied to Text Only?

  2. Please provide the missing mathematical definitions for the quality proxy, the LoRA rank schedule, the stylization evaluation metrics, and the contents of Table 7.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.