acceptodds
Under review as a conference paper at ICLR 2027

Learning Visual Corrections from EEG

Abstract

EEG recorded during visual perception carries information about viewed objects and their attributes, offering a basis for neural guidance of image editing. The challenge is to translate this information into precise semantic and appearance changes while preserving unrelated source content. We introduce EEGRAFT, which learns to align neural and visual representations to guide image editing from EEG. The editor’s source-conditioned visual states query the aligned EEG summary and unpooled features through separate attention pathways, adapting neural guidance throughout generation through time-gated residual updates. At inference, the editor requires only a source image and EEG recorded while viewing the target, without access to the target image, text instructions, or edit masks. To support systematic study of this task, we construct Neve, a benchmark covering appearance and semantic editing with 29,275 source–target pairs and 2.71 million EEG trials, by generating source images while preserving the measured correspondence between the original viewed stimuli and their EEG recordings. Our method outperforms six baselines across all six aggregate metrics, reducing C-LPIPS by 22.7% and increasing the recovery–preservation score R-Edit from 0.386 to 0.585 relative to the strongest baseline for each metric. With the editor and source image fixed, correctly paired EEG produces more accurate edits than mismatched or disabled EEG conditioning. We hope this work advances the use of neural signals for visual content creation and interaction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
EEG, Image editing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Z2YvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EEGraft edits a source image toward a target using EEG recorded while a participant viewed the target, with no text or masks at inference. A LaBraM encoder aligned to ViT-H features is read by an OmniGen2 editor through two gated cross-attention paths (pooled summary and unpooled tokens), and an auxiliary loss matches the EEG-induced change to the source-target latent difference. The NEVE benchmark (29,275 pairs from three EEG datasets) uses the real stimuli as targets and Gemini-edited versions as sources. EEGraft beats six neural editing baselines on all six metrics, and correct EEG beats null and mismatched EEG.

Critical strengths
  1. The evaluation is designed with care. Change-region metrics and R-Edit, which gives 0.5 to copying the source, stop whole-image scores from rewarding an edit that does nothing.
  2. The fixed-editor controls are the right experiment. Correct EEG beats mismatched EEG from the same and from a different participant on all six metrics (C-LPIPS 0.512 to 0.479, C-MAE 0.245 to 0.197), so the recordings do carry some target information.
Critical weaknesses
  1. The baseline comparison says little about EEG. All six baselines score below source copying on R-Edit (0.275 to 0.386 against 0.5), and LoongX and MindCustomer return blurs and noise in Figure 4. EEGraft with EEG switched off already beats every baseline on every metric (C-LPIPS 0.520 against 0.620), so the 22.7% headline mostly reflects OmniGen2 and in-domain training. The part due to correct EEG is 0.512 to 0.479 in C-LPIPS. The source-only editor (Table 5, R-Edit 0.480) also looks undertrained next to the null-EEG model (0.521).
  2. The benchmark may not test target-specific decoding. Sources are Gemini edits of the target photo, so the editor learns to undo one synthetic recolor or replacement. Neither mismatch control holds the concept fixed, so the gap may reflect category information rather than the target's appearance, which is what appearance edits need. On the 200 official THINGS images R-Edit is 0.527, barely above copying.
Questions for authors
  1. Clarification: was the source-only editor trained with the same steps, losses and LoRA settings as EEGraft, and why does it fall below copying?
  2. Experiment: Correct versus Mismatched gaps per subset with confidence intervals over participants, plus a same-concept mismatch control.
  3. Clarification: is the Table 16 retrieval single-trial or averaged, and is each edit conditioned on one trial?
  4. Experiment: a decode-then-edit baseline on the same backbone (EEG to ViT-H embedding, then OmniGen2 image conditioning) would show whether the dual readout matters.
5 comments

We thank the reviewer for the careful reading and detailed feedback, particularly the suggestions for controlled comparisons that we have incorporated into the experiments and analyses below.

W1 — Baseline quality and the contribution of EEG

Thank you for highlighting the concerns about baseline quality and the contribution of EEG. All six baselines were adapted to target-viewing EEG on NEVE and fully optimized for both appearance and semantic editing under a common evaluation protocol, preserving their core editing mechanisms (Appendix L).

Source images provide scene structure and content to preserve, which OmniGen2 continues to use through its NEVE-trained source-conditioning pathway when EEG is disabled. External methods also receive the source, but their editing mechanisms may be less effective at translating target-related EEG into precise local corrections while preserving unrelated content. For example, Brain-Supervised, the strongest external baseline on R-Edit at 0.386, maps EEG-derived attributes to source-conditioned updates. Attribute-level guidance can leave spatial and appearance details underspecified; edit-magnitude calibration adjusts update strength without supplying those details. R-Edit exceeds the source-copy reference of 0.5 only when target recovery outweighs additional preservation error within each editing subset. Figure 5 evaluates both sides of this balance and shows that EEGraft achieves stronger target recovery and less additional error in low-change regions than the external methods.

The reported 22.7% C-LPIPS reduction, from 0.620 to 0.479, compares complete editing systems. With the same OmniGen2 backbone, EEG representations and training protocol, and matched readout capacity, separate readouts reduce C-MAE by 13.2% and raise R-Edit by 0.056 relative to concatenation. Q4 further reports the requested decode-then-edit comparison using the same backbone and EEG decoder.

We then hold the learned editor fixed to measure the information supplied by the target's EEG at inference. Null, Mismatched, and Correct EEG use the same full-model checkpoint within each task, source images, and evaluation protocol. Null disables both EEG readouts; Mismatched replaces the correct recording with EEG from another stimulus viewed by the same participant.

\hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Null EEG}&0.310&0.520&0.242&0.386&0.696&0.521\\\\ \text{Mismatched EEG}&0.307&0.512&0.245&0.383&0.692&0.532\\\\ \text{Correct EEG}&\mathbf{0.278}&\mathbf{0.479}&\mathbf{0.197}&\mathbf{0.420}&\mathbf{0.752}&\mathbf{0.585}\\\\ \hline \end{array}$$ Correct pairing reduces C-LPIPS by **6.4%** and C-MAE by **19.6%** relative to Mismatched EEG, while increasing R-Edit by **0.053**. Relative to Null, C-MAE falls by **18.6%** and R-Edit rises by **0.064**, demonstrating better target recovery and recovery–preservation balance with the source and learned editor fixed. Task-wise results and participant-level confidence intervals are reported in Q2. The source-only editor uses the same OmniGen2 initialization and is fully trained with matched data, training budgets, LoRA settings, and optimization protocols (Q1). It retains the target-recovery objectives and sets $\lambda_\Delta=0$, since prediction-difference supervision requires an EEG condition to switch on or off. Null instead retains the shared LoRA learned during EEG-conditioned training. To distinguish these training effects, we compare Source-only, No-Δ and Full with EEG disabled at inference; No-Δ removes $L_\Delta$ while keeping all other training settings unchanged. $$\begin{array}{lrrr} \hline \text{Model without EEG at inference}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{R-Edit ↑}\\\\ \hline \text{Source-only}&0.632&0.317&0.480\\\\ \text{No-Δ, Null EEG}&0.641&0.305&0.518\\\\ \text{Full, Null EEG}&0.520&0.242&0.521\\\\ \hline \end{array}$$ Even without $L_\Delta$, the conditionally trained editor reaches R-Edit of **0.518** with EEG disabled, compared with **0.480** for Source-only. Adding $L_\Delta$ further reduces Null C-LPIPS by **18.9%** and C-MAE by **20.7%**, while increasing R-Edit from **0.518 to 0.521**. These gains persist because Null uses the learned shared LoRA; stop-gradient blocks the difference loss through the EEG-disabled prediction, but the conditioned prediction still updates those shared parameters. Conditional training therefore produces a retained R-Edit advantage, and difference supervision further improves target-region recovery. ---

W2 — Target-specific EEG information and benchmark construction

Thank you for raising the concern about target-specific decoding. To address your concern about source-image regularities, we replaced Gemini with gpt-image-2.5 under the same construction protocol, which constrains edits to be plausible and preserves unrelated content (Appendix A). We kept the target–EEG pairs unchanged and evaluated the same trained editor without fine-tuning, holding source images fixed across EEG conditions within each task.

\hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Null EEG}&0.322&0.531&0.251&0.381&0.687&0.517\\\\ \text{Mismatched EEG}&0.313&0.527&0.255&0.384&0.691&0.526\\\\ \text{Correct EEG}&\mathbf{0.280}&\mathbf{0.476}&\mathbf{0.194}&\mathbf{0.422}&\mathbf{0.748}&\mathbf{0.583}\\\\ \hline \end{array}$$ Correct EEG outperforms both controls, reducing C-MAE by **23.9%** relative to Mismatched EEG and **22.7%** relative to Null. Its R-Edit remains **0.583**, compared with **0.585** on the original sources, demonstrating that the EEG benefit persists across source generators. We further tested **Same-category attribute-mismatched EEG** on test subsets supporting this comparison, holding the model, source image, participant, and target category fixed while substituting EEG from a target with different attributes. **Appearance editing.** $$\begin{array}{lrrrrrr} \hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Same-category attribute-mismatched EEG}&0.217&0.363&0.189&0.523&0.766&0.572\\\\ \text{Correct EEG}&\mathbf{0.190}&\mathbf{0.347}&\mathbf{0.169}&\mathbf{0.544}&\mathbf{0.848}&\mathbf{0.615}\\\\ \hline \end{array}$$ **Semantic editing.** $$\begin{array}{lrrrrrr} \hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Same-category attribute-mismatched EEG}&0.428&0.673&0.244&0.239&0.591&0.522\\\\ \text{Correct EEG}&\mathbf{0.396}&\mathbf{0.656}&\mathbf{0.235}&\mathbf{0.254}&\mathbf{0.623}&\mathbf{0.544}\\\\ \hline \end{array}$$ Correct EEG improves all six metrics in both tasks, including C-MAE reductions of **10.6%** for appearance and **3.7%** for semantic editing, demonstrating target-specific information beyond category cues. On the **200 Official THINGS test images**, Correct EEG also reduces C-MAE from **0.279 to 0.228** and raises R-Edit from **0.487 to 0.527** relative to Mismatched EEG with the editor, source images, and participant fixed. This set contains one stimulus per concept, so within-concept attribute mismatches are unavailable. --- ## Q1 Source-only training and the source-copy reference Thank you for asking about the source-only training setup. To clarify the training comparison, the source-only editor matches EEGraft's OmniGen2 initialization, data, optimization protocol, LoRA settings (rank 32, dropout 0.05), and training steps (**2,400 for appearance and 3,000 for semantic editing**). It retains the same flow, endpoint, and decoded-image target-recovery objectives. Prediction-difference supervision matches the EEG-induced prediction change to the source-to-target displacement; we set **$\lambda_\Delta=0$** because source-only has no EEG condition to switch on or off. R-Edit assigns **0.5** to copying the source without correcting it; exceeding this reference requires target-region recovery to outweigh added preservation error. Without target EEG, source-only relies on source cues and learned visual priors. Its recovery gains do not offset that error, yielding R-Edit of **0.494 for appearance and 0.461 for semantic editing** (**0.480 overall**).

Q2 Task-wise EEG gaps and same-concept mismatch

Thank you for requesting task-wise gaps, participant-level uncertainty, and a same-concept control. We report Correct–Mismatched differences on the complete 1,600-pair appearance and 1,200-pair semantic test sets, with the checkpoint, source images, and evaluation protocol fixed within each task.

We estimated 95% confidence intervals for Δ=Correct−Mismatched\Delta=\mathrm{Correct}-\mathrm{Mismatched} using 10,000 paired participant-cluster bootstrap resamples, stratified by dataset, retaining each sampled participant's observations and condition pairing. Each resample follows the original metric aggregation, computing R-Edit from summed errors before taking the ratio. Collateral denotes added target error in regions to be preserved relative to the source.

\hline \text{Metric}&\text{Appearance Δ}&\text{Appearance 95\\% CI}&\text{Semantic Δ}&\text{Semantic 95\\% CI}\\\\ \hline \text{T-LPIPS ↓}&\text{−0.032}&\text{[−0.037, −0.026]}&\text{−0.025}&\text{[−0.030, −0.021]}\\\\ \text{C-LPIPS ↓}&\text{−0.027}&\text{[−0.034, −0.021]}&\text{−0.041}&\text{[−0.045, −0.038]}\\\\ \text{C-MAE ↓}&\text{−0.051}&\text{[−0.060, −0.043]}&\text{−0.044}&\text{[−0.048, −0.040]}\\\\ \text{C-SSIM ↑}&\text{+0.040}&\text{[+0.029, +0.052]}&\text{+0.033}&\text{[+0.028, +0.039]}\\\\ \text{PixCorr ↑}&\text{+0.054}&\text{[+0.042, +0.067]}&\text{+0.068}&\text{[+0.063, +0.073]}\\\\ \text{R-Edit ↑}&\text{+0.059}&\text{[+0.051, +0.067]}&\text{+0.045}&\text{[+0.040, +0.051]}\\\\ \text{Collateral ↓}&\text{−0.005}&\text{[−0.009, −0.002]}&\text{−0.010}&\text{[−0.013, −0.006]}\\\\ \hline \end{array}$$ All intervals favor Correct EEG in both tasks, showing improved target recovery alongside lower collateral error. The requested same-category attribute-mismatch control is reported in W2, where holding category fixed confirms target information beyond category cues. ## Q3 EEG trial averaging for retrieval and editing Thank you for requesting clarification on the EEG input protocol. We understand the importance of specifying how many recordings condition each result. Both Table 16 retrieval and editing use EEG signals averaged over four trials from the same participant viewing the same stimulus. Averaging is performed at the EEG signal level, before encoding, to reduce trial-to-trial noise and stabilize stimulus-related responses. Each edit is conditioned on this four-trial average. ## Q4 Decode-then-edit with the same backbone Thank you for suggesting a decode-then-edit comparison on the same backbone. We implemented it with the same frozen EEG decoder, mapping its 1,024-dimensional ViT-H embedding through a learned bridge to a VAE reference latent for OmniGen2's native reference-conditioning pathway. Both methods use the same backbone initialization, data splits, LoRA settings, editing losses, training steps, and inference and evaluation protocols, receiving only the source image and EEG at inference. The intermediate visual representation may leave local correction details underspecified, whereas EEGraft lets source-conditioned visual states directly read both the aligned summary and unpooled EEG features. $$\begin{array}{lrrrrrr} \hline \text{Method}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Decode-then-edit}&0.432&0.623&0.281&0.257&0.533&0.545\\\\ \text{EEGraft}&0.278&0.479&0.197&0.420&0.752&0.585\\\\ \hline \end{array}$$ Decode-then-edit achieves positive net editing performance with R-Edit of 0.545. EEGraft improves all six metrics, reducing C-MAE by 29.900% and raising R-Edit to 0.585, demonstrating better target recovery and recovery–preservation balance under the same decoder and backbone. The matched concatenation control in W1 further tests readout organization using the same summary and unpooled features; separate readouts improve R-Edit by 0.056.
Reviewer_yK7HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EEG RAFT, an EEG-based image editing method that modifies the semantics and appearance of a source image based on EEG signals recorded while a user views a target image. Built upon OmniGen2, the method introduces dual-path EEG Cross-Attention and Time-Gated Residual Updates to incorporate EEG information into the image generation process, along with additional loss functions to supervise the editing process. Furthermore, the paper constructs NEVE, a benchmark comprising approximately 29K source–target image pairs.

Critical strengths
  1. The paper is well-written.

  2. The proposed NEVE benchmark represents a valuable dataset contribution.

Critical weaknesses
  1. The proposed dual-path EEG cross-attention represents an incremental improvement.

  2. The proposed method is not compared with the baselines on publicly available benchmarks.

  3. OmniGen2 already incorporates source conditioning, making the proposed conditioned EEG readout appear to be a natural application of standard cross-attention within an existing source-conditioned DiT.

  4. The paper claims that the model can dynamically adapt neural signal guidance throughout the generation process through time-gated residual updates, but the benefits of this mechanism have not been directly demonstrated.

Questions for authors

Please refer to Weaknesses.

5 comments

We appreciate the reviewer’s thoughtful assessment and constructive suggestions, and respond below with additional experiments and clarification of our methodological contributions and evaluation.

W1 — Contribution of the dual-pathway EEG readout

Thank you for raising the question of methodological novelty. Target-viewing EEG provides information about the desired image, while the source determines the correction required to reach it. The same target can require different changes to different sources, so EEGraft uses the evolving, source-conditioned visual state to retrieve target evidence. The available evidence also depends on its representation. An aligned summary supplies a learned target representation, but a summary-only interface merges feature differences through pooling, which expanding the summary into more tokens cannot recover. EEGraft retains the pre-pooling features alongside this summary. For encoder features P=(p1,…,pn)P=(p_1,\ldots,p_n), the two memories are

\qquad M_g=\Pi_g(g(P)), \qquad M_p=\Pi_p(P).$$ Providing both representations leaves the question of how each contributes to the visual update. Reading concatenated memories through one attention places their features under a shared normalization. EEGraft gives each memory separate readout parameters and attention normalization, allowing its own feature selection and residual scaling. Let $J_t$ denote the backbone's native joint attention between generation tokens $Z_x$ and source tokens $Z_s$. The update is $$\begin{aligned} H &= J_t(Z_x,Z_s),\\\\ H' &= H+a_g(t)r_g(H,M_g)+a_p(t)r_p(H,M_p). \end{aligned}$$ Here, $r_g$ and $r_p$ are independently parameterized multi-head cross-attention readouts, including their output projections. The current visual state $H$ supplies their queries, while $a_g(t)$ and $a_p(t)$ scale their residuals over generation time. Both memories therefore remain available for retrieval in relation to the source, with separate control of their contributions. Prediction-difference supervision trains the combined neural guidance to produce the required correction. With the source and noise held fixed, the conditional prediction $v_c$ and EEG-disabled prediction $v_o$ are supervised to satisfy $$v_c-\operatorname{sg}(v_o)\approx z_y-z_s.$$ This matches the EEG-induced prediction change to the source-to-target displacement; the corresponding loss ablation is reported in W3. We evaluate the representation and readout choices under this common training objective. The concatenated and dual-pathway variants are fully trained with the same EEG representations, OmniGen2 backbone, data, training objectives, and optimization protocol, with matched readout capacity. The table also includes the existing single-representation ablations. $$\begin{array}{lrrr} \hline \text{Configuration}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{R-Edit ↑}\\\\ \hline \text{Summary-only}&0.492&0.206&0.557\\\\ \text{Unpooled-only}&0.517&0.223&0.533\\\\ \text{Concatenated single-path}&0.519&0.227&0.529\\\\ \text{Dual-pathway EEGraft}&\mathbf{0.479}&\mathbf{0.197}&\mathbf{0.585}\\\\ \hline \end{array}$$ Summary-only already provides an effective editing baseline. EEGraft further reduces C-LPIPS by **2.6%** and C-MAE by **4.4%**, while increasing R-Edit from 0.557 to 0.585 (**+0.028**), improving target-region recovery and the joint recovery–preservation score. Summary-large expands the summary to match the dual-pathway conditioning-token budget, but does not recover this performance (Appendix G). The full model also outperforms Unpooled-only, supporting access to both representations. The concatenated single-path editor performs below Summary-only, showing that adding unpooled features to a shared memory does not itself improve editing in this comparison. With the same two representations, separate readouts reduce C-LPIPS by **7.7%** and C-MAE by **13.2%** relative to concatenation, and increase R-Edit by **0.056**. These comparisons establish the task value of organizing aligned and pre-pooling EEG evidence into separate readouts for source-conditioned correction, beyond both the effective summary-based editor and single-path fusion. ---

W2 — Evaluation on public benchmarks

Thank you for highlighting evaluation on public benchmarks. Existing EEG decoding datasets pair viewed images with EEG for retrieval or reconstruction. We did not identify a public benchmark for source-image correction using target-viewing EEG, motivating NEVE's dedicated protocol. To address your concern, we evaluated EEGraft on L-Mind [1], a public editing dataset recording EEG, fNIRS and PPG while participants view image–text pairs and read editing instructions aloud. We adapted it for EEG-conditioned editing.

\hline \text{Method}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{PVS-EEG}&0.5089&0.6614&0.3595&0.2622&0.4346&0.4793\\\\ \text{EEGraft}&\mathbf{0.4784}&\mathbf{0.6474}&\mathbf{0.3412}&\mathbf{0.2685}&\mathbf{0.5634}&\mathbf{0.5221}\\\\ \hline \end{array}$$ On L-Mind, EEGraft outperforms PVS-EEG on all six metrics. It reduces C-LPIPS by **2.1%** and C-MAE by **5.1%**, and raises R-Edit from 0.4793 to 0.5221 (**+0.0428**). Crossing the source-copy reference of 0.5 indicates a positive recovery–preservation balance. These gains on a public benchmark complement NEVE's target-viewing EEG correction task. **References** [1] Pengfei Zhou et al. "[Neural-Driven Image Editing](https://openreview.net/forum?id=UAc6RL9Tt4)." NeurIPS, 2025. [Dataset](https://huggingface.co/datasets/Lance1573/L-Mind). --- **W3 — Contribution beyond OmniGen2's native source conditioning** Thank you for highlighting what the EEG readout adds to OmniGen2's native source conditioning. Native joint attention $J_t$ provides a visual state conditioned on the source image, while EEG records the participant's response to the desired target. The same target can require different corrections for different sources, so EEGraft uses this visual state to retrieve target evidence from a visually aligned EEG summary $g$ and pre-pooling encoder features $P$. Their projected memories are $M_g=\Pi_g(g)$ and $M_p=\Pi_p(P)$, and the added EEG update in each block is $$\begin{aligned} H_t &= J_t(Z_x,Z_s),\\\\ H'_t-H_t &=a_g(t)\\,r_g(H_t,M_g) +a_p(t)\\,r_p(H_t,M_p). \end{aligned}$$ Each readout queries its EEG memory from $H_t$, with independent projections and attention normalization. Separate time gates scale their residual contributions. The memories remain fixed during sampling, while their contributions adapt to the evolving visual state and generation time. We tested OmniGen2's native interface with a decode-then-edit baseline using the **same frozen EEG decoder**. Its **1,024-dimensional ViT-H embedding** is mapped through a learned bridge to a **VAE reference latent**, which enters the native reference-conditioning pathway together with the source image. This baseline uses neither EEGraft's dedicated readouts nor their time gates. Both methods match backbone initialization, data splits, backbone LoRA settings, editing losses and training steps, and inference and evaluation protocols. At inference, both receive only the source image and EEG. $$\begin{array}{lrrrrrr} \hline \text{Method}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Decode-then-edit}&0.432&0.623&0.281&0.257&0.533&0.545\\\\ \text{EEGraft}&\mathbf{0.278}&\mathbf{0.479}&\mathbf{0.197}&\mathbf{0.420}&\mathbf{0.752}&\mathbf{0.585}\\\\ \hline \end{array}$$ The decode-then-edit baseline achieves positive net editing performance, with R-Edit of **0.545**. EEGraft improves all six metrics, reducing C-LPIPS by **23.1%** and C-MAE by **29.9%**, and increasing R-Edit to **0.585** (**+0.040**). These gains establish better target recovery and recovery–preservation balance under the same EEG decoder and backbone. Under the same backbone and training protocol (W1), EEGraft improves R-Edit by **0.028** over Summary-only and **0.056** over concatenating the same Summary and Unpooled representations into one readout. These results support both pre-pooling feature access and separate readouts within the source-conditioned editor. These comparisons fix the training objectives; we separately test supervision of the EEG-induced correction. At $t=0$, the current editor predicts $v_c$ with EEG and $v_o$ without EEG from the same source and initial noise. In addition to target supervision, we use $$L_\Delta =\mathbb E\\!\left[ \operatorname{mean}\rho_\beta\\!\left( v_c-\operatorname{sg}(v_o)-(z_y-z_s) \right) \right].$$ The reference is the current editor's detached EEG-off prediction, while target supervision trains the conditioned endpoint toward the target. Removing difference supervision raises C-LPIPS from **0.479 to 0.503** and C-MAE from **0.197 to 0.212**, and lowers R-Edit from **0.585 to 0.545**. This ablation establishes that correction supervision improves target recovery and the joint editing score. ---

W4 — Direct evidence for time-gated neural guidance

Thank you for asking for direct evidence of the time-gating mechanism. To test whether generation-time dependence improves on learned residual scaling, we compare a time-dependent model T with a learned constant-gate model S. Both retain the two EEG pathways, the same backbone and training objectives, and are fully trained under the same optimization protocol. For layer ℓ\ell and pathway bb, the injected update is

=a_{\ell,b}(t)\\,R_{\ell,b}(H_t,M_b),$$ where $R_{\ell,b}$ is the readout before gating. The comparison changes only whether the gate explicitly depends on generation time, with the two models defined by $$\begin{aligned} e(t)&=[t,\\ t^2,\\ \sin(\pi t),\\ \cos(\pi t),\\ \sin(2\pi t)]^\top,\\\\ a^T_{\ell,b}(t)&=\sigma\\!\left(w_{\ell,b}^{\top}e(t)+c_{\ell,b}\right),\\\\ a^S_{\ell,b}&=\sigma(c_{\ell,b}). \end{aligned}$$ S fixes the time coefficients to zero while learning a separate bias for every layer and pathway, so it retains both state-dependent attention and learnable EEG residual strength. The comparison therefore tests the value of varying that strength with generation time. Both models are evaluated with correctly paired EEG. $$\begin{array}{lrrrrrr} \hline \text{Gate configuration}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Learned constant gates (S)}&0.321&0.541&0.258&0.384&0.703&0.511\\\\ \text{Learned time-dependent gates (T)}&\mathbf{0.278}&\mathbf{0.479}&\mathbf{0.197}&\mathbf{0.420}&\mathbf{0.752}&\mathbf{0.585}\\\\ \hline \end{array}$$ Time-dependent gating outperforms learned constant gating on all six metrics, reducing C-LPIPS by **11.5%** and C-MAE by **23.6%**, and increasing R-Edit from 0.511 to 0.585 (**+0.074**). This establishes that learning when to apply stronger or weaker EEG residuals improves target recovery and the joint recovery–preservation score. To characterize the adjustments learned by T, we examine gate values and post-gating residuals. Gate values describe the learned scaling, while the residuals capture its combined effect with the evolving attention readout. For both analyses, we first average the appearance and semantic measurements at each corresponding layer, pathway, and sampling step. The pathway mean gates change from the first to the final step as follows. $$\begin{array}{lrrr} \hline \text{Pathway}&\text{First-step mean gate}&\text{Final-step mean gate}&\text{Relative change}\\\\ \hline \text{Summary}&0.121&0.106&\text{−12.3\\%}\\\\ \text{Unpooled}&0.119&0.106&\text{−10.8\\%}\\\\ \hline \end{array}$$ These mean decreases reflect different temporal adjustments across layers. Summary gates decrease in 22 layers and increase in the remaining 10; Unpooled gates decrease in 21 layers and increase in the remaining 11. The model thus learns distinct temporal schedules across layers. The actual neural update depends on both the gate and the state-dependent readout $R_{\ell,b}(H_t,M_b)$. To characterize their combined effect, we measure post-gating residuals using the same averaging across the two task editors. Their relative injection magnitude decreases from the first to the final sampling step by **40.1% for Summary** and **10.7% for Unpooled**. Larger relative updates occur in shallower Transformer blocks. These measurements show how learned scaling and evolving attention jointly shape the EEG updates over sampling steps and layers.
Reviewer_HCJpOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces EEGraft for editing a source image using EEG recorded while a participant views the target image. Source-conditioned visual states query pooled EEG summaries and unpooled encoder features through separate attention pathways, with time-gated residual updates guiding generation. Training combines flow matching, target supervision, and an auxiliary prediction-difference objective. The authors construct NEVE from existing EEG–image resources by generating modified source images while retaining measured target–EEG correspondences. Experiments cover appearance and semantic editing with concept-disjoint partitions, comparisons against six adapted baselines, and controls assessing the contributions of correctly paired EEG, the two readout pathways, and the training objectives.

Critical strengths
  1. Inverting the data generation direction, modifying viewed natural stimuli into synthetic sources while keeping the original viewed stimulus as the target, cleverly solves the lack of paired source, target-EEG, and target data without synthesizing fake neural signals. Importantly, the authors enforce strict concept-disjoint splits across all datasets, design region-aware evaluation metrics that penalize collateral changes outside the edit region, and report comprehensive diagnostic controls, including null EEG, within-subject and cross-subject mismatched EEG, a source-only editor trained without EEG, and a token-capacity control.

  2. The proposed dual-pathway EEG readout and prediction-difference supervision are also well motivated and carefully ablated. Allowing source-conditioned visual states to query both global summary tokens and unpooled electrode tokens, while explicitly supervising the difference between EEG-enabled and EEG-disabled one-step predictions, effectively encourages the pretrained editor to use neural cues.

Critical weaknesses
  1. A cross-comparison between the main baseline table and the control tables reveals that when EEGRAFT is evaluated with null EEG or mismatched EEG, or even when a baseline editor is trained entirely without EEG, it already substantially outperforms all six external baselines, even when those baselines are given the correct EEG. This points to two major confounds: first, the external baselines rely on older generative backbones that are not competitive with OmniGen2; second, because every source image in NEVE is AI-modified while every target image is an unedited natural photograph from standard datasets, a fine-tuned OmniGen2 model can exploit subtle AI-inpainting artifacts or canonical object priors to infer which foreground object was altered and revert it toward the natural image distribution without relying on EEG. By contrast, the actual gain from replacing mismatched EEG with correct EEG is relatively modest.

  2. The adapted external baselines do not cleanly isolate the proposed method’s advantage. They originate from different neural modalities and editing formulations, and all six obtain aggregate R-Edit scores below the source-copy reference of 0.5, compared with 0.585 for EEGraft. Their adaptation and optimization are therefore central to interpreting the large reported gaps. Appendix L acknowledges the differences, but clearer matching of backbone capabilities, objectives, and tuning effort is needed before attributing the gains specifically to the proposed neural readout. The within-model controls are stronger evidence than broad superiority over the original methods.

Questions for authors
  1. Can you report the results for the source-only model (Trained w/o EEG), Null EEG, Mismatched EEG, and Correct EEG separately for the appearance and semantic test sets? Specifically, how much does Correct EEG improve over Mismatched EEG on the semantic editing subset alone?

  2. Why does Null EEG substantially outperform Trained w/o EEG when neither configuration uses EEG at inference? Why does training the shared backbone LoRA alongside the EEG pathways (with stop-gradient on the unconditioned branch in the difference loss) make the EEG-disabled branch so much better at source-only target recovery than directly training the exact same OmniGen2 backbone without EEG?

  3. For appearance edits, does correct EEG outperform EEG from the same participant and object category but with different target attributes? A corresponding matched comparison for semantic edits would clarify which target-specific information is supplied by EEG beyond the source image and broad category cues.

6 comments

We thank the reviewer for the careful reading and constructive feedback, and address the concerns below with additional experiments and clarifications.

W1 — Source-image priors, backbone differences, and the contribution of EEG

Thank you for highlighting the backbone and source-prior concerns. Our LoongX adaptation uses the 12B FLUX.1-dev [1] for both tasks; its image generator and OmniGen2 both use rectified flow. With both methods fully trained and evaluated under a common NEVE protocol, EEGraft reduces C-MAE from 0.293 to 0.197 (32.8%) and raises R-Edit from 0.337 to 0.585.

Source images already provide scene structure and content to preserve, which OmniGen2 uses through its source-conditioning pathway even without EEG. A plausible explanation for the external baselines' lower scores is difficulty translating target-related EEG into the required local corrections while preserving unrelated content. To isolate our gain within the same backbone, we compare with a fully trained source-only OmniGen2 under matched conditions. EEGraft reduces C-LPIPS by 24.2% and C-MAE by 37.9%, and raises R-Edit from 0.480 to 0.585 (Table 5).

To measure the benefit of correct EEG pairing within this editor, we fix the checkpoint and source image, including any inpainting traces (Table 2). Correct EEG improves all six metrics over Mismatched and Null EEG. Relative to Mismatched EEG, it reduces C-LPIPS by 6.4% and C-MAE by 19.6%, and raises R-Edit from 0.532 to 0.585; C-MAE also falls by 18.6% relative to Null. These controls establish the benefit of recording–target correspondence with source cues and model priors unchanged.

We further examine the role of source construction through a change of generator. NEVE requires natural, category-appropriate colors and realistic, scene-compatible replacements, preserves unrelated content, and modifies object groups consistently to reduce unnaturalness cues and target leakage (Sec. 3.1 and Appendix A). Keeping this construction protocol and the target–EEG pairs fixed, we replaced the source generator with gpt-image-2.5 and evaluated the same editor without fine-tuning.

\hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Null EEG}&0.322&0.531&0.251&0.381&0.687&0.517\\\\ \text{Mismatched EEG}&0.313&0.527&0.255&0.384&0.691&0.526\\\\ \text{Correct EEG}&\mathbf{0.280}&\mathbf{0.476}&\mathbf{0.194}&\mathbf{0.422}&\mathbf{0.748}&\mathbf{0.583}\\\\ \hline \end{array}$$ Correct EEG again improves all six metrics over both controls, reducing C-MAE by **23.9%** relative to Mismatched EEG and **22.7%** relative to Null. Full-model performance remains comparable across the original/new generators, with C-LPIPS of 0.479/0.476, C-MAE of 0.197/0.194, and R-Edit of 0.585/0.583. The same-backbone and fixed-editor comparisons establish EEG's contribution beyond source-image and generative priors, with correct pairing improving results across both generators. **References** [1] Black Forest Labs. "[FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)." Model card, 2024.

W2 — Baseline adaptation, optimization, and the contribution of the neural readout

Thank you for highlighting baseline adaptation and optimization. All six NEVE adaptations preserve their core editing mechanisms, use target-viewing EEG, and follow a common evaluation protocol. Each is fully trained with an architecture-appropriate procedure for both tasks. Brain-Supervised and PVS-EEG retain attribute- and distance-mediated editing with source-conditioned mappings; PVS-EEG uses multiple training anchors. Both receive 2,500 updates at batch size 512 (1.28 million cached-sample draws), with internal selection and edit-magnitude calibration. MindArt retains its linear visual mapping with direction and magnitude supervision of editing residuals, while MindPainter adapts pretrained Paint-by-Example for brain-conditioned editing. MindCustomer retains representation learning (42,000 exposures), conditional generation (42,000), and source-image embedding optimization. LoongX retains FLUX.1-dev and reference-image conditioning, with 96,000 EEG-alignment and 42,000 editing exposures.

R-Edit exceeds the source-copy reference of 0.5 only when target-region recovery outweighs added preservation error within each editing subset. Target improvement alone can therefore still score below 0.5. Brain-Supervised, the strongest external R-Edit baseline (0.386), maps EEG-derived attributes to source-conditioned updates. Similar attribute representations may require different local corrections, leaving spatial or appearance details underspecified. Edit-magnitude calibration adjusts strength but cannot supply missing details. EEGraft achieves stronger target recovery with less added error in low-change regions than all six baselines (Figure 5), yielding R-Edit 0.585.

To isolate readout design, we compare single-path concatenation with dual-pathway readouts using the same OmniGen2 backbone, Summary and Unpooled EEG representations, objectives and optimization protocol, with matched readout capacity.

\hline \text{Readout}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Concatenated single-path}&0.307&0.519&0.227&0.406&0.712&0.529\\\\ \text{Dual-pathway EEGraft}&\mathbf{0.278}&\mathbf{0.479}&\mathbf{0.197}&\mathbf{0.420}&\mathbf{0.752}&\mathbf{0.585}\\\\ \hline \end{array}$$ The dual-pathway readout improves all six metrics, reducing C-LPIPS by **7.7%** and C-MAE by **13.2%**, and increasing R-Edit by **0.056**. Summary-only uses pooled EEG; Summary-large expands it to EEGraft's conditioning-token budget (Appendix G). EEGraft outperforms both, demonstrating the value of unpooled EEG features. Figure 6 and Q2 isolate prediction-difference supervision. Q3 fixes participant and target category, showing that correct pairing supplies target information beyond coarse category cues. ### Q1 Results for appearance and semantic editing Thank you for requesting task-wise results, particularly for semantic editing. All conditions use the same complete test sets and evaluation protocol. Trained w/o EEG uses no EEG during training or inference; the four EEG controls share one checkpoint. Null disables both readouts at inference; Mismatched uses another stimulus's EEG from the same participant; Cross-Mismatched changes both participant and concept. **Appearance editing, 1,600 pairs.** $$\begin{array}{lrrrrrr} \hline \text{Method}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Trained w/o EEG}&0.275&0.465&0.269&0.436&0.702&0.494\\\\ \text{Null EEG}&0.221&0.386&0.216&0.512&0.795&0.548\\\\ \text{Mismatched EEG}&0.222&0.374&0.220&0.504&0.794&0.556\\\\ \text{Cross-Mismatched EEG}&0.218&0.384&0.216&0.505&0.785&0.562\\\\ \text{Correct EEG}&\mathbf{0.190}&\mathbf{0.347}&\mathbf{0.169}&\mathbf{0.544}&\mathbf{0.848}&\mathbf{0.615}\\\\ \hline \end{array}$$ **Semantic editing, 1,200 pairs.** $$\begin{array}{lrrrrrr} \hline \text{Method}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Trained w/o EEG}&0.541&0.854&0.380&0.187&0.445&0.461\\\\ \text{Null EEG}&0.429&0.699&0.277&0.218&0.564&0.485\\\\ \text{Mismatched EEG}&0.421&0.697&0.279&0.221&0.555&0.499\\\\ \text{Cross-Mismatched EEG}&0.431&0.686&0.275&0.222&0.546&0.498\\\\ \text{Correct EEG}&\mathbf{0.396}&\mathbf{0.656}&\mathbf{0.235}&\mathbf{0.254}&\mathbf{0.623}&\mathbf{0.544}\\\\ \hline \end{array}$$ On semantic editing, Correct EEG reduces C-LPIPS by **5.9%** and C-MAE by **15.8%** versus Mismatched EEG, raising R-Edit from 0.499 to 0.544 (**+0.045**). Only Correct EEG exceeds the source-copy reference of 0.5 among these five conditions. Appearance editing reduces these errors by **7.2%** and **23.2%**, with R-Edit increasing by **0.059**. Correct EEG leads on all six metrics in both tasks.

Q2 Null EEG and training without EEG

Thank you for asking why Null EEG outperforms a model trained without EEG. Both use OmniGen2 and are fully trained under matched data and optimization protocols, but their LoRA parameters are learned with different supervision. Null retains the full editor's jointly trained LoRA when its EEG readouts are disabled. Trained w/o EEG uses the applicable target-recovery objectives with λΔ=0\lambda_\Delta=0, because prediction-difference supervision matches the EEG-induced change to the source-to-target displacement,

vc−sg⁡(vo)≈zy−zs.v_c-\operatorname{sg}(v_o)\approx z_y-z_s.

The source-only model has no EEG condition to switch on or off, so this objective is disabled. In full EEGraft, stop-gradient blocks backpropagation through vov_o in the current loss evaluation, while LΔL_\Delta updates the shared LoRA through vcv_c. Those updates also affect subsequent EEG-disabled predictions. To test whether this supervision improves Null recovery, we evaluate No-Δ, which removes LΔL_\Delta with all other training settings unchanged, on both complete test sets.

\hline \text{Model}&\text{EEG at inference}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Source-only}&\text{No EEG}&0.389&0.632&0.317&0.329&0.592&0.480\\\\ \text{No-Δ}&\text{Null EEG}&0.377&0.641&0.305&0.338&0.679&0.518\\\\ \text{Full}&\text{Null EEG}&0.310&0.520&0.242&0.386&0.696&0.521\\\\ \text{No-Δ}&\text{Correct EEG}&0.297&0.503&0.212&0.406&0.728&0.545\\\\ \text{Full}&\text{Correct EEG}&0.278&0.479&0.197&0.420&0.752&0.585\\\\ \hline \end{array}$$ With EEG disabled in both models, adding $L_\Delta$ reduces C-LPIPS by **18.9%** and C-MAE by **20.7%**, increases C-SSIM from 0.338 to 0.386, and raises R-Edit by **0.003**. Removing this supervision leaves Null C-MAE at 0.305, close to source-only's 0.317. This ablation demonstrates that prediction-difference supervision improves target-region recovery through training effects retained when the EEG readouts are disabled. ### Q3 EEG information beyond object category Thank you for suggesting the matched comparison. On test subsets supporting within-category attribute mismatches, we fixed the model, source image, participant and target category, and replaced correct EEG with recordings corresponding to different target attributes. **Appearance editing.** $$\begin{array}{lrrrrrr} \hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Same-category attribute-mismatched EEG}&0.217&0.363&0.189&0.523&0.766&0.572\\\\ \text{Correct EEG}&\mathbf{0.190}&\mathbf{0.347}&\mathbf{0.169}&\mathbf{0.544}&\mathbf{0.848}&\mathbf{0.615}\\\\ \hline \end{array}$$ **Semantic editing.** $$\begin{array}{lrrrrrr} \hline \text{EEG condition}&\text{T-LPIPS ↓}&\text{C-LPIPS ↓}&\text{C-MAE ↓}&\text{C-SSIM ↑}&\text{PixCorr ↑}&\text{R-Edit ↑}\\\\ \hline \text{Same-category attribute-mismatched EEG}&0.428&0.673&0.244&0.239&0.591&0.522\\\\ \text{Correct EEG}&\mathbf{0.396}&\mathbf{0.656}&\mathbf{0.235}&\mathbf{0.254}&\mathbf{0.623}&\mathbf{0.544}\\\\ \hline \end{array}$$ Correct EEG improves all six metrics in both tasks. For appearance and semantic editing, C-MAE falls by **10.6%** and **3.7%**, and R-Edit rises by **0.043** and **0.022**, respectively. These gains with participant and category held fixed demonstrate target-related EEG information beyond broad category cues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.