acceptodds
Withdrawn from ICLR 2027

AFoB: Adaptive Foveated Blur with Dual-Region Preservation for EEG-Image Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Brain-signal-based image retrieval commonly aligns EEG or MEG recordings with pretrained visual representations, making the construction of the image-side target an important design choice. Variability in brain responses and informative content beyond the image center motivate two complementary questions: how much visual detail should be retained, and where should it be preserved? We propose Adaptive Foveated Blur (AFoB), a perception-inspired framework that combines response-conditioned blur with dual-region detail preservation. A joint predictor takes brain-signal and fixed-blur reference-image embeddings and estimates the background-blur strength and concentration parameters controlling the extents of two preserved regions. One region remains centered on the image, while the second is located at the centroid of a low-level saliency map computed from the clean image without a learned localization module. A differentiable soft mask combines the two regions and blends the original image with its blurred counterpart. A frozen CLIP encoder maps the resulting image to a target for contrastive brain–image alignment, through which the brain encoder and foveation predictor are jointly trained. On 200-way zero-shot retrieval, AFoB improves Top-1 / Top-5 accuracy over UBP from 50.2% / 80.0% to 53.8% / 84.3% in the intra-subject setting and from 12.2% / 32.0% to 17.1% / 40.1% in the inter-subject setting on THINGS-EEG, with a proportionally larger gain in the latter, and from 25.6% / 55.6% to 39.6% / 69.8% intra-subject on THINGS-MEG. Analysis of the predicted parameters on THINGS-EEG reveals variation across images within individual subjects, rather than a single transformation per subject. AFoB provides a computational framework for response-conditioned visual abstraction and motivates further research connecting selective information preservation with models of human visual cognition.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG-based visual decoding, contrastive learning, adaptive foveation, saliency-guided vision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_g7DMOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the impact of the amount of visual information retained in image-side targets and its spatial distribution on model performance during the EEG/MEG–image alignment process. The authors propose AFoB, which uses brain signals and image features to jointly predict the blurring intensity and the extent of two detail-retention regions, and determines the location of an additional retention region based on the center of image salience. Under an evaluation protocol that uses real paired brain signals to construct candidate features, AFoB reported Top-1 and Top-5 scores higher than the UBP baseline on THINGS-EEG and THINGS-MEG. Internal ablation analysis indicates that the contribution of per-sample adaptation is more stable than that of the additional retention region.

Critical strengths
  1. This paper combines the retention strength of visual information with spatial location as an extension of existing UBP methods, which is reasonably justified.
  2. AFoB uses a lightweight MLP, adaptive Gaussian blur, and saliency-based localization that requires no additional training; its module design is clear, and it does not require explicit supervision of the blur parameters.
Critical weaknesses

There are some details regarding the methods and the fairness of the experimental protocols that require further discussion. For more details, please see the “Questions” section.

Questions for authors
  1. According to the definition in Equation 9 of the manuscript, the representation of each candidate image xjx_j requires the brain signal eje_j that truly corresponds to that image. However, in standard EEG-to-image retrieval, we may only have Query: eie_i and Gallery: (x1,…,x200)(x_1,\ldots,x_{200}). We do not know the correspondence between xjx_j and eje_j. Therefore, I would like the authors to further explain why they can use ground-truth matched brain recordings to construct candidate image representations and still interpret the results as brain-to-image retrieval. More importantly, the embeddings of the candidate images are no longer determined purely by xjx_j, but rather by vj=vj(xj,ej)v_j=v_j(x_j,e_j). Consequently, the model’s performance may stem in part from brain-dependent target adaptation and cannot be entirely attributed to its ability to decode brain signals into visual semantics.
  2. In the Abstract and Tables 1–2, the authors interpret the results as performance improvements relative to UBP. However, Section 4.1 clearly states that UBP uses fixed image reference embeddings during testing; AFoB constructs candidate embeddings using ground-truth paired brain recordings; the numerical differences between the two methods do not prove retrieval improvement under identical evaluation conditions. Therefore, I ask the authors to explain: Given that the evaluation inputs for the two methods are inconsistent, why do the Abstract and Tables 1–2 still present the numerical gap between them as a key result?
  3. According to the definition in Sec. 3.2, Eq. (2), the prediction of AFoB parameters depends on both the EEG/MEG representation and the image representation. In Section 4.4, Table 3, the ablation experiment conducted after removing the brain input altered two things simultaneously: during the training phase, EEG/MEG was no longer used to predict foveation parameters, and during the testing phase, the candidate embeddings no longer relied on the true, paired held-out EEG/MEG data. Therefore, I believe the current experiment cannot distinguish between training gains and gains from privileged pairing during testing. I hope the authors will include a rigorous control experiment.

Please note: Since I am not an expert in this field, my understanding of the methodological background in this area is limited. Therefore, I suggest that AC adjust my rating as appropriate based on the author’s response to the comments above! As far as I'm concerned, if the author can clearly address the concerns mentioned above, I'd be happy to raise the score to 3. However, please let AC decide on the final rating.

Reviewer_rEEsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Adaptive Foveated Blur (AFoB), a framework for adapting image-side representations in EEG/MEG-based image retrieval. Motivated by variability in neural recordings and the importance of visual information beyond the image center, the authors introduce a dual-region preservation mechanism that jointly controls background blur strength and the extent of two preserved regions. Specifically, a predictor uses brain embeddings and fixed-reference image embeddings to estimate three adaptive parameters. The first preserved region is centered on the image, while the second is positioned using a non-learned saliency centroid derived from edge and color information. A differentiable mask combines the original and blurred images, and the resulting representations are aligned with brain signals through contrastive learning using a frozen CLIP image encoder. Experiments on THINGS-EEG and THINGS-MEG report 200-way paired-target ranking results. Several ablation studies investigate adaptive blur, dual-region preservation, saliency-based placement, and brain-signal conditioning.

Critical strengths

1.The paper addresses a meaningful problem in brain-image alignment by investigating how visual details should be selectively preserved when neural recordings contain incomplete or noisy information. Extending conventional center-focused foveation to response-conditioned dual-region preservation is a reasonable design choice. The framework integrates adaptive blur prediction, saliency-based region placement, differentiable image composition, and contrastive learning into a coherent pipeline, providing a useful perspective on adaptive image-side representations for neural decoding. 2.The authors evaluate the framework on both THINGS-EEG and THINGS-MEG and conduct targeted ablation studies examining adaptive blur, preservation-region count, alternative placement strategies, and brain-signal conditioning. These experiments provide useful insights into the behavior of individual components. The explicit acknowledgment of the paired-target evaluation limitations also helps clarify the scope of the experimental findings.

Critical weaknesses

1.The primary concern is the validity of the reported retrieval improvements. As described in Section 4.1 and Equation (9), AFoB constructs candidate image representations using their ground-truth matched held-out brain recordings, introducing privileged test-time information unavailable in conventional retrieval. In contrast, UBP uses fixed image features at test time. Although the authors acknowledge this difference, the improvements reported in Tables 1 and 2 are obtained under different evaluation conditions and cannot establish a controlled performance advantage over UBP. Furthermore, the image-only ablation does not fully isolate the contribution of brain-signal conditioning, since removing brain input also changes the information available during test-time candidate construction. Without a standard retrieval evaluation and controlled baseline comparisons, the claimed comparative retrieval advantage over UBP remains insufficiently established. 2.The necessity of the proposed dual-region preservation mechanism is also not convincingly established. Table 3 shows a 2.0-percentage-point Top-1 improvement over single-fovea preservation on THINGS-EEG, while MEG Top-5 accuracy is slightly lower (69.8% vs. 69.9%). Moreover, learned placement outperforms the proposed saliency-based placement on MEG Top-5 (71.1% vs. 69.8%). Since the saliency-based second region is not guaranteed to lie outside the central region, further analysis is needed to demonstrate that it consistently preserves meaningful noncentral information. These inconsistent improvements, together with the absence of reported multiple-seed evaluations and statistical significance tests, weaken the empirical justification for the proposed design.

Questions for authors

1.Could the authors evaluate AFoB under a standard image-gallery retrieval protocol without using ground-truth matched held-out brain recordings to construct candidate features? How much of the reported improvement remains under this evaluation setting? 2.Could the authors provide a controlled comparison with UBP under identical test-time information and preprocessing settings to isolate the gains attributable to the proposed method? 3.Given the inconsistent benefits of dual-region preservation in Table 3, could the authors provide multiple-seed evaluations and further analyze whether the second region consistently preserves meaningful noncentral visual information?

Reviewer_99voOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work uses a learned blurring function to create blurred images for the task of EEG to image retrieval. It follows past work that shows foveated blurring the images mimics human perception and improves retrieval accuracy. It is physiologically aligned and useful considering improvements in retrieval accuracy.

Critical strengths

The adaptive blur is physiologically aligned and given eeg datasets like things eeg use rapid visual presentation it is likely that the majority of visual processing happens around 1-2 focal points.

Critical weaknesses

I am not convinced that the retrieval gains can be explained from using the learned foveation function instead of information leakage. The AFoB transform needs the ground truth EEG for a candidate image. Which means when trying to retrieve image j given EEG i, one still needs EEG j first to convert image j into AFoB compatible blurred state. Needing EEG j at test time creates an information leak.

VariantEEG leakEEG Top-1 / Top-5EEG Δ vs UBPMEG Top-1 / Top-5MEG Δ vs UBP
UBP (reproduced)No50.2 / 80.0–25.6 / 55.6–
AFoBYes53.8 / 84.3+3.6 / +4.339.6 / 69.8+14.0 / +14.2
Learned 2nd foveaYes52.4 / 83.3+2.2 / +3.339.4 / 71.1+13.8 / +15.5
Random 2nd foveaYes52.8 / 82.7+2.6 / +2.738.9 / 69.2+13.3 / +13.6
Single foveaYes51.8 / 81.8+1.6 / +1.837.9 / 69.9+12.3 / +14.3
Fixed blur (subject means)No49.9 / 79.6−0.3 / −0.428.4 / 56.5+2.8 / +0.9
Image-only predictorNo49.0 / 79.8−1.2 / −0.226.0 / 54.8+0.4 / −0.8

Additionally the authors claim in the abstract that AFoB “improves Top-1/Top-5 accuracy over UBP”. This is contradicted by the experiments where there is no direct comparison possible. In the main text it is mentioned that “The released UBP evaluator uses fixed reference image features at test time, whereas the AFoB rows use Equation equation 9. Their numerical differences therefore do not establish a controlled retrieval improvement under identical evaluation inputs”. This prevents any direct comparison and the ablations point towards any gains coming from information leakage.

Questions for authors

I would like some experiments showing zero-shot evaluations. Eg one where you train with AFoB and test with UBP type fixed blur and a cross EEG AFoB where you use T(e_i, x_j) for the transform. Otherwise it is not possible to quantify the gains. Right now the ablations point towards there being no gains in the zero shot settings and gains only appearing in privileged information situations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.