- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this work, the authors ask whether a frozen diffusion model's conditionable interface is a valid tool for estimating how much information a given signal carries about its stimulus, using the difference in conditional likelihood as a proxy. To this end, they construct a phantom that mimics a stimulus representation without requiring a brain: they define a chain of progressively coarser image averages, exploiting the ordering imposed by the data-processing inequality on the degraded stimuli, but not necessarily on their encoded representations or probe scores. In each trial, they perform a pairwise discrimination task in which the foils differ by one stimulus attribute – category, color, flip, blur, or layout – and assess the performance in a permutation-normalized manner.
When comparing four different interfaces on SDXL, they find that the 1-token interface underperforms relative to the null in the low-seam layout foils, while the 257-token readout of the same encoder pass, as well as the caption-based arms, do not. This directionality also replicates for SD1.5 and two independent stimulus samples from THINGS, with magnitudes attenuated by about 3x on the additional stimulus sets, showing that the effect depends on adapter scale. In the second part of this paper, they audit EEG-to-image decoding performance on the THINGS-EEG dataset.
Critical strengths
-
A cheap, brain-free audit. The nested-averaging phantom needs no brain data and can be reused. The paper documents two pitfalls: the noise-pairing bug (§2.1), and foil-dependent image-side bias (Table 3). Together they show that a raw two-way Δ ELBO comparison across images can’t be interpreted without a null. This is useful to anyone who uses diffusion likelihoods as a readout.
-
A controlled comparison with replication. The 1-token and 257-token readouts share one ViT-H forward pass, and the gap between them holds across two backbones and three stimulus sets.
Critical weaknesses
Critical Weaknesses
-
The proposed explanation of the probe failure needs stronger support. The negative layout efficiency and the gap between pooled and patch-grid conditioning are interesting findings. However, it remains unclear how much of the negative score reflects an interaction between conditioning and the different denoising difficulties of true images and layout foils. Table 3 shows substantial differences in baseline difficulty, and the supplementary implementation uses different empty conditions across interfaces. A control that holds the candidate image fixed while comparing conditions derived from the true image and its layout foil would help clarify the mechanism. The adapter-scale experiment supports a dependence on conditioning strength, but the claim that the inversion “deepens monotonically” is stronger than Table 4 supports: roll2d becomes less negative between scales 0.8 and 1.0. These issues matter because the paper makes causal claims about why the observed failure occurs.
-
The broader claims about brain-decoding systems and calibration exceed the demonstrated scope. The related-work discussion does not adequately support the claim that the tested pooled-embedding interface represents most EEG and fMRI reconstruction pipelines. The description of the cited fMRI systems is inaccurate: MindEye and Brain-Diffuser use patch-level image representations. The discussion also omits published EEG methods with alternative conditioning pathways, including DreamDiffusion (ECCV 2024), BrainVis (ICASSP 2025), and GWIT (ICASSP 2025). These works matter because the paper attributes the failure to the conditioning interface; the authors should explain how these methods fit their taxonomy and which conclusions extend beyond the tested architectures. The evaluated NICE and DCTD reconstruction pipelines also need clearer identification and references to establish what the “deployed decoders” claim covers. Furthermore, the proposed connection between adapter scaling in §4 and vector shrinkage in §5 remains insufficiently demonstrated: these are different interventions, and the authors acknowledge that shrinkage does not fully calibrate the renderer. Finally, “overconfident by a conformally guaranteed margin” overstates the guarantee. Split conformal prediction guarantees marginal coverage under its assumptions; the observed set sizes do not establish a guaranteed minimum set size or overconfidence margin. Together, these issues limit the generality and strength of the paper’s conclusions.
Questions for authors
-
Diagnostic control: Could you hold the candidate image fixed and compare its denoising error under conditions derived from the true image and its layout foil? Does the pooled interface still show a systematic preference for the incorrect layout condition?
-
Clarification: The supplementary code uses different empty conditions across interfaces. How do these choices affect Table 3 and the comparison of efficiencies? Is the main interface gap stable across repeated mismatched-condition assignments?
-
Scope: Please identify and reference NICE and DCTD, explain whether the evaluated decoding heads reproduce published reconstruction pipelines, and correct the description of the cited fMRI conditioning interfaces. Which conclusions do you expect to extend to other conditioning architectures?
-
Interpretation: What evidence establishes a shared mechanism between adapter scaling and vector shrinkage? Please also specify what quantity the conformal guarantee covers and distinguish that guarantee from the empirical set-size results.