acceptodds
Withdrawn from ICLR 2027

DS-LCCA: Differentially Supervised Lightness–Chromaticity Complementary Alignment for Neural Visual Decoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Decoding visual representations from brain signals helps reveal how the brain represents visual information and enables new forms of brain–computer interaction. However, existing approaches to visual supervision adaptation do not explicitly account for the distinct spatial sensitivities of human vision to Lightness and Chromaticity variations, potentially limiting neural–visual alignment. To address this limitation, we propose Differentially Supervised Lightness-Chromaticity Complementary Alignment (DS-LCCA), which constructs component-specific visual supervision in CIELUV space. The Lightness branch reweights multiscale discrete wavelet coefficients to attenuate spatial detail, whereas the Chromaticity branch applies Gaussian smoothing to suppress fine-scale Chromaticity variations. Each transformation preserves the remaining coordinates before conversion back to RGB. Four independent branches learn neural–visual alignment through symmetric contrastive objectives. Three Residual Projection Encoders align EEG or MEG recordings with CLIP ResNet-50 features of the original and transformed images for retrieval and classification. A dedicated ATM–ViT-H-14 branch learns original-image representations for reconstruction. Its neural embeddings condition a diffusion prior to produce the primary visual condition, while linear projections of the Lightness and Chromaticity embeddings provide auxiliary conditions. Separate IP-Adapters jointly condition a pretrained SDXL generator, and the auxiliary projections are optimized through the generation loss. Extensive experiments on THINGS-EEG and THINGS-MEG demonstrate that DS-LCCA outperforms state-of-the-art methods across retrieval, classification, and reconstruction. The code is available at https://anonymous.4open.science/r/DS-LCCA-4CB5.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Electroencephalogram, Magnetoencephalography, Visual Decoding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9aYuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

DS-LCCA adapts visual supervision for EEG/MEG decoding by processing lightness and chromatic coordinates separately in CIELUV space. Three independently parameterized Residual Projection Encoders align neural recordings with frozen CLIP RN50 features of the original and two transformed images. Retrieval and classification use the union of the three branch-specific top-k sets. A separate ATM–ViT-H-14 alignment branch supplies the primary reconstruction condition through a diffusion prior, while generation-trained affine projections of the lightness/chromatic embeddings supply auxiliary conditions to SDXL through separate IP-Adapters. Experiments cover THINGS-EEG and THINGS-MEG, within- and held-out-subject discrimination, reconstruction, component/operator ablations, encoder assignments, and temporal/spatial/spectral analyses. The principal contribution claimed is the value of component-specific visual supervision rather than a new contrastive objective or foundation model.

Critical strengths
  1. Thie paper propose a coherent, task-motivated supervision design with an explicit inference pathway. Separately modifying lightness structure and chromatic variation is an intelligible alternative to treating the original image representation as the only alignment target. The manuscript defines the transformations, independent alignment losses, diffusion prior, and auxiliary generation projections explicitly. In particular, the reconstruction formulation specifies a gradient pathway from the generation objective to the auxiliary projections, rather than assuming that an arbitrary projection automatically supplies useful conditioning. This makes the proposal a testable methodological hypothesis.

  2. Empirical coverage extends beyond a single aggregate score. The study includes EEG and MEG, subject-wise retrieval tables, multiple classification candidate-set sizes, reconstruction metrics, and qualitative examples. The chromatic-operator comparisons and the four primary–auxiliary encoder assignments are useful existing controls, and the appendices provide substantial parameter and input-window analyses. The paper also explicitly discloses that its discrimination output is a union containing up to 3k candidates and that Figure 6 does not define an ordering of that union. This transparency should be credited, even though it does not resolve comparability with the baseline rows.

Critical weaknesses
  1. Unfair Top-k Evaluation Protocol. The proposed union-based retrieval metric returns up to 3k candidates, making its reported Top-1/Top-5 results not directly comparable to conventional Top-k baselines. A fair comparison using identical candidate budgets and a single ranked prediction is necessary.

  2. Inconsistencies in Reported Results. Several numerical discrepancies appear across tables, particularly in the MEG results (Tables 1 and 6), subject-level averages (Table 5), and reconstruction metrics. These inconsistencies undermine the reliability of the experimental conclusions and require clarification.

  3. Insufficient Validation of the Core Contributions. The experiments do not adequately isolate the benefits of lightness/chromaticity decomposition from generic multi-branch ensembling or increased model capacity. Critical controls, including original-image-only baselines and capacity-matched ensembles, are missing.

  4. Limited Justification of Component-Specific Processing. The lightness and chromaticity transformations do not guarantee perceptual disentanglement, as the resulting images retain information from other components. The effectiveness of the proposed transformations and wavelet decomposition requires stronger quantitative and theoretical justification.

Questions for authors
  1. Which rows in Tables 1 and 5–7 are ordinary ranked Top-k accuracy, and which use Eq. (11)? Were the baseline rows evaluated with the identical union rule? For the same checkpoints, please report R/L/C separately, mean union size and overlap, and a label-free single-ranking fusion at exactly Top-1 and Top-5. Explain the source of the Figure 6 rankings. Union coverage may remain as a separately named, cardinality-matched measure.

  2. How should the MEG subject-wise mean of 41.125%/71.50%, the Table 6 average of 34.3%/63.9%, and the Table 1 result of 31.28%/60.46% be reconciled? Please identify the exact subjects, checkpoints, scoring rules, and averaging steps. Also clarify the Table 5 baseline averages, the Table 9 classification aggregation, and the 0.5294 versus 0.5249 SwAV values. Paired-ID prediction/score outputs would resolve these issues directly.

  3. Are quantitative R-only and matched three-original-view discrimination controls available? Is R retained in the Table 1 single-component rows? For generation, please provide or locate a matched G-only result and an original-image-only multi-condition control, with the same trainable capacity and sampling settings.

  4. Please specify the RGB conversion and gamut policy, image resolution, wavelet boundary handling, and treatment of signed/near-zero lightness in detail-only images. What measured lightness and chromaticity changes remain after conversion back to RGB? How do those measurements support the intended interpretation given the L-dependent relationship between u*, v* and u', v' in Eq. (3)?

  5. Which validation split and selection policy determined checkpoints and all reported defaults? Please identify the concept/subject split manifests, independent runs, and paired uncertainty outputs. For reconstruction, provide the stage-specific freeze/train schedule, condition/prompt specification, sampler and seed manifest, number and selection of generated outputs, and exact metric/aggregation functions. Pointing to the corresponding files in the linked repository is sufficient where these details already exist. These requests concern the reported tasks and datasets, not a new application domain.

Reviewer_XNWAOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DS-LCCA for decoding images from EEG and MEG. Each stimulus is converted to CIELUV. Two supervision images are built. One keeps only the wavelet approximation of L*. The other applies Gaussian smoothing to u* and v*. Three residual MLP encoders align neural signals with CLIP RN50 features of the original, lightness and chromaticity images. A fourth ATM branch with ViT-H-14 is used for generation. Its diffusion prior gives the main condition. Linear projections of the lightness and chromaticity embeddings give two extra conditions. All three go into SDXL through IP-Adapters. Retrieval and classification take the union of the top-k sets of the three branches. The paper reports state-of-the-art results on THINGS-EEG and THINGS-MEG for retrieval, classification and reconstruction. It adds ablations on subbands, smoothing operators, encoders, time windows, channels and frequency bands.

Critical strengths

The idea is simple and cheap. Separate treatment of luminance and chroma is a reasonable prior. It is easy to add to existing pipelines. The ablation coverage is wide. It spans subbands, chroma operators, encoder pairs, time, channels and bands. This makes the pipeline easy to inspect, even though the central claim is not isolated.

Critical weaknesses

The headline numbers are not comparable with the baselines. Retrieval and classification use \mathrm{Hit}_{\cup}@k. A hit counts if any of three branches succeeds. So top-1 allows up to 3 guesses and top-5 up to 15 guesses among 200 candidates. Baselines get one list. The union is also never ablated. Table 1 shows single branches at 49.6 and 52.5 top-1, against 66.35 for the union. The gain may come from the ensemble and not from the supervision design. There is no control with three branches on the original image, or on random blurs. Table 9 reports 66.35 retrieval for RPE-RN50 but 12.80 classification, which is the chromaticity-only value in Table 1. The tables do not describe one consistent protocol. The central claim is not tested. The claim is that differential treatment of lightness and chroma matters. The implementation sets all wavelet detail to zero, so lightness is a low-pass. Chroma is also a low-pass. Nothing compares the same operator on both channels. Nothing swaps the operators. Nothing compares against a blur on RGB, such as UBP. Nothing tests auxiliary conditions from the original image with the same extra adapters. The reconstruction gain in Table 4 may come from extra conditioning capacity. Hyperparameters are also tuned on the test set, with no validation split. Kernel size, sigma, depth and 18 wavelet bases are swept, and the chosen defaults win on nearly every metric. No seeds, variance or significance tests are given. The numbers have internal problems. Tables 10 and 12 report AlexNet(2) values such as 0.03 and 0.28. This is a two-way identification metric with chance at 0.5. Values below chance mean the metric is computed wrongly or defined differently. Table 6 lists MEG intra-subject top-1 for DS-LCCA as 22.5, 66.0, 50.5 and 25.5. The mean is 41.1, but the average column says 34.3. Table 1 says 31.28. SwAV for the same full model is 0.5294 in Tables 1, 3 and 8 and 0.5249 in Tables 2, 14 and 15. Table 14 shows occipital-only reconstruction at AlexNet(2) 0.49 against 0.85 for all channels. Table 13 shows occipital-only retrieval close to the best. This gap is unexplained. Baselines look weak. Table 7 gives ATM 7.40 top-1 for 200-way EEG classification. The authors’ own ATM variant gets 10.85 in Table 9. Baseline numbers appear to come from other protocols. Section 4.1 claims the best result on every task. Table 8 shows SeeEEG higher on CLIP. Fig. 3 lists BRENA, HA and NEED, but no detailed table reports them. MEG reconstruction is compared only with ATM.

Questions for authors

What are the retrieval and classification results using the single original-image branch alone, scored with one list? Please also give the union of three original-image branches trained with different seeds. How are AlexNet(2) values below 0.5 possible? Please give the exact metric definition and code path. Please explain the MEG intra-subject averages. Table 6 subjects average 41.1 and 71.5, but the table and Table 1 report other values. Were kernel size, sigma, depth and wavelet basis chosen on a held-out split? If not, please rerun with a validation split. What happens if the same low-pass is applied to both L* and chroma? What if the operators are swapped? In Table 4, what is the result when the two auxiliary adapters take conditions from the original image? Please report results over several seeds with confidence intervals. Please add BRENA, HA, NEED and SeeEEG to the retrieval tables under the same protocol.

Reviewer_jcHBOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the mismatch between brain and visual representations in EEG- and MEG-based visual decoding by proposing a framework called DS-LCCA, inspired by the different sensitivities of human vision to luminance and chrominance spatial frequencies, as established in visual psychophysics. In the CIELUV color space, the proposed method constructs two degraded yet perceptually aligned RGB visual supervision signals through wavelet-based low-frequency preservation (luminance branch) and Gaussian smoothing (chrominance branch), respectively. The model trains four independent alignment branches using symmetric InfoNCE contrastive learning. Three RPE-RN50 branches are aligned with the original, luminance, and chrominance images, respectively, for retrieval and zero-shot classification, with hit rates computed by taking the union of the Top-k candidate sets from the three branches. The fourth ATM-ViT-H-14 branch, combined with a diffusion prior, provides the primary conditioning signal for image generation. This signal, together with auxiliary conditions obtained through linear projections of the luminance and chrominance branches, guides SDXL through three IP-Adapters to generate images. The paper reports experimental results on retrieval, classification, and reconstruction using the THINGS-EEG and THINGS-MEG datasets, along with multiple ablation studies.

Critical strengths
  1. Well-motivated and intuitive interdisciplinary approach: Drawing on findings from visual psychophysics (Campbell & Robson, 1968; Mullen, 1985), particularly the low-pass characteristics of human chrominance perception and the preservation of coarse spatial structures in luminance perception, the paper translates these perceptual priors into dimensionality reduction and smoothing operations on image-side teacher supervision signals for brain decoding. This provides a reasonably novel perspective on the problem.
  2. Coverage of multiple modalities and downstream tasks: Under a unified conceptual framework, the paper evaluates two brain signal modalities with high temporal resolution, EEG (THINGS-EEG) and MEG (THINGS-MEG), across three core decoding tasks: within-subject and cross-subject retrieval, zero-shot classification, and image reconstruction.
  3. Comprehensive ablation and sensitivity analyses: The paper provides detailed experimental tables and visualizations covering a wide range of factors, including wavelet subbands (LL vs. LH/HL/HH), chrominance smoothing operators (K-means, mean shift, Gaussian smoothing, etc.), combinations of brain encoders and visual backbones, temporal windows, electrode subsets from different brain regions, frequency bands, and hyperparameters associated with wavelet basis functions and Gaussian kernels.
Critical weaknesses
  1. Unfair comparison. Equation (11) defines the joint Top-KK hit rate as the union of three branches, allowing up to 3K3K candidates instead of the KK candidates used by baselines and single-branch ablations. This makes the comparisons unfair and undermines the claimed improvements.
  2. Insufficient justification of the wavelet strategy. Equation (4) presents a multiscale wavelet weighting scheme, but Table 2 shows that only level-2 low-frequency components are retained, with high-frequency components set to zero. This is essentially simple low-pass filtering, and the effects of retaining high-frequency information are not adequately investigated.
  3. Reference formatting errors. Some references are incorrectly formatted, such as "ErnstDmenuAClandRHS.Commissioninternationaledel’eclairagec.1976."
  4. Suspicious numerical patterns in Table 12. For intra-subject retrieval across the 400–1000 to 900–1000 time windows, Top-1 accuracies (8.10, 6.74, 5.38, 4.02, 2.66, 1.30) decrease by exactly 1.36 per step, while Top-5 accuracies decrease by exactly 3.72.
Questions for authors
  1. Could the authors report the individual Top-1 and Top-5 retrieval performance of the original, lightness, and chromaticity branches? Given that the current union-based evaluation allows up to 3K3K candidates, a fixed-KK comparison with baselines and individual branches is necessary to establish the actual effectiveness of the proposed method.
  2. Although Table 18 shows that two wavelet decomposition levels perform best, what is the theoretical or perceptual justification for this choice? Since all high-frequency coefficients are discarded, could the authors compare this strategy with conventional low-pass filtering to demonstrate the specific benefit of wavelet decomposition?
  3. In Table 12, the results across the 400–1000 to 900–1000 ms windows appear to follow remarkably consistent linear trends across multiple metrics. Could the authors please double-check these values and confirm that the results for each time window were obtained from independent evaluations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.