acceptodds
Under review as a conference paper at ICLR 2027

Predictive Structure Is Distributed Across Depth in EEG Foundation Models

Abstract

Intermediate layers often outperform final layers when foundation models are used as frozen feature extractors, but the source of this advantage is unclear. We study this question in EEG foundation models by separating three factors: how representations are read out, whether intermediate layers retain predictive features that complement the final layer, and whether useful depth structure transfers across datasets. Across five models and three tasks, improving the final-layer readout consistently recovers substantial predictive performance, yet intermediate layers remain advantageous in most settings. Residual analyses further show that intermediate representations contain predictive variation that complements final-layer features, including cases where the intermediate layer is weaker on its own. This pattern persists with a nonlinear cross-layer mapping. We also find that layers selected on one dataset remain useful on another, and that a compact set of source-derived task directions retains predictive value after transfer. Together, these results show that intermediate-layer utility cannot be reduced to either poor final-layer pooling or better standalone representations. Instead, predictive structure is distributed across depth, and its usefulness depends on how representations are accessed and combined.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG foundation models, representation analysis, transfer learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_7dthOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors first show that the final layer classification readout is crucial for EEG FMs on downstream tasks and even after accounting for this, intermediate layers retain complementary task-predictive information that is not fully recoverable from the final representation. They also show this task-predictive information can partly transfer across downstream datasets.

Critical strengths
  1. The authors disentangle the contributions of readout design and layer-wise representations to downstream decoding performance. This is an interesting and often overlooked aspect in EEG FM evaluation. The experiments clearly demonstrate that both readout selection and representation depth can substantially influence performance.
  2. The cross-dataset transfer experiments provide additional evidence that task-relevant representations can generalize beyond a single dataset. The finding that source-selected layers and predictive directions remain useful on another dataset of the same task strengthens the results and highlights the potential transferability of these representations.
Critical weaknesses
  1. Limited mechanistic insight and novelty.

While the authors demonstrate that predictive information is distributed across layers, similar findings have been established in vision and language models. I feel that the investigation stops at a relatively superficial level, mainly demonstrating where predictive information can be accessed rather than explaining what that information represents or why it is distributed across depth. More specifically, for EEG, different downstream tasks rely on distinct physiological features. For example, ERP decoding depends on waveform morphology, peak amplitudes, and latencies, whereas MI relies more heavily on sensorimotor oscillations and their temporal dynamics. Investigating how these features are represented and transformed across layers would provide a much more meaningful explanation of the observed phenomenon. Furthermore, the analysis is not sufficiently connected to the pretraining objectives. MAE, JEPA, contrastive and autoregressive objectives may encourage different representations and potentially different depth-dependent behaviors. Although the evaluated models have different pretraining strategies, the current experiments do not disentangle the effects of the pretraining objectives from differences in architecture and tokenization.

  1. Limited practical relevance beyond frozen representation probing

The study only focuses on frozen EEG foundation models. However, previous benchmarks have shown that linear probing often performs substantially worse than full or parameter-efficient fine-tuning on downstream EEG tasks (Yang et al., 2026; Liu et al., 2026; Lu et al., 2026). This raises questions about the practical implications of the findings. During fine-tuning, the model can refit its representations according to downstream data, potentially making previously inaccessible task-relevant information available at the final layer. Consequently, the intermediate-layer advantages observed under frozen probing may diminish or even disappear after fine-tuning. While the paper provides useful guidance on improving frozen feature extraction through layer selection and better readouts, it remains unclear how these findings could improve EEG foundation models under more practical adaptation settings or inform the development of better pretrained representations.

References

Yang, Liuyin, et al. "Are EEG foundation models worth it? comparative evaluation with traditional decoders in diverse BCI tasks." The Fourteenth International Conference on Learning Representations. 2026.

Liu, Dingkun, et al. "EEG-FM-Compass: progress, benchmarking, and future directions for EEG foundation models." National Science Review (2026): nwag466.

Lu, Ziling, et al. "OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models." arXiv preprint arXiv:2606.00815 (2026).

Questions for authors
  1. Can the authors investigate which physiologically meaningful EEG features are represented at different depths, such as ERP waveform morphology and latency, MI-related mu/beta oscillations, or sleep-specific spectral and temporal patterns? Could these differences help explain the task-dependent layer preferences?

  2. To what extent is the observed distribution of predictive information determined by the pretraining objective? Would reconstruction-based, latent-prediction, and autoregressive models show similar depth-dependent behavior under a controlled comparison using the same backbone and pretraining data?

  3. Do the intermediate-layer advantages and complementary residual features persist after partial or full fine-tuning? In particular, does adapting the final layers to downstream supervision reduce the additional benefit of earlier representations?

  4. Have the authors examined whether layers selected for one task remain useful for another, such as selecting an intermediate layer on MI and evaluating it on ERP? This could help distinguish generally transferable representations from task-specific layer preferences.

  5. Interpretation of Figure 4: Since most of the performance degradation caused by removing source-derived task directions can be recovered through classifier refitting, to what extent do these directions represent uniquely valuable task information rather than features preferentially used by a particular classifier? Could the authors further clarify what these intervention results establish about the underlying representations?

6 comments
AuthorsOpenReview ↗

Rebuttal from the Author to Reviewer 7dth (1/2)

Dear Reviewer,

Thank you for recognizing the value of separating readout design from representation depth and for highlighting our cross-dataset transfer findings. Your questions pinpointed three decisive steps toward deepening our paper:

  1. grounding depth preferences in physiological measurements;
  2. isolating the role of pretraining objectives under controlled conditions;
  3. testing whether depth advantages survive downstream adaptation.

While our primary contribution establishes how readout choice, layer depth, and source-defined predictive directions govern frozen EEG evaluation and reuse, these new experiments anchor our observations in concrete physiological diagnostics and demonstrate how depth accessibility shifts with pretraining and adaptation. We agree that the original manuscript did not sufficiently connect these empirical patterns to physiological correlates or controlled pretraining comparisons.

We have completed new experiments addressing all three questions, alongside real MI–ERP cross-task layer transfer and equal-rank direction-intervention controls. We will integrate these results into the revised manuscript with the scope of each conclusion explicitly stated.

1. Physiologically grounded representation diagnostics across depth

We added direct decoding of two predefined physiological measurements across all twelve CBraMod layers using ridge regression and fixed readouts within participant-disjoint splits: log mean C3/C4 power over 8–30 Hz for OpenBMI (spanning the mu/beta range) and baseline-corrected mean PO8 amplitude over 150–200 ms for N170.

Physiological measurementParticipantsFinal-layer R²Final plus residual R²Paired R² gain [95% CI]
OpenBMI: log C3/C4 power, 8–30 Hz540.240.33+0.09 [0.05, 0.13]
N170: PO8 amplitude, 150–200 ms400.140.60+0.46 [0.35, 0.60]

The layerwise profiles show that both physiological quantities are more accessible at earlier depths. For N170 amplitude, R² stays at approximately 0.59–0.63 in layers H01–H05 before dropping to 0.14 at H12, and combining final-layer features with intermediate residuals restores R² to 0.60. For OpenBMI band power, the profile is less monotonic (0.44 at H02, 0.33 at H06, and 0.24 at H12). The sharper depth decay of N170-amplitude accessibility aligns closely with the larger classification depth benefit observed on N170, providing a concrete physiological association for our task-dependent findings.

2. Controlled comparison of pretraining schemes (MAE vs. JEPA)

We agree that comparing the five original foundation models cannot isolate pretraining objectives from differences in architecture, tokenization, and training data. In order to make direct mechanism analysis, We therefore trained paired MAE and JEPA models with an identical eight-block encoder, tokenizer, HBN data pool, 25-channel input protocol, sampling stream, temporal masking, optimizer schedule, and nominal data exposure across three paired seeds (implementation details will be listed in revised manuscript). MAE reconstructs masked waveform patches, whereas JEPA predicts normalized teacher representations.

For each model, depth gain is measured after fixing the validation-selected final-layer readout, and residual gain is final-plus-residual performance minus the final-layer baseline. The table reports JEPA-minus-MAE differences in these gains.

Task / metricDifference in depth gain (%) [95-CI]Difference in residual gain (%) [95-CI]
OpenBMI / BA+0.89 [−0.06, 1.83]+0.30 [−0.39, 0.99]
N400 / AUROC+3.32 [2.22, 4.48]+3.20 [2.35, 4.10]
N170 / AUROC+5.66 [3.96, 7.19]+9.65 [6.62, 12.23]

Both pretraining schemes exhibit depth benefits, with markedly larger JEPA-over-MAE depth and residual gains on the ERP tasks (while OpenBMI differences are smaller and span zero). Crucially, inspecting the absolute scores reveals a central evaluation insight: pretraining objectives can differ primarily in how predictive information is distributed across depth rather than in peak representational quality. On N170, MAE improves from 69.42 (final layer) to 74.42 (selected layer), whereas JEPA improves from 63.82 to 74.47; by comparison, a randomly initialized encoder scores 70.59 at the final layer and 70.47 after depth selection. Thus, evaluating only the final layer would make JEPA appear worse than both MAE and a random encoder, whereas accounting for depth shows that both pretrained models converge to virtually identical selected-layer performance and clearly outperform the random baseline.

AuthorsOpenReview ↗

Rebuttal from the Author to Reviewer 7dth (2/2)

3. Persistence after partial fine-tuning

Adapting final layers reduces the N170 depth advantage without eliminating it, and residual utility persists. We fine-tuned CBraMod's final two blocks on OpenBMI and N170 (3 seeds, first 10 blocks frozen, development splits for early stopping and layer selection), then froze the adapted encoder and refitted all diagnostic probes, preprocessing, and cross-layer mappings.

Task / metricFinal (frozen → adapted)Selected (frozen → adapted)Post-adaptation depth gain [95% CI]
OpenBMI / BA76.09 → 77.9276.17 → 77.920.00 [0.00, 0.00]
N170 / AUROC68.77 → 74.5377.45 → 77.45+2.92 [1.34, 4.40]

Since N170's selected layer is in the frozen blocks, its score stays 77.45 by construction. Key findings:

  1. Adapting the final two blocks recovers +5.76% [4.52, 6.98] at N170's final layer (68.77 → 74.53), improving feature accessibility.
  2. A +2.92% [1.34, 4.40] depth gap persists (down from ~8.67%).
  3. On the adapted encoder, intermediate residuals still beat the adapted final layer by +0.88% [0.37, 1.39] and shuffled residuals by +1.84% [1.01, 2.99].

On OpenBMI, final and selected scores match after adaptation, and residual augmentation yields −0.41% [−0.75, −0.08], closing a negligible frozen gap. Thus, depth and readout choices should be reassessed after adaptation. We will restrict our claim to partial fine-tuning in the revision.

4. Directional layer reuse across tasks

We added MI-to-ERP cross-task transfer using CBraMod (OpenBMI→N170, OpenBMI→N400, N170→OpenBMI). Source depth is the mode of source-validation-selected layers (ties broken toward shallower depth). On target tasks, we fix the target's validation-selected final readout and refit classifiers, so only layer identity transfers (unlike Section 4.3's fixed source-interface transfer).

Transfer / metricSource − final [95% CI]Source − random layers [95% CI]Source − first layer [95% CI]
OpenBMI→N170 / AUROC+9.50 [7.59, 11.35]+1.73 [0.87, 2.48]+0.11 [−0.67, 0.87]
OpenBMI→N400 / AUROC+6.76 [5.62, 7.89]+1.89 [1.43, 2.32]+0.12 [−0.75, 0.99]
N170→OpenBMI / BA−0.26 [−1.13, 0.60]+0.00 [−0.44, 0.43]+0.69 [0.02, 1.35]

Random-layer baseline averages all non-final layers (N170/N400: 40 subjects; OpenBMI: 54). MI-selected depth transfers well to both ERP tasks, outperforming final and random layers while matching the first layer (+0.11 and +0.12 pp, CIs spanning zero), indicating a reusable shallow region. N170→OpenBMI yields no gain. Cross-task layer reuse is directional and depends on target signal properties (added to Section 4.3 and Appendix F.2.3).

5. What Figure 4 establishes after classifier refitting

We agree that the large drop under a frozen classifier reflects reliance on removed coordinates, while post-refit recovery shows alternative predictive features remain.

To isolate direction utility beyond classifier dependence, we added equal-rank deletion (removing 4 dimensions from the source-fitted PCA200 space, averaged over 20 random and 20 label-shuffled bases; energy-matched perturbations reported separately) and positive augmentation (appending 4 source residual coordinates to target final features).

HMC→ISRUC model4-coord. aug. − final [95% CI]Extra post-refit loss vs. random deletion [95% CI]
CBraMod+1.17 [0.72, 1.69]1.70 [1.24, 2.20]
REVE+0.67 [0.30, 1.07]0.78 [0.31, 1.28]
LaBraM+0.97 [0.67, 1.28]2.17 [1.70, 2.65]

Across 98 ISRUC subjects (paired bootstrap on macro-F1), augmentation also beats equal-rank random augmentation by 1.01%, 0.63%, and 0.92%. Positive augmentation and persistent post-refit excess deletion loss show that source residual directions carry genuine, transferable utility under tested interfaces. Recovery and transferable utility coexist across three model families. We will revise Section 4.4, Figure 4, and Tables S15–S16 accordingly.

Thank you for the detailed response and the additional experiments. I appreciate the authors' openness in discussing the results, including where the new findings challenge the original claims. I have also read the responses to the other reviewers. Unfortunately, the new controls have made me less convinced of the paper's main claims.

  1. Complementary information across layers. In the response to Reviewer PC6L, when the final layer is kept at its native dimension, the residual gains drop from +9.57 / +8.96 / +6.11 / +2.12 to +1.22 / +0.48 / +0.71 / −0.01. This suggests that much of the apparent "complementary" information comes from compressing the final layer to 200 PCA dimensions, rather than from information that the final layer does not contain. The remaining gains are around 1 percentage point or less. I think this substantially weakens the paper's second main claim. Could the authors also clarify whether the +0.88 residual gain after fine-tuning (reported in the response to my review) was obtained using the PCA200 final layer?

  2. Physiological diagnostics (my Q1). Thank you for adding these analyses. However, the mean PO8 amplitude in the 150–200 ms window is a linear function of the raw input, while band power is also relatively straightforward to extract from the signal. The fact that early layers decode these features better (R² around 0.6 in H01–H05 versus 0.14 at H12) may simply indicate that deeper layers progressively lose information about the original input. This does not yet tell us much about what different layers actually represent, or why. I think a raw-input baseline using the same ridge readout, together with a randomly initialized encoder, would be important for interpreting these results.

  3. Pretraining objectives (my Q2). The controlled MAE/JEPA comparison is useful. However, on N170, the randomly initialized encoder (70.59) outperforms the final layers of both MAE (69.42) and JEPA (63.82), while the selected layers reach around 74.4. Together with point 2 and the cross-task results, where the source-selected layer performs almost identically to the first layer (+0.11 and +0.12, with intervals including zero), this suggests a simpler interpretation: for ERP tasks, layers closer to the input may work best because deeper layers in the current models discard waveform-level information. I find this observation interesting in itself, but it supports a different conclusion than the claim that predictive information is distributed across depth.

  4. Fine-tuning (my Q3). In the new experiments, only the last two blocks are adapted, while the selected N170 layer lies within the frozen blocks and therefore cannot change. This leaves my original question largely unanswered, as the experiment does not test full fine-tuning or parameter-efficient adaptation methods such as LoRA, which are more representative of the practical settings I was interested in.

  5. Minor clarification. The REVE OpenBMI→BCIC-IV-2a source-minus-final gain is reported as +2.28 [0.42, 4.24] in the response to Reviewer PC6L, but as +2.74 in the new transfer table. Could the authors clarify the discrepancy?

Overall, I think the new experiments reveal interesting findings, but they also suggest the paper needs a more substantial reframing, along the lines the authors themselves have outlined and that Reviewer 3nUD has also suggested. In particular, the distinction between genuinely complementary information across layers and the preservation of input-level information in earlier layers seems important for interpreting the results.

Since a revised manuscript reflecting this reframing is not yet available, I cannot assess how well the authors would address these issues in the current round. I therefore maintain my rating. Nevertheless, I appreciate the authors' efforts and believe this work could become a strong paper with clearer interpretation and more carefully scoped claims.

Reviewer_PC6LOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how predictive information is distributed across different layers of EEG foundation models. The authors examine three questions: whether better readout strategies can recover final-layer performance, whether intermediate layers provide complementary predictive information, and whether useful layer-specific representations transfer across datasets. Experiments are conducted using five EEG foundation models across several downstream tasks. The results show that improved readouts substantially increase final-layer performance, while intermediate representations still provide additional predictive value in many settings. Residual analyses and cross-dataset transfer experiments further support the usefulness of intermediate-layer features.

Critical strengths

The paper investigates an interesting and practically relevant question about representation learning in EEG foundation models. I particularly appreciate that the authors separate the effects of readout selection and layer selection, rather than directly attributing intermediate-layer improvements to better representations.

The experimental analysis is relatively comprehensive. The evaluation covers five EEG foundation models, multiple downstream tasks, and participant-disjoint validation. The residual analysis, permutation controls, and cross-dataset experiments provide additional evidence beyond conventional layer-wise performance comparisons.

Critical weaknesses

The interpretation of complementary information across layers needs further justification. The residual analysis mainly relies on PCA-reduced representations and relatively simple linear or nonlinear reconstruction models. Therefore, it is difficult to determine whether intermediate layers genuinely retain information unavailable in the final layer, or whether this information is simply difficult to recover under the selected representation and reconstruction settings. The negative augmentation results with unreduced ERP features in Table S14 also suggest that the conclusions are sensitive to the representation dimensionality and probing protocol.

The generalizability of the transfer findings remains somewhat limited. The cross-dataset experiments focus on only two foundation models and two dataset pairs. In particular, the motor imagery transfer experiment involves only nine target participants, and the exact sign-flip test for REVE does not reach the conventional 0.05 significance level. Additional transfer settings would make the conclusions about reusable predictive structure more convincing.

Questions for authors

Could the authors further investigate how sensitive the residual analysis is to the representation dimensionality and the capacity of the cross-layer mapping? E.g., Table S14 reports negative augmentation gains for unreduced ERP representations. It would be useful to clarify how these findings affect the interpretation of complementary predictive information across layers.

For the cross-dataset transfer experiments, a broader evaluation would help establish whether the observed transferability is a general property of EEG foundation models.

2 comments
AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer PC6L (1/2)

Dear Reviewer,

Thank you for recognizing the value of separating readout selection from layer selection, and for your constructive suggestions to strengthen the soundness and generality of our claims. We have refined the interpretation of the residual analysis with new dimensionality and capacity sensitivity checks, and substantially expanded the cross-dataset transfer evaluation across additional datasets, task families, and a third model family. We will incorporate these results into the revised manuscript and state the scope of each conclusion explicitly.

1. Interpretation and sensitivity of the residual analysis

We will sharpen the interpretation of our residual analysis so that it matches its exact operational scope. Rather than claiming that the raw final hidden state has irreversibly lost task information, the analysis tests whether intermediate-layer variation unexplained by a specified cross-layer mapping provides incremental predictive utility beyond the tested final-layer interface. Likewise, we do not present residual augmentation as a universally superior decoder over a well-chosen single layer.

We agree that the negative unreduced ERP results in Table S14 (where augmentation decreases AUROC by 6.18–18.32 percentage points across those four conditions) are an important boundary condition. We will retain these results and explicitly discuss how high-dimensional estimation variance affects unreduced concatenation.

Following your suggestion, we conducted a dimension scan for CBraMod and REVE on N170 and N400 across PCA100/200/400 and native readout dimensions, holding the layer and readout fixed within each fold. Crucially, we also tested a targeted asymmetric control: keeping the intermediate representation at 200 dimensions while expanding the final representation to its full native readout dimension. This directly isolates whether residual utility is merely an artifact of compressing final-layer features.

The table below reports final-plus-residual minus final-layer AUROC using cross-fitted training residuals and participant-disjoint evaluation.

Task / modelFinal PCA200, intermediate PCA200Final native, intermediate PCA200
N170 / CBraMod+9.57 [7.33, 11.83]+1.22 [0.37, 2.02]
N170 / REVE+8.96 [7.04, 10.87]+0.48 [0.31, 0.66]
N400 / CBraMod+6.11 [4.55, 7.66]+0.71 [0.44, 0.98]
N400 / REVE+2.12 [1.14, 3.07]−0.01 [−0.15, 0.11]

Here, native readout dimensions are 5,600 for CBraMod and 14,336 for REVE. Expanding the final representation to its full native dimension reduces the gain magnitude relative to PCA200, and when both representations remain unreduced, the scan still does not establish a positive augmentation benefit across these four conditions. Nevertheless, even with no compression bottleneck on the final layer (5,600 and 14,336 dimensions), three of the four conditions retain statistically positive gains from the 200-dimensional intermediate residuals, while REVE/N400 does not. Thus, the evidence demonstrates genuine incremental utility under specified representation and probe settings, with clear and transparent dimension dependence. We will report this complete scan alongside, rather than in place of, the original Table S14 results.

2. Mapping capacity and task-head capacity sensitivity checks

To test sensitivity to mapper and classifier capacity, we varied both components across three diagnostic conditions (CBraMod/OpenBMI, CBraMod/N170, and CSBrain/N400), comparing ridge, a one-hidden-layer MLP, and a regularized two-hidden-layer MLP (width 512) paired with either a linear or nonlinear prediction head. Within each comparison, the same head family is used for the final-layer baseline, residual augmentation, and size-matched shuffled-residual augmentation.

On CBraMod/OpenBMI, for instance, pairing the regularized nonlinear mapper with a nonlinear prediction head yields +0.94 [0.33, 1.56] over the final-layer baseline and +1.33 [0.60, 2.08] over shuffled-residual augmentation, confirming that incremental residual utility persists under both nonlinear reconstruction and nonlinear probing. At the same time, because the wider MLP did not improve held-out reconstruction R2R^2 over ridge in these diagnostic checks, we explicitly note that these controls do not claim to exhaust all possible nonlinear mapping capacity.

AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer PC6L (2/2)

3. Additional datasets and large target cohorts extend layer reuse to four task families

To directly address the generalizability concern, we expanded our evaluation with a larger MI target cohort and two additional task families (SSVEP and P300). Together with the original sleep and MI pairs, the layer-transfer benchmark now spans five dataset pairs across four task families (as we keep original MI transfer with 9 participants).

For each pair, we compare the source-selected layer against the mean performance across all non-final layers, holding the readout and target splits fixed. This random-layer baseline tests whether source-side depth selection adds value beyond the general utility of intermediate layers. Target labels are used only to refit the classifier, while layer selection is performed strictly on the source. The table reports source-selected minus random-layer balanced accuracy (An asterisk marks a 95% confidence interval that includes zero.).

Additional transfer / taskTarget participantsCBraModREVE
OpenBMI→WBCIC-SHU / MI51+0.31*+0.77
Benchmark→BETA / SSVEP69+3.54+3.65
ERP CORE P3→Won2022 / P30055+0.67+0.07*

These additions expand MI evaluation beyond the original nine-participant target to an additional 51-participant cohort and demonstrate that source-layer reuse generalizes beyond sleep and MI. In particular, SSVEP shows substantial gains in both models (+3.54% and +3.65% across 69 participants), while the 51-participant MI and 55-participant P300 cohorts clarify task- and architecture-dependent boundaries.

4. A third model family replicates the sleep-transfer findings**

We further extended HMC→ISRUC (98 target participants) to a third model family, LaBraM. Source-selected depth improves macro-F1 by +1.07 [0.61, 1.55] over the final layer and +0.65 [0.30, 1.00] over the random-layer expectation. Its absolute score reaches 70.29, closely matching the 70.27 achieved by target-validation-selected depth, confirming that source-side selection remains effective without target-side layer search.

Under the separate PCA200 direction-transfer protocol, appending four source-defined residual coordinates to LaBraM's target final-layer features yields +0.97 [0.67, 1.28] macro-F1, replicating the sleep direction-transfer result in a third architecture. We will clearly distinguish this three-model replication on HMC→ISRUC from the broader four-task layer-reuse evaluation.

5. Retaining transparency on the original REVE/MI cohort

We agree with your assessment of the original nine-participant REVE/BCIC-IV-2a experiment: its source-versus-final gain is +2.28 [0.42, 4.24] with positive differences in 7/9 participants, yet the two-sided exact sign-flip test yields p = 0.0781. We will retain this exact-test result and describe it candidly as a positive estimate with substantial small-sample uncertainty. While the additional transfer experiments (51, 55, and 69 participants) and third-model replication on the existing 98-participant sleep cohort establish the broader generality of source-interface transfer, they do not alter the statistical qualification on this original nine-participant comparison.

We will expand section 4.3 and Table1 from original manuscript with the new transfer pairs and random-layer controls, and separate layer reuse from direction transfer in section 4.4 and the Discussion. Together, these results support a clear, well-bounded conclusion: source-selected interfaces provide practical within-task reuse across datasets and model families, outperforming explicit layer controls while exhibiting transparent task- and architecture-dependent limits.

Reviewer_8KnLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors study how to extract useful features from a frozen EEG model. Across five models and three main tasks, changing how the final layer is read improves prediction. Intermediate layers often provide a further gain. In the sleep staging experiments, some feature directions learned on one dataset also help prediction on another.

Critical strengths

The paper addresses a practical question. When reusing a pretrained EEG model, a researcher needs to decide which layer to use and how to combine its outputs across channels and time. The experiments show that these choices can affect performance.

Critical weaknesses

My main concern is novelty in light of the information bottleneck literature. Tishby and Zaslavsky (2015) analyze layer representations in terms of compression and the information they retain about a target. This motivates asking what earlier layers may still offer on a different task. Evci et al. (2022) directly demonstrated the value of selecting features across layers for visual transfer. In EEG, Širca et al. (2026) found that pooling accounted for much of the weak performance they observed when training only the classification head. Given these precedents, I would like to understand what the cross-dataset intervention in Table S15 teaches us beyond the usefulness of intermediate features in another setting.

Questions for authors

Does Table S15 reveal a property of the transferred features that goes beyond what the information bottleneck view and earlier layer-transfer experiments would lead us to expect? If so, which result shows it?

4 comments
AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 8KnL (1/3)

Dear Reviewer,

Thank you for recognizing the practical importance of determining how to access frozen EEG representations. Our central contribution investigates what can be reused across datasets:

  1. an intermediate layer selected on the source dataset,
  2. task-relevant directions identified within that layer.

The former establishes a practical cross-dataset transfer strategy, while the latter probes the geometric stability of the transferred features. We agree that the original manuscript did not sufficiently foreground this connection, and we will restructure the presentation to make it explicit.

1. Positioning cross-dataset layer reuse beyond single-dataset selection and pooling

While the information-bottleneck perspective motivates examining what different layers retain about a target [Tishby and Zaslavsky, 2015, prior work primarily evaluates representation depth within a single domain. Specifically, Head2Toe selects intermediate features using target-task training data, and Širca et al. establish the importance of pooling across depth on EEG. Building on these findings, we ask a distinct, deployment-oriented question: can a layer selected on one EEG dataset remain effective on another without repeating layer selection on the target?

This question is central to within-task cross-dataset reuse, where the prediction task remains unchanged across shifts in acquisition hardware and participant cohorts. Notably, the existence of useful intermediate layers on individual datasets does not guarantee that a source-selected layer will generalize across cohorts. To test this directly, we keep the encoder frozen, transfer the source-selected layer, and refit only the linear classifier on target labels.

2. Source-selected layers generalize across datasets and task paradigms

To isolate the effect of source-side selection, we added random-layer and target-selected-layer controls, and expanded our evaluation to an additional motor imagery (MI) dataset as well as two new task paradigms: SSVEP and P300. Within each transfer pair, the readout and target splits are held fixed. Final uses the final encoder block with the same readout as Source, while Random reflects the uniform expectation across all non-final blocks (each fitted independently). All strategies fit a target classifier and are evaluated on held-out participants.

The table below reports performance gains in percentage points (ΔFinal=Source−Final\Delta\text{Final} = \text{Source} - \text{Final}; ΔRandom=Source−Random\Delta\text{Random} = \text{Source} - \text{Random}), where an asterisk (*) indicates a paired 95% confidence interval (CI) that includes zero:

Transfer / TaskTarget ParticipantsCBraMod ΔFinal\Delta\text{Final}CBraMod ΔRandom\Delta\text{Random}REVE ΔFinal\Delta\text{Final}REVE ΔRandom\Delta\text{Random}
HMC →\rightarrow ISRUC / Sleep staging98+1.20+0.64+1.13+3.48
OpenBMI →\rightarrow BCIC-IV-2a / MI9+5.02+2.56+2.74+3.72
OpenBMI →\rightarrow WBCIC-SHU / MI51+1.45+0.31*+0.88+0.77
Benchmark →\rightarrow BETA / SSVEP69+9.39+3.54+12.75+3.65
ERP CORE P3 →\rightarrow Won2022 / P30055+4.72+0.67+1.11+0.07*

Source-selected layers outperform the final layer across all 10 model–transfer pairs, and these gains span five dataset pairs and four distinct EEG paradigms. Together, these controls disentangle the general benefit of accessing intermediate layers from the specific value of selecting them on the source dataset.

3. Source selection closely matches target-side layer selection

On HMC →\rightarrow ISRUC and OpenBMI →\rightarrow BCIC-IV-2a, source-selected layers perform within −0.19-0.19 to +0.27+0.27% of target-validation-selected layers across both CBraMod and REVE (under the same source readout). This indicates that a source-selected layer can serve as a reliable proxy without requiring target-side layer search.

We further replicated this result on a third foundation model, LaBraM, on HMC →\rightarrow ISRUC: source selection outperforms Final by +1.07+1.07 % and Random by +0.65+0.65 %, while differing from target-side selection by only +0.02+0.02 %.

AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 8KnL (2/3)

4. What the direction intervention establishes

Table S15 tests a finer-grained hypothesis: whether specific source-defined directions within a selected layer retain predictive utility on an unseen target dataset. Even when a source-selected layer transfers well at the representation level, its informative subspaces could still misalign across cohorts. Our direction-level interventions directly probe this alignment. While consistent with the information-bottleneck perspective, this analysis tests a subspace property that layer-level accuracy alone cannot establish.

On HMC →\rightarrow ISRUC, we extract source-fitted PCA200 features, fit a linear mapping from the final layer to the selected intermediate layer on HMC, and identify four task directions from the resulting residuals. All transformations are frozen and applied directly to ISRUC, where we either append these four residual coordinates to the target final-layer features or ablate the corresponding four directions from the target selected-layer features. (This PCA200 subspace protocol is distinct from the full-layer selection comparisons in Rebuttal 1/3.)

The table below reports macro-F1 differences along with paired whole-participant bootstrap 95% CIs across all 98 ISRUC participants (conditioned on the fitted source transformations). All augmentation baselines append exactly four coordinates and use an identical target-classifier fitting procedure:

ComparisonCBraModREVELaBraM
Residual-coordinate augmentation −- final layer+1.17 [0.72, 1.69]+0.67 [0.30, 1.07]+0.97 [0.67, 1.28]
Residual-coordinate augmentation −- random-coordinate augmentation+1.01 [0.60, 1.50]+0.63 [0.27, 1.01]+0.92 [0.63, 1.22]
Residual-coordinate augmentation −- ordinary supervised-coordinate augmentation+0.54 [0.13, 1.02]+0.44 [0.05, 0.83]+0.58 [0.26, 0.91]
Extra loss from task-direction deletion vs. random deletion (after refitting)1.70 [1.24, 2.20]0.78 [0.31, 1.28]2.17 [1.70, 2.65]

Appending source-defined residual coordinates consistently improves target prediction over the final layer across all three foundation models. Moreover, residual-derived directions outperform ordinary supervised directions extracted directly from the unresidualized source intermediate layer (+0.54+0.54, +0.44+0.44, and +0.58+0.58 %). Applying both bases to the same residual representation yields consistent gains of +0.51+0.51, +0.55+0.55, and +0.59+0.59 % for CBraMod, REVE, and LaBraM, respectively—isolating the benefit of the residual direction-identification procedure beyond random or standard supervised subspace controls.

AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 8KnL (3/3)

5. Post-refit ablation provides complementary evidence

We also standardized the ablation controls so that every deletion removes a subspace of identical rank (exactly four dimensions). For the null distributions, we independently evaluate 20 random bases and 20 label-shuffled bases and report their mean performance; energy-matched perturbations are reported separately.

The severe performance drops observed with a frozen classifier in our original Table S15 primarily reflected the source classifier's reliance on those specific coordinates. When the target classifier is refitted, much of that loss is recovered, showing that redundant predictive cues remain available elsewhere in the layer. Crucially, however, a statistically significant excess loss persists relative to equal-rank random deletion across all models (+0.78+0.78 to +2.17+2.17 %), which demonstrates that these source-identified directions carry transferable predictive utility. Label-shuffled controls corroborate this finding.

We will revise our manuscript as soon as possible to reframe our contribution.

I appreciate the authors’ detailed rebuttal. Nevertheless, I stick to my recommendation of rejection. The results constitute incremental empirical evidence specific to EEG, yet no new theoretical findings are demonstrated.

Reviewer_3nUDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper asks why intermediate layers of EEG foundation models often give better downstream decoding than the final layer. It covers 5 models and 3 tasks. It tests 3 hypotheses:

  • H1: the final layer looks weak because the selection of the linear probing strategy is not done correctly. To do this it compares 5 fixed readouts (global mean, global mean + temporal SD, channel-wise mean, channel-wise mean + temporal SD and temporal binning. Best readout is picked at the final layer on validation data.
  • H2: intermediate layers contain information the final layer does not. They predict intermediate features from final-layer features with a ridge regression. What cannot be predicted is called the residual. They test whether adding this residual to the final layer features improves decoding compared with same residual shuffled across trials
  • H3: this depth advantage transfers to a new dataset of the same task. They pick the layer and readout on one dataset and reuse them unchanged on another (HMC to ISRUC for sleep staging and OpenBMI to BCIC-IV-2a for motor imagery). For sleep they also extract class-separating directions from the first dataset’s residuals, check that removing them on the second dataset hurts the classifier and that adding them to the final layer helps
Critical strengths
  • The question is useful. Evaluation of EEG foundation models is everywhere and often relies on naive linear probing.
  • They show that the readout choice alone has a large effect (e.g. +25 points on OpenBMI).
Critical weaknesses
  • The paper feels unfinished. The ideas are interesting, but most analyses end without a clear rule to take away. The topic is ideal to propose concrete recommendation e.g. first select the readout on held-out data, then scan intermediate layers or the other way around. I would also re-report the performance of each foundation model and show how their ranking changes on each task depending on the readout and the layer selection. Even a new metric would be very useful for the community, e.g. how robust a model is to the readout choice, or how stable its representations are across layers (which could also motivate training strategies that do not lose information across depth).

  • Figures are unclear. Figure 2: the legend is split across the two panels (colour = task in one, symbol = model in the other); use one shared legend. Table 1 mixes absolute scores and gains and is hard to read; a simple plot of the gains would be clearer. Figure 4 is really hard to parse, as it only shows differences against controls instead of actual scores. It is hard for the reader to extract what each figure is meant to show.

  • The most informative results are in the appendix. Fig. S1 is for me the clearest figure of the paper: in one view it shows that the readout matters much more than the layer (on OpenBMI, global-mean curves sit 15 to 25 points below channel-preserving readouts at every layer), and that for the ERP tasks the best layers are early ones. It should be in the main text, possibly replacing main Figure 2... Table S4 is also interesting: the readout gain grows a lot with more training labels (+8 to 9 points from 10% to 100% labels on OpenBMI) while the depth gain barely changes, which is useful for practitioners.

  • The residual analysis has little practical value as shown. In Fig. 3B, final layer + residual almost never beats the selected intermediate layer on its own (e.g. REVE/OpenBMI 74.8 vs 73.6, CBraMod/N400 76.8 vs 75.5). Only CBraMod/OpenBMI beats both, by less than 1 point. So the residual mostly recovers what the intermediate layer already gives.

  • Cross-dataset transfer. There is no comparison with the layer selected directly on the target dataset, nor with a random intermediate layer, so we cannot tell whether the source choice really transfers or whether almost any intermediate layer beats the final one (Fig. S1 suggests the latter).

I think his paper could be very useful for the community, but as it is framed now it is hard to see what the meaningful findings are, which is a shame. With a major reframing around clear, actionable takeaways, I think it could become a strong paper.

Questions for authors
  • My main question is a request: can you reframe the paper to give clear takeaways and one clear message per figure

Others, less critical:

  • Does the ranking of the foundation models change when each is evaluated with the selected readout and layer instead of final-layer mean pooling?
  • How does simply concatenating several layers (or learning a weight per layer) compare with selecting one layer or adding residuals?
  • In the transfer experiments, how does the source-selected layer compare with the best layer selected on the target and with a random intermediate layer?
  • Why does CBraMod select layer 1 for motor imagery, and how does it compare with a classical baseline such as CSP+LDA?
  • Why does neither the readout nor the depth help on SEED-V? Is it a limit of the approach for this kind of task?
6 comments
AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 3nUD (1/3)

Dear Reviewer:

Thank you for recognizing the importance of evaluating EEG foundation models beyond naive linear probing and for your concrete suggestions. We agree that the practical findings need to be stated more clearly. We will comprehensively revise the manuscript shortly, including its framing, main figures, and discussion, to incorporate the additional experiments and clarifications. In this rebuttal, we first address your concerns point by point and present the supporting results.

1. Clear takeaways and actionable evaluation protocols

We recommend selecting the final-layer readout first, then evaluating depth with that readout fixed. All selection uses development participants, followed by evaluation on separate test participants. The strategy can be decomposed into following steps for clarity:

StepDecisionPractical purpose
1Select the final-layer readoutEstablish a strong final-layer baseline.
2Select depth at that fixed readoutMeasure the additional value of intermediate layers.
3Reuse the source-selected interface on a new datasetAvoid repeating target layer search; refit the target classifier.

For example, in OpenBMI and N170 task with CBraMod, we have following improvements:

CBraModFinal meanSelected final readoutSelected depth
OpenBMI50.4276.0976.17
N17057.1068.7777.45

And there is another actionable protocol by increasing probe-fitting labels, Table S4 from original manuscript has already shown that

OpenBMI modelIncrease in readout gainIncrease in joint-selection gain beyond the selected final readout
CBraMod+8.82+0.38
LaBraM+8.77+1.27
CSBrain+8.43+1.91

The validation pool is fixed. More fitting labels do not remove the importance of readout choice. We will bring this finding into the main discussion and report readout sensitivity, fixed-readout depth gains, and selection frequencies alongside absolute scores.

2. Model ranking do changed with the evaluation protocols

Yes, changing the evaluation protocols actually change the model ranking. Averaging ranks across the three tasks reveals substantial shifts: REVE improves from 1.67 to 1.00, CBraMod improves from 4.00 to 2.67, while CSBrain and LUNA worsen (1.67→2.67 and 4.00→5.00, respectively).

ModelMean+ Readout+ Depth
REVE1.671.331.00
CSBrain1.672.002.67
CBraMod4.003.002.67
LaBraM3.674.333.67
LUNA4.004.335.00

As for individual ranking, CBraMod moves from fifth to second on OpenBMI; REVE overtakes CSBrain after depth selection on N170. The detailed changed in ranking among each tasks are listed in the appendix of revised manuscript.

AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 3nUD (2/3)

3. Practical role of residual analysis and multilayer fusion

We agree that the results do not establish residual augmentation as a generally better decoder than using a well-chosen single layer. We will tone down its practical role and present residualization primarily as an exploratory analysis of incremental predictive utility. The detailed residual-augmentation and multilayer-fusion comparisons below, together with their protocols, will be included in the Supplementary Material of the revised manuscript. The main text will retain a brief summary of the practical takeaway and the scope of the analysis.

To address your suggestion, we added comparisons with direct concatenation and learned layer weights. All compact methods use 200 total classifier-input features and the same participant-disjoint evaluation splits. The three-layer methods use the first, middle, and final layers; learned scalar fusion adds three trainable layer-weight parameters. Scores below are participant-averaged BA or AUROC, expressed as percentages, with 54 participants for OpenBMI and 40 for N170.

Task / modelFinalSelectedFinal + residualFinal + selectedThree-layer concat.Learned layer weights
OpenBMI BA / CBraMod69.8668.2268.5568.4165.7666.15
OpenBMI BA / REVE66.3774.1470.1170.6867.2869.17
N170 AUROC / CBraMod64.8574.5970.7771.3771.2374.28
N170 AUROC / REVE68.2378.2474.3775.0375.0371.92

Neither three-layer method has a higher point estimate than Selected in these four conditions. Residual augmentation exceeds Selected only for CBraMod/OpenBMI, by 0.33 percentage points, with a paired 95% confidence interval of [−0.67, 1.34]; even there, Final remains stronger. Thus, these comparisons do not demonstrate a practical advantage of residual augmentation over the stronger single-layer baseline. They support evaluating Final and Selected before adopting more complex fusion methods.

The comparison also depends on the feature budget. Removing dimensionality reduction from three-layer concatenation gives the following results:

Task / modelSelected, 200 featuresUncompressed three-layer concatenation
OpenBMI BA / CBraMod68.2276.19
OpenBMI BA / REVE74.1479.05
N170 AUROC / CBraMod74.5977.70
N170 AUROC / REVE78.2478.87

These are unequal-budget comparisons: the selected-layer baseline remains restricted to 200 features, whereas concatenation uses uncompressed features from three layers. They therefore do not establish that concatenation is superior to an uncompressed selected layer. We will report them separately from the matched-budget comparisons in the Supplementary Material.

Accordingly, we will use residualization to investigate a narrower question: whether intermediate-layer variation unexplained by a specified cross-layer mapping provides additional predictive utility beyond the tested final-layer interface. We will revise the main text and conclusions to make this distinction explicit.

4. Source-selected versus random and target-selected layers

We ran both requested controls. In every comparison, the source readout and target splits are fixed. “Random” is the mean score across all non-final layers. Target selection uses target development participants; all strategies refit the target classifier and are evaluated on held-out participants. We extended Table 1 with the target-selected-layer result and the random-intermediate-layer result:

Transfer / modelFinalRandomSourceTargetSource − random [95% CI]
Sleep / CBraMod70.5671.1271.7671.56+0.64 [0.28, 1.01]
Sleep / REVE71.6669.3072.7972.97+3.48 [2.72, 4.32]
MI / CBraMod66.9869.4371.9971.72+2.56 [1.80, 3.21]
MI / REVE77.0876.1079.8279.82+3.72 [2.39, 4.98]

In above comparison, Source selection exceeds random-layer expectation and closely matches target selection. Source-minus-target differences range from −0.19 to +0.27. Fixed-layer controls also show that the source choice coincides with CBraMod’s first layer and REVE’s middle layer on MI.

In order to demonstrate that such transfer can be more generalizable, we add following cross-dataset transfer experiment and more task to examine this phenomenon (An asterisk marks a 95% confidence interval that includes zero.):

Transfer / taskTarget participantsCBraMod gainREVE gain
OpenBMI→WBCIC-SHU / MI51+0.31*+0.77
Benchmark→BETA / SSVEP69+3.54+3.65
ERP CORE P3→Won2022 / P30055+0.67+0.07*

Overall, our within-task cross-dataset transfer offers a practical way to find a reusable choice without repeating the target layer search.

AuthorsOpenReview ↗

Rebuttal From the Author to Reviewer 3nUD (3/3)

5. Why H01, and comparison with CSP+LDA

H01 comes from the OpenBMI→BCIC protocol with 21 common channels and a fixed channel mean + standard deviation readout. Its source-validation advantage over H03 is very small:

CBraMod layerOpenBMI source-validation BA
H0171.18
H0371.14

This supports a useful shallow region. The input and selection protocol differs from the main within-dataset experiment, so the selected layer can differ.

We also added CSP with shrinkage LDA on the same accepted left/right trials, common channels, and participant splits. Frequency band and component count are selected on inner development participants.

MethodBCIC-IV-2a BA
CSP + shrinkage LDA62.62
CBraMod final layer, full readout66.98
CBraMod H01, full readout71.99

H01 exceeds CSP+LDA by 9.38% (with CI95 [2.93, 15.47]). This establishes the requested classical baseline comparison; the near-tied source scores support recommending a shallow region rather than assigning H01 a unique physiological role.

6. Readout and depth on SEED-V

SEED-V exposes a limitation of the gains obtainable through our current frozen-feature evaluation: selecting depth does not provide a reliable improvement over the selected final-layer readout. However, one clarification is that readout selection does help some models. The results below separate readout improvement from the additional effect of selecting depth while holding the selected readout fixed. All gains are in balanced-accuracy percentage points.

ModelReadout gainAdditional depth gain [95% CI]
CBraMod+5.00+0.28 [−2.78, 3.61]
REVE−1.67+0.69 [−1.94, 3.19]
LaBraM+3.47−2.50 [−5.97, 0.83]
LUNA+1.25−0.56 [−4.31, 2.92]
CSBrain+0.28+1.11 [−1.53, 4.03]

A possible task-related limitation lies in the evaluation interface. Our SEED-V protocol averages window-level class probabilities within each video; it does not learn a video-level temporal model. If useful emotion-related information requires temporal relationships that this aggregation discards, changing the readout or layer alone may be insufficient.The cohort of 16 participants also leaves uncertainty about smaller effects, but this does not provide positive evidence that depth selection would help with more data.

Thus, we regard SEED-V as an empirical boundary of the current approach under the tested models, readouts, and participant-disjoint protocol. The revised manuscript will state this limitation in the main discussion and include the full results and selection diagnostics in the Supplementary Material.

7. Clear figures and stronger main-text takeaways

We deeply thanked that the reviewer thought our paper will be useful for community. We will re-organize our paper to make it more clearly as:

Reviewer SuggestionPlanned revisionMain Takeover
Fig. 2 and split legendsBring representative Fig. S1 profiles into the main text; use a shared legend.When readout explains weak final-layer performance, and when depth adds value.
Missing practical comparisonsShow protocol-specific scores and rankings; discuss Table S4.Interface choices affect model comparison and remain important with more labels.
Table 1 mixes scores and gainsSeparate absolute transfer scores from paired gains and intervals.Whether source selection improves on random layers and approaches target selection.
Fig. 4 lacks absolute contextShow augmentation scores and intact → deletion → refitting scores, with matched controls.Which source-defined directions help target prediction and how much performance recovers.

And we will also add some new visualization from Appendix to make our results more understandable. The forthcoming revision will connect each main result to an explicit evaluation decision, with detailed controls retained in the supplement.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to the Reviewers: Revised Manuscript Uploaded

Dear Reviewers,

Thank you all for your constructive feedback and the continued discussion. We have now uploaded a substantially revised manuscript, incorporating the additional experiments and controls presented during the rebuttal. We have also reorganized the paper to clarify our central contributions, distinguish supported findings from limitations, and improve the presentation. Major revisions are highlighted in blue for ease of review.

We summarize the key revisions below:

  1. L41–72, L469–519: Reframe the paper around predictive accessibility and reusable layer–readout interfaces, rather than claiming that intermediate layers contain information intrinsically absent from the final layer. The revised Introduction and Discussion clarify our contributions and practical implications. @8KnL, @7dth, @3nUD

  2. L288–340, Figure 2: Redesign the main evaluation figures to present representative layer–readout profiles, readout and depth gains, and changes in model rankings. Additional learned-readout and label-budget controls are consolidated in Appendix B. @3nUD

  3. L344–359, Appendix C: Add systematic representation-dimensionality, decoder-capacity, and matched-budget fusion controls. We explicitly acknowledge that residual augmentation gains diminish substantially with native-dimensional features and do not consistently exceed the stronger single-layer baseline. @PC6L, @7dth, @3nUD

  4. L364–396, Figure 3, Table 1: Expand cross-dataset transfer evaluation to five dataset pairs across four task families, with comparisons against Final, Random, First, and target-selected layers. Source-selected layers outperform Final in all ten CBraMod/REVE comparisons, with positive intervals over Random in eight. @PC6L, @3nUD, @8KnL

  5. L409–423, Figure 4, Appendix E: Strengthen the source-derived subspace transfer analysis with ordinary supervised-coordinate controls, rank-matched ablations, and post-refitting evaluations. These experiments provide evidence for transferable predictive directions while distinguishing classifier dependence from recoverable information. @8KnL, @PC6L

  6. L427–463, Appendix F: Add signal-derived feature probes, controlled MAE/JEPA pretraining comparisons, and all-block LoRA adaptation experiments. The adaptation results demonstrate that the intermediate-layer advantage observed under frozen extraction need not persist after parameter-efficient adaptation. @7dth

We have also revised Figures 1–4 and reorganized the supplementary material to make the experimental evidence and its limitations easier to follow.

Detailed responses to individual concerns are provided in our earlier rebuttal comments, with the corresponding evidence now incorporated into the manuscript.

We hope these revisions address your concerns and provide a clearer basis for evaluating our contributions. We appreciate your time and consideration and welcome any further discussion.

Best regards,
The Authors

AuthorsOpenReview ↗

Follow-up on Revised Manuscript and Author Responses

Dear Reviewers,

Thank you again for your constructive feedback. We have uploaded our revised manuscript and provided detailed responses, including additional experiments and clarifications addressing the concerns raised during the discussion.

As the discussion period is approaching its deadline (about 24 hours remaining), we would greatly appreciate it if you could take another look at our responses and revised manuscript, and let us know whether any concerns remain. We would be happy to provide further clarification while the discussion is still open.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.