acceptodds
Desk rejected at ICLR 2027

TopoJEPA: Topology-Grounded Joint-Embedding Prediction for EEG Foundation Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

15 views

Abstract

Self-supervised EEG foundation models based on masked autoencoding have shown strong potential for learning transferable representations from large-scale unlabeled data. However, reconstruction objectives inherently emphasize electrode-level signal details, which may not constitute the most transferable structure for downstream EEG decoding. This raises a fundamental question: what should an EEG model predict to capture the intrinsic organization of neural dynamics? We introduce TopoJEPA, a Topology-Grounded Joint-Embedding Predictive Architecture built on the principle of Grounding Before Prediction. Instead of directly predicting raw signals or unconstrained latent representations, TopoJEPA first constructs a target space aligned with EEG's intrinsic spatiotemporal organization and then learns predictive dynamics within this space. Specifically, Topological Target Grounding (TTG) maps heterogeneous electrode configurations onto a unified hierarchical topology and learns hemisphere- and region-level representations through reconstruction. To prevent reconstruction-specific signal details from dominating the learned target space, a Reconstruction-Specific Branch isolates such residual information in an auxiliary pathway while retaining reconstruction fidelity. Building on the grounded representations established by TTG, Topology-Grounded Autoregressive Prediction (TGAP) then models their spatiotemporal evolution in joint-embedding space. Across ten datasets covering six representative EEG decoding tasks, TopoJEPA achieves competitive performance against existing self-supervised and EEG foundation-model baselines, with particularly strong linear-probing performance across multiple tasks. These results demonstrate the effectiveness of constructing a stable, topology-grounded target space before predictive learning, enabling transferable EEG representations that capture meaningful spatiotemporal structure while reducing reliance on reconstruction-specific signal details. Code is available at https://anonymous.4open.science/r/TopoJEPA-F8B3.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG foundation models, self-supervised learning, joint-embedding predictive architecture, representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YyBROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TopoJEPA, a two-stage self-supervised framework for learning transferable EEG representations through topology-grounded latent prediction. In the first stage, Topological Target Grounding (TTG) maps heterogeneous EEG electrode configurations into a fixed set of hemisphere- and region-level tokens and learns their representations through waveform and spectral reconstruction. A separate reconstruction-specific branch is introduced to accommodate information that may be useful for signal recovery but less relevant to transferable representations. In the second stage, Topology-Grounded Autoregressive Prediction (TGAP) freezes the grounded target pathway and trains a context encoder to predict future topology representations under structured spatiotemporal attention and masking constraints. The method is evaluated on ten datasets covering six EEG decoding task families, using both full fine-tuning and linear probing. The reported results demonstrate competitive downstream performance and particularly strong linear-probing results. Additional experiments investigate target-space grounding, the reconstruction-specific branch, and spatiotemporal attention constraints.

Critical strengths
  1. A meaningful attempt to integrate structured spatial representations with latent predictive learning for EEG. The combination of topology-constrained tokenization, hierarchical hemisphere/region representations, and autoregressive latent prediction provides a coherent framework for handling heterogeneous EEG montages. The reconstruction-specific branch is also a reasonable architectural choice for reducing the influence of low-level reconstruction requirements on the prediction targets. These components address relevant challenges in EEG foundation model development.
  2. Broad empirical evaluation and informative component analyses. The evaluation covers ten datasets from six task families, including emotion recognition, motor imagery, cognitive workload, task difficulty, vigilance, and clinical EEG classification. The strong linear-probing performance on several benchmarks is encouraging. The analyses of grounding configurations, auxiliary representations, attention constraints, and masking strategies provide useful evidence that these design choices influence downstream representation quality.
Critical weaknesses
  1. The main methodological claim is not sufficiently isolated from training-budget and architectural confounders.

The central idea of establishing a fixed reconstruction-trained target encoder before predictive learning is closely related to SALT, which the authors acknowledge. Therefore, the primary methodological novelty should be demonstrated through the EEG-specific topology grounding and reconstruction-specific representation design, rather than the two-stage training principle alone.

More importantly, the experiments do not conclusively isolate the benefit of fixed target-space grounding from differences in training procedures and computational budgets. In Appendix E.1, the EMA-JEPA reference is trained for 80 epochs, whereas TopoJEPA involves 80 epochs of TTG followed by another 80 epochs of TGAP. The comparison also differs in target construction and training configurations. Consequently, the improvement over EMA-JEPA cannot be attributed specifically to freezing a topology-grounded target space.

A more decisive evaluation would compare fixed and EMA-updated targets using the same grounded initialization, architecture, objective, training data, and matched computational budgets. Without these controls, the principal mechanism underlying the reported improvement remains insufficiently established.

  1. The necessity and specific contribution of the proposed topology structure remain incompletely demonstrated.

The paper emphasizes hierarchical topology as a central factor in learning transferable EEG representations. However, some ablation results provide mixed evidence for this claim. In Figure 3 and Table 15, the full-channel encoding variant achieves slightly higher linear-probing Balanced Accuracy than TopoJEPA (37.63% versus 37.29%). In Table 2, removing the hemisphere-to-region attention constraint also slightly improves linear-probing Balanced Accuracy (37.68% versus 37.29%). Although the proposed model performs better under full fine-tuning in these comparisons and offers computational advantages over electrode-level tokenization, the results do not establish that the chosen anatomical hierarchy is intrinsically more effective for learning transferable representations.

The evaluation would also benefit from a stronger distinction between topology-specific inductive bias and the general benefits of token compression, masking, and representation capacity. Comparisons against alternative or randomly constructed spatial groupings under matched token counts and model capacity are needed. In addition, the claimed generality across heterogeneous montages would be more convincing with explicit cross-montage transfer and robustness experiments, rather than downstream performance comparisons alone.

Questions for authors

Q1. Can the authors provide a computationally matched comparison between fixed-target and EMA-target JEPA? Specifically, how does TopoJEPA compare against an EMA-JEPA trained with an equivalent total computational budget? More importantly, what happens when both approaches start from the same TTG-pretrained target encoder and differ only in whether the target encoder remains frozen or is updated by EMA? This would directly test the central claim of Grounding Before Prediction.

Q2. How much of the improvement is attributable to anatomical topology rather than token compression or hierarchical attention? Could the authors compare the proposed nine topology tokens with random electrode groupings, alternative region partitions, or geometry-based learned groupings under matched representation dimensions and computational budgets? The higher linear-probing scores of certain topology-relaxed variants make this an important question for understanding the claimed mechanism.

Q3. Could the authors clarify the implementation of the spatiotemporal attention mask in Equation (5)? According to Equation (5), hemisphere tokens at the first time step appear to have no valid attention keys, since neither the historical cross-role condition nor the within-step hierarchical condition can be satisfied. This seems to produce fully masked attention rows, which may lead to numerical instability under a standard masked-softmax implementation. Could the authors clarify how these cases are handled in practice, for example, whether self-attention entries are explicitly preserved or a special treatment is applied to fully masked rows? It would also be helpful to clarify whether the implemented attention mask exactly follows Equation (5).

7 comments

We thank the reviewer for the careful evaluation and constructive comments. To address your concerns, we (1) clarify the methodological contributions of Grounding Before Prediction, (2) provide controlled EMA comparisons with matched initialization, architecture, optimization, and computational budgets, (3) introduce equal-token topology controls and additional cross-task evaluations to further examine the contribution of anatomical grouping, and (4) clarify the implementation of the spatiotemporal attention mask. We provide detailed responses and supporting evidence below. We address your concerns below, and would appreciate it if you would be willing to raise your score if you find your concerns addressed.

W1 & Q1. Methodological Contribution and Controlled Evaluation of Grounding

We thank the reviewer for the careful assessment of our methodological contribution and experimental design. We would like to emphasize that the central methodological question of TopoJEPA is how to construct and exploit effective prediction targets for EEG foundation models. Given the low signal-to-noise ratio, non-stationarity, and complex spatiotemporal dependencies of EEG, we propose Grounding Before Prediction to establish a structured and stable latent target space before predictive learning. We further develop an interconnected two-stage framework, where topology-aware grounding and reconstruction-specific auxiliary routing shape the target representations, while spatiotemporal attention constraints guide subsequent predictive learning. This design provides a unified approach to learning transferable EEG representations.

Regarding the EMA comparison, our original experiment was intended to compare two distinct predictive learning paradigms rather than serve as a strict ablation of target freezing. In preliminary experiments, combining EMA-updated targets with our spatiotemporal constraints frequently led to training instability. We therefore adopted established I-JEPA/V-JEPA optimization practices for the EMA baseline rather than directly imposing TopoJEPA's complete configuration.

To directly address the reviewer's concerns about initialization, architecture, optimization, and computational budget, we introduce four controlled configurations to distinguish TTG initialization, target updating, and spatiotemporal predictive learning, as summarized in Table R1.

Table R1. Configuration differences and estimated total pretraining compute.

SettingScratch EMA-JEPATTG-initialized EMA-JEPATopoJEPA (vanilla)TopoJEPA
Target InitializationRandomTTGTTGTTG
Target UpdateEMAEMAFrozenFrozen
PredictionSame-timeSame-timeSame-timeNext-step
Spatiotemporal Constraints✗✗✗✓
Encoder Size (Context / Target)384×12 / 384×12384×12 / 384×12384×12 / 384×12384×10 / 256×8
Epochs (TTG + Prediction)0 + 8080 + 8080 + 8080 + 80
Total Compute (×10¹⁸ FLOPs)1.7163.1083.1081.796

All configurations share the same pretraining corpus, nine topology tokens, time-consistent spatial masking, two-layer predictor (width 192), Smooth L1 loss, and AdamW settings (learning rate 5×10⁻⁴, weight decay 10⁻⁴, global batch size 1,024). Importantly, TTG-EMA and TopoJEPA (Vanilla) share the same TTG checkpoint, encoder architectures (384×12), prediction objective, optimization settings, and total compute (3.108 × 10¹⁸ FLOPs), differing only in target updating. Standard EMA requires matching context and target architectures for parameter updates, whereas Grounding Before Prediction allows greater architectural flexibility. As shown in Section 3.4 and Fig. 3(b), a medium-sized target encoder (256×8) also achieves comparable performance.

Table R2. SEED-V results (balanced accuracy, mean ± standard deviation).

ConfigurationLinear ProbingFull Fine-tuning
Scratch-EMA0.3059 ± 0.00690.3805 ± 0.0056
TTG-EMA0.2961 ± 0.00350.3703 ± 0.0050
TopoJEPA (Vanilla)0.3446 ± 0.00920.4001 ± 0.0072
TopoJEPA0.3729 ± 0.00350.4157 ± 0.0108

We would like to highlight two points.

First, regarding the role of grounding and target freezing, TTG initialization alone does not improve EMA-JEPA over its scratch counterpart. This suggests that the structural prior established during grounding may not be effectively preserved as EMA targets continue to evolve. More importantly, starting from the same TTG initialization, with matched architecture, optimization, and computational budget, TopoJEPA (Vanilla) outperforms TTG-EMA by 4.85 percentage points in linear probing and 2.98 points in fine-tuning. This controlled comparison directly supports the importance of maintaining a fixed grounded target space.

Second, regarding the broader value of grounding, the complete TopoJEPA further improves upon its vanilla version while requiring less computational cost. A fixed grounded target space provides a stable and extensible foundation for incorporating spatiotemporal attention constraints, which we found difficult to combine with EMA-based learning in preliminary experiments. Together with the flexibility of encoder architecture selection (Section 3.4) and the Reconstruction-Specific Auxiliary Branch for separating reconstruction-related information from topology targets (Section 3.5), these advantages represent important motivations for Grounding Before Prediction beyond target freezing alone.

We have incorporated these additional comparisons and computational details into Section 3.4 and Appendix E.1 of the revised manuscript, with the corresponding changes highlighted in blue.

Reviewer_cvTeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TopoJEPA learns EEG representations in two stages. TTG constructs hemisphere- and region-level representations through waveform and spectral reconstruction, with an auxiliary pathway that bypasses the target encoder. TGAP then freezes the target pathway and trains a context encoder to predict representations at the next time step, using spatial masking and attention constraints. The context encoder is evaluated on several EEG decoding tasks. The main empirical result is strong linear probing: a simple head can use the pretrained features without adapting the encoder. Full fine-tuning is also competitive, although the gains vary across tasks.

Critical strengths
  1. The linear-probing results are the strongest part of the paper. The gains on SEED-V stand out, and the improvements on DEAP, BAT, and SFT show that the result extends beyond a single task family (Tables 1 and 11). Useful information is available to a simple linear head without adapting the encoder.
  2. The auxiliary-branch ablations are also informative. The auxiliary tokens help reconstruction, but adding them to the prediction targets hurts downstream decoding (Table 15; Figures 3 and 5). Removing the branch also hurts. These comparisons support keeping the reconstruction pathway separate and show why reconstruction quality alone is a poor guide to choosing prediction targets.
Critical weaknesses
  1. My main concern is the comparison with EMA-JEPA. TopoJEPA receives an additional reconstruction-pretraining stage, and the weight decay settings also differ (Appendix E.1; Tables 4–5). I cannot tell how much of the reported gain comes from fixing the target space and how much comes from these other choices. A comparison with aligned settings and a comparable total compute budget, including target-encoder preparation, is needed to support the Grounding Before Prediction claim.
  2. The topology ablation changes both the electrode grouping and the number of tokens. The electrode-level variant is comparable in linear probing and weaker in full fine-tuning, but uses a much longer sequence (Table 15). This supports the compact representation, while leaving the value of the prescribed grouping unclear. A control with the same number of learned tokens and no prescribed groups would be more informative. The slightly better linear-probing result without hemisphere-to-region ordering adds to this concern (Table 2), although the authors acknowledge it. Since these ablations focus on SEED-V, a check on a different task family would help establish how broadly the finding holds.
Questions for authors
  1. Does the advantage over EMA-JEPA persist with a comparable total compute budget and aligned weight decay and masking? Please also specify the EMA variant's online and target encoder architectures. This result would determine how strongly the comparison supports fixed targets.
  2. What happens if the nine-token count, dimensions, and training budget are kept fixed while the prescribed electrode grouping is removed? Is such a control available on SEED-V and at least one other task family? It would isolate the contribution of topology at the same degree of compression.
  3. How is temporal causality enforced in the Latent Predictor in Equation (7): does it process each time step independently, or use its own causal mask? Equation (5) defines the context encoder's mask, while the predictor receives a sequence of context outputs. Please clarify whether later context steps are inaccessible when computing each prediction. This is a request for architectural clarification; I am not asserting that future context is used.
3 comments

We thank the reviewer for the careful and constructive evaluation. To address your concerns, we (i) provide controlled EMA comparisons with matched initialization, architecture, optimization, and computational budgets, (ii) introduce equal-token controls to further examine the independent contribution of anatomical topology, and (iii) clarify how temporal causality is enforced in the Latent Predictor. We address each point with additional evidence and detailed clarifications below.We address your concerns below, and would appreciate it if you would be willing to raise your score if you find your concerns addressed.

W1 & Q1. Clarification and Controlled Comparison with EMA-JEPA

Our original EMA-JEPA comparison was intended to evaluate distinct predictive learning paradigms rather than isolate the effect of target freezing. In preliminary experiments, incorporating TopoJEPA's spatiotemporal constraints into EMA-JEPA led to unstable training or collapse in some configurations. We therefore followed established optimization practices from I-JEPA and V-JEPA for the original EMA baseline.

To directly address the reviewer's concerns about initialization, masking, optimization, and total compute, we introduce four configurations to distinguish the effects of TTG initialization, target freezing, and spatiotemporal predictive learning, as summarized in Table R1.

Table R1. Configuration differences and estimated total pretraining compute.

SettingScratch EMA-JEPATTG-initialized EMA-JEPATopoJEPA (vanilla)TopoJEPA
Target InitializationRandomTTGTTGTTG
Target UpdateEMAEMAFrozenFrozen
PredictionSame-timeSame-timeSame-timeNext-step
Spatiotemporal Constraints✗✗✗✓
Encoder Size (Context / Target)384×12 / 384×12384×12 / 384×12384×12 / 384×12384×10 / 256×8
Epochs (TTG + Prediction)0 + 8080 + 8080 + 8080 + 80
Total Compute (×10¹⁸ FLOPs)1.7163.1083.1081.796

All configurations share the same pretraining corpus, nine topology tokens, time-consistent spatial masking, a two-layer predictor (width 192), Smooth L1 loss, and AdamW settings (learning rate 5×10⁻⁴, weight decay 10⁻⁴, global batch size 1,024). Compute estimates include TTG pretraining where applicable. Crucially, TTG-initialized EMA-JEPA and TopoJEPA (vanilla) use the same TTG initialization, encoder architectures, optimization settings, and computational budget, differing only in target updating.Table R2 reports their performance on SEED-V under linear probing and full fine-tuning.

Table R2. SEED-V results (balanced accuracy, mean ± standard deviation).

ConfigurationLinear ProbingFull Fine-tuning
Scratch EMA-JEPA0.3059 ± 0.00690.3805 ± 0.0056
TTG-initialized EMA-JEPA0.2961 ± 0.00350.3703 ± 0.0050
TopoJEPA (vanilla)0.3446 ± 0.00920.4001 ± 0.0072
TopoJEPA0.3729 ± 0.00350.4157 ± 0.0108

First, regarding grounding and target freezing, TTG initialization alone does not improve EMA-JEPA over its scratch counterpart, suggesting that the learned structural prior may not be effectively preserved as EMA targets evolve. More importantly, under strictly matched conditions, TopoJEPA (vanilla) outperforms TTG-initialized EMA-JEPA by 4.85 percentage points in linear probing and 2.98 points in fine-tuning. This controlled comparison directly supports the importance of maintaining a fixed grounded target space.

Second, regarding the broader value of grounding, the complete TopoJEPA further improves upon its vanilla version at lower computational cost. A fixed grounded target space provides a stable and extensible foundation for predictive learning, supporting temporal-causal and spatial-hierarchical constraints that caused instability in some of our preliminary EMA-based experiments. Together with flexible context and target encoder architectures (Section 3.4) and the Reconstruction-Specific Auxiliary Branch for separating reconstruction-specific information (Section 3.5), this extensibility represents an important motivation for Grounding Before Prediction beyond target freezing alone.

We have incorporated these additional comparisons and computational details into Section 3.4 and Appendix E.1 of the revised manuscript, with the corresponding changes highlighted in blue.

W2 & Q2. Contribution of Topology Grouping Beyond Token Compression

We thank the reviewer for raising this important question. We would first like to clarify that our original Full-channel ablation was designed to evaluate the overall benefits of the compact topology interface, including representation effectiveness, computational efficiency, and adaptability to different electrode configurations. As discussed in Section 3.5, replacing electrode-level tokens with nine topology representations substantially reduces computational complexity while providing a consistent representation interface across electrode configurations. The results demonstrate improved fine-tuning performance without requiring electrode-level representations. To further isolate the contribution of anatomical grouping from token compression, we conduct an additional equal-token comparison.

Specifically, we introduce two additional controls: Global Queries, where each query can attend to all electrode embeddings without predefined anatomical grouping, and Random Groups, which use fixed random electrode groupings. All configurations share the same nine-token interface, encoder capacity (width 384, depth 10), 3,456-dimensional downstream representation, and training settings. We evaluate them on SEED-V and DEAP-V under both linear probing and full fine-tuning.

Table R3. Equal-token topology controls (balanced accuracy, mean ± standard deviation).

DatasetEvaluationGlobal QueriesRandom GroupsTopoJEPA
SEED-VFull Fine-tuning0.3910 ± 0.00150.3842 ± 0.00980.4157 ± 0.0108
SEED-VLinear Probing0.3312 ± 0.00830.3471 ± 0.00440.3729 ± 0.0035
DEAP-VFull Fine-tuning0.5640 ± 0.00490.5936 ± 0.00380.5941 ± 0.0061
DEAP-VLinear Probing0.5469 ± 0.00170.5659 ± 0.00140.5767 ± 0.0028

The results demonstrate that anatomical topology grouping contributes beyond token compression. With identical token counts and encoder capacity, TopoJEPA achieves the highest mean linear-probing performance on both datasets and the highest mean full fine-tuning performance. The advantage is particularly consistent under linear probing, where anatomically organized representations outperform both global queries and random groupings.

These results directly address the reviewer's concern: the benefit of TopoJEPA cannot be attributed solely to reducing the number of tokens, as prescribed anatomical grouping provides additional representational value at the same degree of compression.

We have incorporated these matched-token controls and the corresponding analysis into Appendix E.2 (Table 17) of the revised manuscript, with changes highlighted in blue. For complementary cross-dataset evaluations across different electrode configurations, please also refer to our response to Reviewer 1 (W4 & Q3).

Q3. Temporal Causality in the Latent Predictor

We thank the reviewer for this architectural question. The Latent Predictor does not process each time step independently; instead, it performs next-step prediction with a one-step temporal shift and its own explicit attention masks. Specifically, in an eight-step window, context from steps 1–7 is used to predict target representations at steps 2–8. Thus, step 1 serves only as the initial context and is not a prediction target.

The predictor consists of two Transformer decoder blocks, each comprising masked self-attention, masked cross-attention, and a feed-forward network. Prediction queries are initialized with target-time and topology-role embeddings, together with learnable base embeddings. Both attention operations follow the prescribed temporal-causal and spatial-hierarchical rules, preventing access to future context.

For example:

  • Predicting a hemisphere representation at step 8: The query can access context representations of other topology roles from steps 1–7, but not future context.
  • Predicting a region representation at step 8: The query can additionally access the two hemisphere representations at step 8, but not region-level context at that step or any future information. The same-step hemisphere representations come from the online context encoder, not the frozen target encoder.

Therefore, the temporal shift prevents self-prediction, while the predictor's attention masks enforce the intended temporal-causal and spatial-hierarchical information flow. We have clarified this mechanism in Section 2.3 of the revised manuscript, with the corresponding changes highlighted in blue.

Reviewer_LoBtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TopoJEPA learns a compact EEG target space of hemisphere and region tokens through waveform and spectral reconstruction, with an auxiliary branch that bypasses the target encoder. It then freezes the target pathway and trains a context encoder to predict future latent representations under structured spatial masking and temporal attention constraints. Evaluation covers linear probing and full fine-tuning across ten datasets.

Critical strengths
  1. The frozen-encoder results are the strongest part of the paper for me. In Table 1, SEED-V balanced accuracy is 37.29%, versus 25.63% for the strongest listed pretrained baseline. The EEGMat results in Table 10 also support transfer under a subject-independent split, so the practical value extends beyond within-subject emotion recognition.

  2. The ablations in Tables 2, 15 and 16 give useful support for the complete training design. Removing the auxiliary branch or temporal constraint weakens linear probing on SEED-V, and keeping the spatial mask consistent over time helps. I also appreciate the reporting of run-to-run variability and competitive results from less constrained variants.

Critical weaknesses
  1. I see limited methodological novelty. Section A.3 acknowledges SALT’s reconstruction-then-freezing recipe, while Section A.2 discusses hierarchical EEG prediction in THD-BAR and latent aggregation in LUNA. Continuous topology targets and auxiliary routing are meaningful differences, but I am not yet convinced that their incremental contribution is established strongly enough.
  2. The comparison in Section E.1 does not convince me that target freezing explains the gain. EMA-JEPA omits the grounding stage and uses different regularization from the configurations in Tables 4 and 5. Without matched initialization, optimization and total compute, I read this as a comparison of two training recipes.
  3. I find the auxiliary branch’s practical benefit in Table 15 clearer than the information-separation explanation. The diagnostics in Section E.2 measure signal recovery, not task information in each pathway. Section 3.5 appropriately treats the high-frequency explanation as a hypothesis; matched task probes would help test the broader interpretation of these gains.
  4. The topology advantage looks conditional on adaptation. The full-channel variant in Table 15 and relaxed hemisphere-to-region variant in Table 2 have slightly higher linear-probe means but lower fine-tuning means on SEED-V. The authors acknowledge this tradeoff. I see value in the compact interface, but less evidence that this pattern generalizes across tasks.
Questions for authors
  1. From the same grounded initialization under matched optimization and a comparable total budget, how does a frozen target compare with an EMA-updated target?

  2. Using existing checkpoints and matched linear probes, do auxiliary-only features retain less downstream task information than the topology representations?

  3. On one existing subject-independent task, do the full-channel and relaxed-hierarchy ablation checkpoints show the same linear-probing versus fine-tuning tradeoff?

6 comments

We thank the reviewer for the careful evaluation and constructive comments. To address your concerns, we (1) clarify the methodological contributions of Grounding Before Prediction, (2) provide controlled EMA comparisons with matched initialization, optimization, and computational budgets, (3) add task-relevant information probes to further validate the auxiliary branch, and (4) extend topology ablations across six datasets to examine their cross-task effectiveness. We address each point with detailed explanations and supporting evidence below. We address your concerns below, and would appreciate it if you would be willing to raise your score if you find your concerns addressed.

W1. Methodological Novelty and EEG-Specific Target Grounding

We thank the reviewer for the careful assessment of our methodological novelty. We believe that the contributions of TopoJEPA should be considered in the context of the fundamental research questions it addresses, rather than solely in terms of precedents for individual technical components: Can JEPA serve as an effective pretraining paradigm for EEG foundation models? If so, how should its prediction targets be constructed to accommodate the unique characteristics of EEG signals? Our work systematically investigates these questions and introduces Grounding Before Prediction as a central design principle for adapting JEPA to EEG foundation modeling.

Specifically, we summarize our contributions in three aspects.

1. Systematic investigation of JEPA's applicability to EEG foundation models.

Although JEPA has demonstrated success in other modalities, its applicability to EEG foundation modeling remains insufficiently validated at scale. To the best of our knowledge, prior JEPA-based EEG studies have not provided a comparably comprehensive evaluation in terms of both pretraining scale and downstream task coverage.

To address this gap, we pretrain on 3,179.3 hours of EEG data and conduct extensive evaluations across ten downstream datasets spanning six task families. Our objective is to systematically investigate whether latent predictive learning can produce general-purpose EEG representations with broad transferability, thereby exploring an alternative to conventional reconstruction-based pretraining for EEG foundation models.

2. Identifying and validating the importance of Grounding Before Prediction for EEG latent predictive learning.

We argue that JEPA requires careful adaptation to the characteristics of EEG signals. Unlike visual data, EEG exhibits low signal-to-noise ratios, pronounced non-stationarity, and complex spatiotemporal dependencies. Although EEG contains rich underlying neural information, extracting stable and task-relevant representations is challenging due to noise and task-irrelevant variability. We hypothesize that learning stable predictive relationships directly against continuously evolving latent targets, as commonly adopted in EMA-based JEPA, may further complicate this process.

Motivated by this observation, we introduce Grounding Before Prediction: a dedicated grounding stage first establishes a structured and stable EEG latent space, which is subsequently frozen to provide consistent prediction targets. This allows the predictive network to learn hierarchical spatial relationships and temporal predictive dependencies against a fixed representational reference, without simultaneously tracking continuously changing targets.

The value of this design is particularly evident in the encouraging linear-probing results. While several existing state-of-the-art EEG foundation models exhibit substantial performance degradation when their pretrained encoders are frozen, TopoJEPA maintains competitive performance across diverse downstream tasks. This suggests that its pretrained latent representations retain strong task-relevant information that can be directly exploited by downstream models. We therefore believe that Grounding Before Prediction provides a direction worth further exploring in EEG foundation model research.

3. Developing two interconnected stages of EEG-specific learning mechanisms guided by this principle.

In the first stage, we integrate topology-aware constraints and a Reconstruction-Specific Auxiliary Branch to establish a continuous latent target space that accommodates both the hierarchical spatial organization of EEG and waveform reconstruction requirements. Specifically, the topology-aware constraints impose explicit hemisphere- and region-level organization on latent representations, while the auxiliary branch captures additional reconstruction-related information, reducing the dependence of target representation learning on detailed waveform recovery.

In the second stage, we introduce spatiotemporal attention constraints and topology-aware context masking to regulate information flow across topology roles and time, enabling the context encoder to learn hierarchical spatial relationships and temporal predictive dependencies within the fixed topology-grounded target space. This facilitates the exploration of richer spatiotemporal structure in EEG beyond local signal reconstruction.

To further clarify the methodological contributions highlighted by the reviewer, we present targeted experimental evidence demonstrating the distinct roles of the Reconstruction-Specific Auxiliary Branch (W3 & Q2) and continuous topology-grounded representations (W4 & Q3), respectively.

W2 & Q1. Clarification and Controlled Comparison with EMA-JEPA

We thank the reviewer for the opportunity to clarify our EMA-JEPA comparison. Our original comparison was intended to evaluate two distinct predictive learning paradigms rather than serve as a strict ablation of target freezing. In preliminary experiments, we attempted to incorporate hierarchical spatial attention constraints and temporal causal prediction into EMA-JEPA, but training frequently became unstable or collapsed. We hypothesize that continuously evolving targets introduce additional optimization difficulties when learning complex spatiotemporal dependencies. In contrast, Grounding Before Prediction provides a stable target space for extending predictive learning with additional spatiotemporal constraints. This motivated us to follow established optimization practices from two representative JEPA works, I-JEPA [1] and V-JEPA [2], rather than impose identical optimization settings across distinct predictive learning paradigms.

To directly address the reviewer's concern, we construct four configurations to disentangle the contributions of TTG initialization, target freezing, and spatiotemporal predictive learning, as summarized in Table R1. Scratch EMA-JEPA serves as the conventional EMA baseline; TTG-initialized EMA-JEPA examines the effect of grounding initialization; TopoJEPA (vanilla) isolates the effect of target freezing under otherwise identical settings; and TopoJEPA (ours) evaluates the complete framework with spatiotemporal constraints. Notably, TTG-initialized EMA-JEPA and TopoJEPA (vanilla) differ only in the target update strategy, enabling a controlled comparison under matched initialization, architecture, and computational budget.

Table R1. Configuration differences and estimated total pretraining compute.

SettingScratch EMA-JEPATTG-initialized EMA-JEPATopoJEPA (vanilla)TopoJEPA
Target InitializationRandomTTGTTGTTG
Target UpdateEMAEMAFrozenFrozen
PredictionSame-timeSame-timeSame-timeNext-step
Spatiotemporal Constraints✗✗✗✓
Encoder Size (Context / Target)384×12 / 384×12384×12 / 384×12384×12 / 384×12384×10 / 256×8
Epochs (TTG + Prediction)0 + 8080 + 8080 + 8080 + 80
Total Compute (×10¹⁸ FLOPs)1.7163.1083.1081.796

All configurations share the same pretraining data, nine topology tokens, masking strategy, two-layer predictor (width 192), Smooth L1 loss, and AdamW settings (learning rate 5×10⁻⁴, weight decay 10⁻⁴, batch size 1,024). Compute estimates include TTG pretraining where applicable. Unlike EMA, which requires matching encoder architectures for weight updates, Grounding Before Prediction allows greater architectural flexibility, with a medium-sized target encoder (256×8) achieving comparable performance (Section 3.4 and Fig. 3(b)). Table R2 reports the SEED-V results under linear probing and full fine-tuning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.