acceptodds
Under review as a conference paper at ICLR 2027

Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

Abstract

Multimodal brain state decoding has largely focused on fusing paired modalities for prediction, but has rarely explored how their correspondence can be further exploited to enrich training data and improve multimodal representation learning. To address this gap, we propose CoMA-DiT, a bidirectional cross-modal Diffusion Transformer for latent augmentation that treats paired modalities as sources of mutual generative supervision rather than merely as inputs to be fused. CoMA-DiT conditions velocity prediction on the paired modality through cross-modal attention and adaptively injects the resulting variation via a reliability-gated residual mechanism. Experiments on multimodal auditory attention decoding and emotion recognition showed that CoMA-DiT consistently outperformed 20 representative baselines, achieving absolute gains of 4.28% and 6.70% in accuracy and macro-F1 over the no-augmentation baseline, respectively. Extensive ablation, sensitivity, visualization, and interpretability analyses further demonstrated its robustness, generalizability, and ability to capture functionally relevant cross-modal interactions. These findings support a broader view of multimodal learning: Paired modalities can serve not only as inputs for fusion but also as supervision sources that augment one another.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Multimodal Learning, Generative Models, Auditory Attention Decoding, Emotion Recognition, Diffusion Transformer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UU12OpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoMA-DiT augments EEG and EOG representations through bidirectional cross-modal attention and gated residual perturbations, trained with a velocity-prediction auxiliary objective. A second stage adapts the classifier using real and augmented latents. Evaluation on AVGC and DEAP reports improvements in auditory attention decoding and emotion recognition under within-subject chronological splits.

Critical strengths
  1. The approach improves two task-specific backbones across auditory attention and emotion recognition. The augmentation module is unnecessary at inference, making the reported gains practically relevant.
  2. Splitting recordings before windowing addresses an important leakage risk. The loss and architectural ablations in Tables D1-D2 provide useful evidence about the gate, bidirectionality, and auxiliary objectives.
Critical weaknesses
  1. The role of diffusion is not established clearly. Equations (4)-(5) produce augmentation through a separate residual head on noisy Transformer features, rather than a stated reverse-flow sampling procedure. The velocity loss therefore appears to act as auxiliary regularization. Removing it still gives 79.65% AVGC accuracy versus 80.52% for the full model and 76.24% for DARNet. This leaves much of the gain explainable by ordinary learned residual augmentation. Neither the derivation nor the experiments establish a distinct generative mechanism.
  2. The strength and scope of the empirical evidence need qualification. Appendix D.4 treats 36 subject--seed pairs as observations, although runs from the same 12 subjects are clustered. This can overstate significance across subjects. All evaluations are within-subject and use EEG-EOG, so they do not establish generalization to unseen subjects or other modality combinations.
Questions for authors
  1. Is any reverse diffusion or flow integration used to produce augmented latents? If not, how does the method compare with the same noisy cross-attention residual network trained without velocity prediction, under a matched training budget?
  2. Do the gains remain significant after averaging seeds within each subject or using a subject-clustered analysis? Please also clarify which reported evidence supports the broader generalizability claim.
17 comments
AuthorsOpenReview ↗

W1 — Clarifying velocity supervision and explicit integration (1/3)

Thank you for your careful review and for the opportunity to clarify our augmentation mechanism. We agree that learning a conditional velocity field and integrating that field for augmentation are distinct operations, and that a velocity-prediction loss alone should not be presented as evidence of a diffusion sampler. We have clarified that CoMA-DiT (Single-Pass) uses velocity supervision to guide cross-modal latent augmentation, and have additionally implemented CoMA-DiT (Reverse) to examine explicit conditional velocity integration.

Single-Pass: velocity-supervised latent augmentation. For target modality aa conditioned on its paired source bb, we construct a linear data-to-noise path and its velocity target:

\qquad \mathbf V^a=\boldsymbol\epsilon^a-\mathbf Z^a.$$ A conditional Transformer processes $(\mathbf Z^a_\tau,\mathbf Z^b,\tau)$ and produces hidden target tokens $\mathbf H$ and a velocity prediction. Two distinct heads use the shared representation: $$\hat{\mathbf V}=\phi_v(\mathbf H),\qquad \Delta_{\mathrm{SP}}=s\\,g\\,\tanh\\!\left(\phi_\Delta(\mathbf H)\right),\qquad \mathbf Z^{a+}=\mathbf Z^a+\Delta_{\mathrm{SP}}.$$ The squared-error velocity objective supervises the **shared Transformer representation and trainable encoders**, while a separate residual head produces the bounded augmentation. The two heads use the same learned tokens but serve different purposes: the velocity loss shapes the conditional representation during training, and classification, cross-modal consistency, and preservation objectives guide the residual for downstream decoding. In Single-Pass, the predicted velocity is **not numerically integrated**. We now describe this path as *velocity-supervised cross-modal latent augmentation*, distinguishing its single-pass generation from Reverse's integration-based construction. These clarifications make the two mechanisms more explicit: Single-Pass combines cross-modally conditioned residual augmentation with velocity-supervised representation learning, while Reverse examines how the learned conditional velocity field can also be used through numerical integration. **Revision.** Section 3.3 now makes the separate velocity/residual data flows explicit; Section 3.4 describes the loss as conditional velocity supervision. Figure 3 differentiates Single-Pass and Reverse, and Appendix A.4 describes the newly added integration-based path.
AuthorsOpenReview ↗

W1 — Actual reverse-flow augmentation and task-dependent results (2/3)

Reverse: explicit conditional velocity integration. For the direction b→ab\rightarrow a, CoMA-DiT (Reverse) starts from a partially noised observed target, x(τs)=(1−τs)Za+τsϵa\mathbf x(\tau_s)=(1-\tau_s)\mathbf Z^a+\tau_s\boldsymbol\epsilon^a, while keeping the paired source Zb\mathbf Z^b fixed. It integrates the learned source-conditioned velocity field from τs\tau_s to 00 using Heun predictor–corrector updates. Writing Zˉ0\bar{\mathbf Z}_0 for the integrated endpoint, the augmentation is

\mathbf{Z}^{a+}=\mathbf{Z}^{a}+\Delta_{\mathrm{Rev}}.$$ Thus, **Reverse constructs its augmentation from the numerically integrated conditional velocity field**, whereas Single-Pass uses a direct residual head. Because Reverse starts from a partially noised observed target, it performs conditional augmentation rather than unconditional synthesis from pure noise. Both directions of EEG–EOG augmentation are retained, and neither path is required at inference. We also clarify that Reverse uses an endpoint-derived residual and different reliability-gate inputs, so the comparison evaluates the two implemented augmentation paths. We report both paths and their unaugmented backbones on four settings. All entries are mean ± SD (%); the full four-metric results appear in Appendix A.5, Table A2. | Dataset | Backbone accuracy | Single-Pass accuracy | Reverse accuracy | Single-Pass macro-F1 | Reverse macro-F1 | | --- | ---: | ---: | ---: | ---: | ---: | | AVGC | 76.24 ± 4.11 | **80.52 ± 3.82** | 79.73 ± 3.40 | **80.39 ± 3.82** | 79.58 ± 3.48 | | DTU | 83.38 ± 1.63 | **87.31 ± 1.62** | 86.10 ± 1.44 | **86.72 ± 1.70** | 85.74 ± 1.51 | | DEAP-A | 94.92 ± 1.78 | 97.17 ± 1.18 | **97.98 ± 1.36** | 96.86 ± 1.33 | **97.40 ± 1.58** | | DEAP-V | 95.11 ± 1.63 | 97.47 ± 0.92 | **97.84 ± 1.56** | 97.37 ± 0.95 | **97.65 ± 1.64** | Both paths improve their corresponding backbones. Single-Pass has higher mean accuracy on AVGC/DTU by **0.79/1.21 points**, whereas Reverse has higher means on DEAP-A/DEAP-V by **0.81/0.37 points**. These task-dependent rankings suggest that explicit integration provides a useful alternative to the direct residual path, rather than a uniformly preferable replacement. **Revision.** Appendix A.4 gives the initialization, Heun updates, endpoint residual, and gate inputs. Appendix A.5 (Table A2) reports the four-setting comparison; Figure 3 and the main results tables identify the two paths separately.
AuthorsOpenReview ↗

W1 — What the loss ablations and matched controls establish (3/3)

The original AVGC loss ablation provides complementary evidence about the contribution of velocity supervision. Table D1 reports 79.65% accuracy without Ldiff\mathcal L_{\mathrm{diff}}, compared with 80.52% for the full model and 76.24% for unaugmented DARNet. Hence removing the velocity loss reduces accuracy by 0.87 percentage points, but the no-velocity residual model remains 3.41 points above DARNet. These results indicate that the residual augmentation mechanism and the remaining objectives also contribute substantially, with velocity supervision providing an additional improvement under the standard protocol.

To better isolate the velocity supervision, Table D3 additionally uses a fixed optimizer-step budget and common real-validation selection rule. Both rows below retain noisy targets, the same cross-attention residual network, other training objectives, Stage I classification of augmented features, and real-only Stage II adaptation:

Matched-step setting (AVGC)AccuracyMacro-F1PrecisionRecall
Velocity Off + Real-only78.63 ± 3.3978.44 ± 3.3879.26 ± 3.5778.63 ± 3.39
Velocity On + Real-only79.61 ± 2.9779.39 ± 2.9780.30 ± 3.1179.61 ± 2.97

Enabling velocity supervision improves accuracy, macro-F1, precision, and recall by 0.98, 0.95, 1.04, and 0.98 points, respectively. Under this controlled protocol, the improvements support a representation-learning benefit from the velocity objective without adding generated samples during Stage II.

To address whether learned augmentations provide value beyond extra architecture and training, Table D4 compares two further matched-step, same-architecture settings with velocity supervision retained: classifying only real features in both stages (A: 78.35% accuracy) versus classifying real and augmented features in Stage I, followed by real-only Stage II (B: 79.61%). The 1.26-point gain supports a contribution from supervised generated features beyond identical architecture, noising, and training steps. Setting A still computes generated features for auxiliary losses, and it is a control for classification use of augmentation, not a generator-free model. The complete configuration reaches 80.52% under the standard schedule.

SettingStage I classificationStage II adaptationAccuracy (%)Macro-F1 (%)Precision (%)Recall (%)
A: Real-only classificationReal onlyReal only78.35 ± 3.7378.15 ± 3.8179.06 ± 3.7078.35 ± 3.73
B: Augmentation in joint trainingReal + AugReal only79.61 ± 2.9779.39 ± 2.9780.30 ± 3.1179.61 ± 2.97
C: CoMA-DiTReal + AugReal + Aug80.52 ± 3.8280.39 ± 3.8281.00 ± 3.9080.53 ± 3.80

Overall, the evidence supports two complementary roles: velocity matching improves the representation used by Single-Pass residual augmentation, while Reverse uses the learned field for explicit integration. This distinction clarifies the contribution of velocity supervision alongside the broader cross-modal augmentation mechanism.

Revision. Appendix D.1 (Table D1) now explicitly interprets the 79.65% ablation; Appendix D.3 (Table D3) specifies the matched velocity comparison and its budget; Appendix D.4 (Table D4) clarifies the same-architecture generated-feature classification control.

Reviewer_XQuxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CoMA-DiT, a bidirectional cross-modal Diffusion Transformer framework for multimodal brain state decoding. It exploits paired EEG and EOG signals as mutual conditioning information to generate augmented latent representations through cross-modal attention and reliability-gated residual injection. Experiments under different datasets demonstrate improvements over several classification and data augmentation baselines.

Critical strengths

1.The paper addresses cross-modal correspondence for data augmentation in diffusion based manner, providing a reasonable approach to preserving modality-specific information. 2.The experiments cover two brain state decoding tasks and extensive baselines. The ablation studies, backbone comparisons, and statistical analyses provide useful empirical evidence.

Critical weaknesses

1.The methodological novelty is somewhat incremental. The framework mainly combines different modules. Its fundamental distinction from existing routing fusion or feature adaption approaches could be better established. In addition, removing cross-modal attention results in only a modest performance reduction. 2.The role of diffusion modeling in the actual augmentation process is unclear. Although the model is trained with a velocity-prediction objective, augmented representations are directly obtained through residual projections of Transformer hidden states rather than reverse diffusion or flow integration. This raises questions about the necessity of the diffusion formulation. 3.Comparisons with closely related latent-space augmentation methods are insufficient. Most generative baselines operate in the original signal space, whereas CoMA-DiT performs augmentation in the learned latent space. 4.The experimental evaluation focuses exclusively on intra-subject setting, while the effectiveness of CoMA-DiT under unseen-subject conditions remains unexplored. 5.Missing references about unifying multimodal augmentation and fusion based on diffusion models: [1]Adapt and explore: Multimodal mixup for representation learning. Ronghao Lin, Haifeng Hu. Information Fusino 2024. [2]CAAL: EEG-guided cross-modal adaptive alignment for learning consistent representations in EEG-fNIRS fusion. Baole Fu, Jinwu Gao, Shengyan Hou, Zhihuan Jia, Yinhua Liu, Keum-Shik Hong. Information Fusion 2026. [3]DiffAT: Effective data augmentation with diffusion models for time series forecasting. Yang Yu, Ruizhe Ma, Wenbo Gu , Zongmin Ma, EAAI, 2025. [4]R. Lin, Q. He, Y. Zeng, S. Mai, L. Huang and a. H. Hu, "Affection-guided Bottleneck Diffusion for Missing Modality Issue in Multimodal Affective Computing," in IEEE Transactions on Affective Computing, doi: 10.1109/TAFFC.2025.3632063.

Questions for authors

1.Could the authors clarify how the velocity-prediction objective contributes to augmentation when the generated residuals are obtained directly from hidden representations? 2.Have the authors compared the proposed residual generation with augmentation obtained through actual reverse flow integration? 3.How does CoMA-DiT compare with simpler latent-space augmentation strategies, such as Gaussian perturbation, latent Mixup, or conditional feature generation? 4.Could the authors evaluate the framework under cross-subject transfer settings? 5.How sensitive is the method to the diffusion timestep used during augmentation? Are the timesteps sampled independently for different augmented samples? 6.How does the performance change when the paired source modality is randomly shuffled or replaced with an unrelated sample during augmentation? 7.What is the additional training time and computational overhead compared with the original backbone and other augmentation methods?

21 comments
AuthorsOpenReview ↗

Overall Response to Reviewer XQux

We sincerely thank the reviewer for the careful evaluation and insightful suggestions. Your comments have helped us sharpen the distinction between cross-modal augmentation and predictive fusion, clarify the role of velocity supervision, and strengthen the empirical evaluation. We have added controlled analyses, an explicit reverse-flow augmentation variant, latent-space and adapted multimodal comparisons, and cross-subject evaluation. Our responses also address timestep sampling and sensitivity, the effect of disrupting sample-level modality pairing, and the additional training time and computational overhead. These revisions and clarifications strengthen both the methodological presentation and the experimental support for CoMA-DiT. Below, we respond to each comment individually, present the relevant evidence, and identify the corresponding manuscript revisions.

AuthorsOpenReview ↗

W1 — Distinction from fusion and feature adaptation (1/2)

Thank you for your insightful comments on the methodological distinction and the modest effect of removing CMA. We have sharpened the contribution around how paired modalities are used for learning, rather than presenting cross-attention or residual gating themselves as new primitives.

Prediction-time fusion versus training-time augmentation. A fusion module combines modality representations to predict a label for an observed pair. CoMA-DiT additionally constructs a label-preserving training representation for each target, conditioned on its paired source:

\mathbf Z^{b+}=\mathbf Z^b+\Delta^{b\leftarrow a}.$$ The classifier is supervised on both real and augmented multimodal representations: $$\mathcal L_{\mathrm{cls}} =\mathrm{CE}(h(\mathbf r),y)+\mathrm{CE}(h(\tilde{\mathbf r}),y).$$ After joint learning, classifier adaptation uses the learned latent training distribution with the encoders and augmentation module frozen. Test-time predictions use only real paired inputs and do not run the augmentation module. Thus, the distinctive role of CMA is to help construct additional training representations, not simply to provide another fusion block in the inference path. The complete training objective also includes velocity supervision, paired-modality consistency, and residual preservation. **What the CMA ablation actually removes.** The AVGC w/o CMA variant removes source-to-target token attention but retains the reliability gate and the other objectives. The gate still receives both pooled modality representations and their difference. The cross-modal consistency loss still supervises recoverable paired-modality information. This is therefore an ablation of **direct token-level source conditioning within an otherwise cross-modal augmentation framework**, not a removal of all cross-modal information. Consequently, its modest degradation should be interpreted as CMA's incremental contribution in this setting. It does not quantify the contribution of the entire augmentation framework. The numerical architectural results and a same-architecture classification control are given in Part 2. **Revision.** Section 3.1 now distinguishes augmentation from predictive fusion explicitly. Appendix D.2 clarifies which cross-modal paths remain in w/o CMA. The revised Related Work and Appendix C.4 position the framework relative to multimodal mixing, alignment, and diffusion-based methods.
AuthorsOpenReview ↗

W1 — Architectural and same-architecture evidence (2/2)

Architectural evidence on AVGC. Entries are mean ± SD (%). Each variant modifies the stated component while retaining the rest of the full-model protocol.

SettingAccuracyMacro-F1PrecisionRecall
w/o CMA79.99 ± 2.6279.82 ± 2.6480.49 ± 2.6780.00 ± 2.61
Fixed gate (g = 1)78.50 ± 2.8678.34 ± 2.8378.91 ± 2.9678.52 ± 2.84
EOG→EEG only78.90 ± 2.2978.76 ± 2.3279.28 ± 2.3478.91 ± 2.27
EEG→EOG only79.15 ± 3.2578.99 ± 3.2679.68 ± 3.3979.17 ± 3.23
CoMA-DiT80.52 ± 3.8280.39 ± 3.8281.00 ± 3.9080.53 ± 3.80

Keeping CMA improves accuracy, macro-F1, precision, and recall by 0.53, 0.57, 0.51, and 0.53 percentage points, respectively. Fixing the gate at one has a larger effect, reducing accuracy/macro-F1 by 2.02/2.05 points. Retaining only EOG→EEG or EEG→EOG augmentation reduces accuracy by 1.62 or 1.37 points. These results locate the contribution across the augmentation design rather than assigning all of the full-model gain to CMA. The CMA conclusion is specific to this AVGC ablation.

Do generated representations add value beyond the architecture? A second experiment retains the same generator, encoders, gate, auxiliary objectives, target noising, and optimizer-step budget. A classifies only real representations in both stages. B classifies real and augmented representations in Stage I, but also uses real-only adaptation in Stage II.

Stage I classificationAccuracyMacro-F1PrecisionRecall
A: Real only78.35 ± 3.7378.15 ± 3.8179.06 ± 3.7078.35 ± 3.73
B: Real + Aug79.61 ± 2.9779.39 ± 2.9780.30 ± 3.1179.61 ± 2.97

Adding generated-feature classification improves accuracy/macro-F1 by 1.26/1.24 points, with gains on precision and recall as well. Because the two settings have identical architecture and both retain auxiliary generative training and the second training stage, this comparison supports the contribution of task supervision on the generated representations beyond the presence of those components alone.

The added magnitude-matched Gaussian/Mixup comparisons on AVGC and DTU provide another complementary control. They retain the CoMA-DiT reference perturbation magnitude but use random or same-class interpolation directions. CoMA-DiT obtains higher mean scores in both settings. These results support the augmentation formulation without requiring the claim that CMA alone accounts for its benefit.

Revision. Appendix D.2, Table D2, reports the architectural comparisons. Appendix D.4, Table D4, gives the same-architecture controls. Appendices C.4 and D.7 specify and evaluate the magnitude-matched alternatives. We have retained the observed modest CMA effect rather than describing it as the dominant source of improvement.

Reviewer_HjWsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a cross-modal Diffusion Transformer that generates latent augmentations for one modality conditioned on another. It combines cross-attention with reliability-gated residual connections, allowing paired physiological signals to provide generative supervision rather than simply fusion. Across auditory attention decoding and emotion recognition tasks, it outperforms baselines and improves in accuracy and macro-F1.

Critical strengths
  1. The paper has a novel perspective on viewing paired modalities as mutual generative supervision. The writing is also clear and easy to follow.
  2. The experiments are extensive, with 20 baselines and ablations.
Critical weaknesses
  1. The role of diffusion is unclear. The augmentations come from a single forward pass without iterative denoising, and the predicted velocity is used only as a training loss, so this is not a genuine diffusion process.
  2. The gain cannot be attributed to the generative component, as opposed to the added parameters, the extra training stage, or noise regularization. A no-generation control with the same architecture should be reported.
  3. The paper claims a general multimodal paradigm but tests a single modality pair (EEG–EOG), with only one dataset per task.
Questions for authors

How does CoMA-DiT compare with the same cross-attention residual module trained without noise or the velocity objective, or with latent noise or mixup of equal magnitude?

13 comments
AuthorsOpenReview ↗

Overall Response to Reviewer HjWs

We sincerely thank the reviewer for the careful evaluation and insightful suggestions. Your comments have helped us clarify the augmentation mechanism, strengthen the experimental controls, and broaden the evaluation of CoMA-DiT. In response, we have clarified the roles of velocity supervision and reverse integration, added targeted control experiments, and extended the evaluation to an additional dataset and new modality configurations. These revisions have improved the clarity and empirical support of our work. We provide point-by-point responses below and indicate the corresponding changes in the revised manuscript.

AuthorsOpenReview ↗

W1 — Clarifying the role of velocity supervision (1/3)

Thank you for your careful examination of the augmentation mechanism. Your observation has helped us distinguish two operations that the original description conflated: learning a conditional velocity field and integrating that field to generate an augmentation. The submitted Single-Pass model performs the former; the added CoMA-DiT (Reverse) explicitly performs the latter.

Single-Pass: velocity-supervised residual augmentation. For direction b→ab\rightarrow a, we construct Zτa=(1−τ)Za+τϵa\mathbf Z^a_\tau=(1-\tau)\mathbf Z^a+\tau\boldsymbol\epsilon^a and process it with clean source tokens Zb\mathbf Z^b. The resulting target representation H\mathbf H feeds two separate heads:

\Delta_{\mathrm{SP}}=s\\,g\\,\tanh\\!\bigl(\phi_\Delta(\mathbf H)\bigr),\qquad \mathbf Z^{a+}=\mathbf Z^a+\Delta_{\mathrm{SP}}.$$ The velocity target is $\boldsymbol\epsilon^a-\mathbf Z^a$, the derivative of the linear clean-to-noise path. Its loss updates the shared Transformer and trainable encoders, shaping the representation used by the residual head. The residual head itself is trained through classification, cross-modal consistency, and preservation objectives. **The predicted velocity is not integrated or directly added to the clean target in Single-Pass.** We now describe this mechanism as velocity-supervised latent augmentation rather than iterative diffusion sampling. **Reverse: explicit conditional velocity integration.** Following your suggestion, we added a path initialized from a partially noised target at $\tau_s$. It holds the source fixed and integrates $$\frac{d\mathbf x}{d\tau}=v_\theta(\mathbf x,\mathbf Z^b,\tau),\qquad \mathbf x(\tau_s)=(1-\tau_s)\mathbf Z^a+\tau_s\boldsymbol\epsilon^a$$ backward to zero using Heun predictor-corrector updates. The integrated endpoint $\bar{\mathbf Z}_0$ defines a bounded, gated displacement: $$\Delta_{\mathrm{Rev}} = s\\,g_{\mathrm{Rev}}\tanh\\!\left(\frac{\bar{\mathbf{Z}}_0-\mathbf{Z}^{a}}{s}\right),\qquad \mathbf{Z}^{a+}=\mathbf{Z}^{a}+\Delta_{\mathrm{Rev}}.$$ Here the velocity directly determines the integration trajectory. The Reverse gate uses the integrated target and source representations, as specified in Appendix A.4. Both variants retain bidirectional modality augmentation; “Reverse” refers to the direction of integration in time, not to exchanging EEG and EOG. Neither augmentation path is required at test time. **Revision.** Figure 3 now contrasts the two paths. Section 3.3 explains the difference, Appendix A.4 details Reverse, and Appendices A.5 and D.3 report the comparisons below.
AuthorsOpenReview ↗

W1 — Single-Pass versus Reverse across datasets and tasks (2/3)

We compare both variants with their unaugmented backbones on AVGC, DTU, DEAP-A, and DEAP-V, using EEG-EOG inputs. DARNet is the backbone for AAD and DBConformer for EMO. Entries are mean ± SD (%); SD is calculated across three runs within each subject and then averaged over subjects. Bold denotes the highest mean in each setting.

Dataset/taskMethodAccuracyMacro-F1PrecisionRecall
AVGCDARNet76.24 ± 4.1173.69 ± 5.3174.14 ± 5.3974.81 ± 4.68
AVGCCoMA-DiT (Single-Pass)80.52 ± 3.8280.39 ± 3.8281.00 ± 3.9080.53 ± 3.80
AVGCCoMA-DiT (Reverse)79.73 ± 3.4079.58 ± 3.4879.22 ± 3.2779.73 ± 3.40
DTUDARNet83.38 ± 1.6382.81 ± 1.7383.30 ± 1.7682.62 ± 1.84
DTUCoMA-DiT (Single-Pass)87.31 ± 1.6286.72 ± 1.7087.30 ± 1.7586.48 ± 1.63
DTUCoMA-DiT (Reverse)86.10 ± 1.4485.74 ± 1.5185.95 ± 1.4685.72 ± 1.56
DEAP-ADBConformer94.92 ± 1.7894.26 ± 2.2094.52 ± 1.9094.30 ± 2.45
DEAP-ACoMA-DiT (Single-Pass)97.17 ± 1.1896.86 ± 1.3397.03 ± 1.2696.83 ± 1.46
DEAP-ACoMA-DiT (Reverse)97.98 ± 1.3697.40 ± 1.5897.75 ± 1.5397.30 ± 1.64
DEAP-VDBConformer95.11 ± 1.6394.91 ± 1.6995.16 ± 1.7394.85 ± 1.67
DEAP-VCoMA-DiT (Single-Pass)97.47 ± 0.9297.37 ± 0.9597.55 ± 0.9397.31 ± 0.96
DEAP-VCoMA-DiT (Reverse)97.84 ± 1.5697.65 ± 1.6497.87 ± 1.5997.61 ± 1.67

Both paths improve the backbone across all four metrics, with task-dependent rankings. Single-Pass performs better on AVGC and DTU, improving accuracy/macro-F1 over DARNet by 4.28/6.70 and 3.93/3.91 percentage points, respectively. Reverse performs better on both DEAP tasks: relative to Single-Pass, accuracy/macro-F1 increase by 0.81/0.54 points on DEAP-A and 0.37/0.28 on DEAP-V, with higher precision and recall as well.

These results show that the velocity-integrated path produces useful augmentations, while the direct residual path remains effective without iterative integration. The comparison concerns the two implemented augmentation paths, including their residual construction and gate inputs, rather than an isolated change of numerical solver. The complete comparison is in Appendix A.5, Table A2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Rebuttal and Key Revisions

We sincerely thank the reviewers and AC for their time and constructive feedback. We appreciate that all three reviewers found the paper informative and considered their soundness concerns minor or fixable. Two rated the presentation as clear and easy to follow (HjWs, UU12). Reviewer HjWs recognized our "novel perspective" on mutual generative supervision, clear writing, and extensive experiments. Reviewer XQux acknowledged the reasonable cross-modal augmentation approach and the useful ablation, backbone, and statistical analyses. Reviewer UU12 highlighted the improvements across tasks, the practical value of training-only augmentation, the careful data splitting, and the supporting ablations.

The subsequent discussion has been particularly encouraging. Reviewer XQux noted that the additional experiments had "mostly addressed" their concerns. Reviewer UU12 recognized that the matched-budget controls directly addressed their main concern, concluding that the revision "makes a stronger case". Reviewer HjWs acknowledged the "impressive amount of additional experiments", recognized the supporting evidence from the controlled comparisons, described the three-modality experiments as "very convincing", and considered the rebuttal and revised paper "much stronger".

We responded to each review concern through new experiments, analyses, and clarifications. Revisions are highlighted in blue in the PDF. The table summarizes our responses, with details in the reviewer discussions.

Concern (reviewer)What we didWhere in the paper
Distinction from fusion and feature adaptation (XQux)Clarified training-time cross-modal augmentation without an additional inference-time fusion module. Analyzed cross-modal attention, gating, and bidirectionality.Sec. 2–3, App. D.2, Table D2
Diffusion mechanism and reverse sampling (HjWs, XQux, UU12)Distinguished velocity-supervised Single-Pass residual augmentation from Reverse integration. Implemented Heun integration and compared both on AVGC, DTU, and DEAP-A/V.Sec. 3.3, Fig. 3, App. A.4–A.5, Table A2
Whether integration itself improves augmentation (XQux, UU12)Compared Euler-1 and Heun-10 with fixed gate inputs, then tested alternative gate inputs. Measured step sensitivity and cost.App. A.5, Table A3, Fig. A1
Contribution of velocity supervision under matched conditions (HjWs, XQux, UU12)Ablated only the velocity loss under matched budgets, yielding +0.98 pp AVGC accuracy when enabled. Added per-subject results.App. D.3, Table D3, Fig. D1
Whether learned augmentations help beyond added capacity and training (HjWs, XQux, UU12)Compared real-only and real-plus-augmented Stage I classification with matched architecture, losses, and steps, and real-only Stage II. Generated-feature classification gave +1.26 pp AVGC accuracy.App. D.4, Table D4
Insufficient latent-space and recent multimodal comparisons (HjWs, XQux)Added magnitude-matched Gaussian and Mixup, conditional latent VAE/GAN controls, and task-adapted M³ixup, CAAL, DiffAT, and ABDiff comparisons on AVGC and DTU.Sec. 2, 4.2, and 4.4, Table 2, App. C.4 and D.10, Figs. D8–D9
Limited datasets and modality combinations (HjWs, UU12)Added DTU for AAD, EEG–EMG for EMO, and joint EEG–EMG–EOG decoding. Reported full comparisons and the three-modality extension.Sec. 4.1 and 4.5, App. E.1–E.3, Tables E1–E3
Unseen-subject generalization (XQux, UU12)Added 12-fold AVGC LOSO evaluation, improving on DARNet by 4.07 pp accuracy and 6.03 pp macro-F1.App. E.4, Fig. E1
Statistical independence (UU12)Averaged three seeds within each subject and recomputed paired exact Wilcoxon tests with metric-wise BH correction.App. D.9, Fig. D7
Timestep sampling and sensitivity (XQux)Clarified independent Stage I timestep sampling for each trial and augmentation direction. Evaluated fixed and randomly sampled Stage II timesteps.Sec. 3.3, App. A.1, App. D.6, Fig. D3
Source pairing and subject variability (XQux, UU12)Tested within-class and global shuffling, reducing AVGC accuracy by 0.44/0.56 pp, and reported per-subject differences.App. D.7, Figs. D4–D5
Computational and inference costs (XQux)Profiled training time, inference latency, parameters, and GPU memory against DARNet, Noise, and DDPM. Confirmed no test-time generator.App. D.8, Fig. D6

We are grateful for the reviewers' thoughtful suggestions and for the opportunity to strengthen this work. Their feedback helped us clarify the mechanism, isolate component contributions, strengthen comparisons, and broaden our evaluation. We would be happy to provide further details during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.