acceptodds
Under review as a conference paper at ICLR 2027

NeuroInject: Bridging Brain and Behavior without Paired Subjects via Intermediate Clinical-State Injection for Depression Recognition

Abstract

Multimodal depression recognition integrates complementary behavioral and neurophysiological evidence to capture different manifestations of depression. Speech and language reflect behavioral and semantic manifestations, whereas EEG captures neurophysiological activity associated with depressive states. Existing multimodal methods typically fuse or align modality representations under naturally paired observations, whereas EEG and behavioral interview datasets are often collected from independent cohorts without subject- or sample-level correspondence. We study this unpaired setting and propose NeuroInject, which preserves the native audio–text pathway of a frozen multimodal large language model while introducing encoded EEG representations into its intermediate computation. Learnable clinical carriers retrieve modality-specific evidence through a shared cross-attention mechanism, accumulate it into a clinical state across layers, and write it back through gated low-rank mappings. EEG and behavioral samples are processed in independent forward passes, with cross-cohort transfer realized through jointly optimized shared parameters and a unified decision space. Experiments on MODMA, EDRA, CMDC, and DAIC-WOZ show that NeuroInject achieves the highest Accuracy among the compared methods on all four datasets, reaching 71.02, 86.81, 91.19, and 65.78, respectively, while AUROC gains are less consistent. Ablations show that the intermediate interaction contributes most of the improvement over a standalone EEG encoder.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Depression detection, electroencephalography (EEG), multimodal large language models, unpaired multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AXagOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the lack of subject-level paired EEG and audio-text interview data in depression recognition and proposes a framework called NeuroInject. The proposed method introduces a shared clinical-state module into the intermediate layers of a pretrained audio-language model, enabling cross-modal knowledge transfer through shared parameters. Experiments on four depression recognition datasets show that the proposed method outperforms the compared baselines in terms of classification accuracy.

Critical strengths

S1. Practical Significance of the Research Problem Collecting paired multimodal data from the same subjects is challenging in real-world clinical depression recognition scenarios. Therefore, the problem addressed in this paper, involving the lack of paired EEG and audio-text data, is of practical significance.

S2. Well-Motivated Architectural Design The proposed intermediate clinical-state injection mechanism provides a shared learning framework for heterogeneous modalities without requiring sample-level alignment, while preserving separate processing pathways for different modalities.

Critical weaknesses

W1. Insufficient Evidence of Cross-Modal Knowledge Transfer The current experiments only demonstrate the effectiveness of the proposed architecture. The paper lacks a direct comparison between single-modality training and multimodal joint training, and thus does not provide sufficient evidence of effective cross-modal knowledge transfer. Based on the existing results, it remains unclear whether the performance improvements come from knowledge transfer or simply from adopting a more complex model architecture.

W2. Insufficient Experimental Evaluation The ablation studies only evaluate the intermediate interaction mechanism and Direct State Fusion on EEG datasets, without independently examining the contributions of other key components, such as shared clinical-state updates, cross-attention, and shared LoRA. Furthermore, the two EEG test sets contain only 11 and 9 subjects, respectively, resulting in relatively small evaluation sample sizes.

Questions for authors

Could the authors conduct the following additional experiments?

  1. Single-modality vs. joint training: Provide a controlled comparison between single-modality training and multimodal joint training under the same NeuroInject architecture. This would help clarify whether the performance improvements come from cross-modal knowledge transfer or the architecture itself.
  2. More comprehensive ablation studies: Conduct more detailed ablation studies on key components, such as shared clinical-state updates, cross-attention, and shared LoRA. Additionally, could the ablation studies be extended to the audio-text datasets?
Reviewer_JdUZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes NeuroInject, a framework for depression recognition when EEG and behavioral (speech/text) data come from independent, unpaired cohorts. It injects encoded EEG representations into intermediate layers of a frozen Qwen2-Audio backbone. The method is evaluated on two EEG datasets (MODMA, EDRA) and two interview datasets (CMDC, DAIC-WOZ).

Critical strengths
  • The paper addresses a practical problem: physiological and behavioral depression data are usually collected from different cohorts, so unpaired.
  • The proposed interface is technically carefully designed, preserving the audio–text pathway of the backbone while handling variable-length inputs.
Critical weaknesses
  • The motivation for combining EEG with speech and language needs stronger justification. These modalities are generally collected under very different settings, and application scenarios that use both are rare in practice.
  • It is unclear how the method exploits unpaired data to improve performance. Since EEG and behavioral samples come from different cohorts and are processed in separate forward passes, the only connection is through shared parameters. The paper does not explain why EEG information from one cohort should help speech/text prediction in a different cohort.
  • The central claim of cross-cohort knowledge transfer is not validated. The ablation (Table 3) is conducted only on the EEG datasets and compares against the standalone EEG encoder, so the gain could simply come from adding a 7B LLM and LoRA on top of the EEG features, rather than from joint training with behavioral data. Key ablations are missing: NeuroInject trained on EEG only vs. jointly, and on behavioral data only vs. jointly. Without these, it is unclear whether the behavioral cohort helps EEG recognition, or vice versa, at all.
  • The accuracy gains are not supported by other metrics. NeuroInject has much lower AUROC than EEGNet on EDRA and lower AUROC than Qwen2-Audio and DepressInstruct on DAIC-WOZ, so the higher accuracy may come from threshold choice rather than better classification. On DAIC-WOZ, balanced accuracy (56.90) is close to chance, so claim of “highest accuracy on all four datasets” is therefore not convincing.
  • The layout of page 3 needs to be improved.
  • The symbols in Equation 1 need clear definitions.
  • The presentation needs significant improvement. The method section is dense with notation and difficult to follow.
Questions for authors
  • The paper should explain more concretely where such a model would be deployed and why combining the two is clinically meaningful.
  • Explain what knowledge is actually transferred between the two.
  • Improve experimental design and validation.
  • Improve writing as suggested in weakness.
Reviewer_RQeEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NeuroInject, a framework for depression recognition using unpaired EEG and behavioral data from different subject cohorts. The main contribution is to enable knowledge sharing between these modalities without requiring paired samples.

Critical strengths
  1. Good motivation. The paper addresses a practical problem: EEG and behavioral datasets are often collected from different subjects, making paired multimodal learning difficult. The motivation is clear and meaningful.

  2. Interesting design. Injecting EEG features into intermediate Transformer layers rather than directly converting them into input tokens is a reasonable idea.

  3. Evaluation on multiple datasets. The method is tested on four datasets with subject-disjoint splits, and achieves improvements in Accuracy.

Critical weaknesses
  1. The paper does not clearly show the benefits of cross-modal learning. The main claim is that EEG and behavioral data can benefit from joint learning, but the experiments do not directly support this. A comparison between EEG-only training and joint training using the same architecture is missing. Without this, it is hard to tell whether the improvements come from cross-modal knowledge sharing or simply from the larger model.

  2. The performance gains seem too small given the model complexity. NeuroInject uses a 7B-scale backbone, while EEGNet is a much smaller model. On EDRA, NeuroInject improves Accuracy by only 1.39% and Balanced Accuracy by 0.40%, while AUROC drops from 94.83% to 80.39%. These results make it difficult to justify the additional model complexity.

  3. The ablation studies are not sufficient. The proposed method combines several components, but their individual contributions are not well studied. In particular, comparisons with simpler approaches, such as input-level injection or standard cross-attention, are missing. It is unclear whether the proposed SCSP is actually necessary.

Questions for authors
  1. Does joint training actually help? How does NeuroInject perform when trained only on EEG data or only on behavioral data? A direct comparison with joint training would help support the main claim.

  2. Is the 7B backbone necessary? Could the authors report the total/trainable parameters and computational costs? Given the small improvements over EEGNet on EDRA, is such a large model justified?

  3. How much does SCSP actually contribute? Would simpler methods, such as input-level injection or standard cross-attention, achieve similar results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.