acceptodds
Under review as a conference paper at ICLR 2027

NeuroPrompt: EEG-Conditioned Emotion-Aware Dialogue Generation with Large Language Models

Abstract

Although large language models can generate fluent dialogue, emotional response generation often relies on explicit emotion cues or manual labels that are rarely available in realistic interaction. Since EEG signals have been widely studied as a physiological source for emotion recognition, they can serve as an implicit affect cue for language models. To address the lack of paired EEG-dialogue corpora, we propose a staged framework for EEG-conditioned dialogue generation. This framework learns cross-subject emotion EEG representations, maps them into the LLM embedding space, and uses the projected representation to guide emotion-adaptive response generation. Under a cross-subject protocol, blind GPT-5.4 evaluation on 1,598 paired dialogue samples shows that EEG-Conditioned LLM achieves a 31.93% win rate and 2.23 average rank, outperforming Text-Only Baseline (22.44%/2.45) and Unconditioned LLM (11.67%/3.17) while approaching Gold-Label Oracle (33.96%/2.14); a 500-sample human questionnaire shows the same trend. We further conduct an online real-time EEG user study in which each participant's EEG is fed to the model during live interaction, and anonymized responses are rated for how well they match the participant's current emotional state. In this online study, EEG-Conditioned LLM also achieves the best human ratings and average rank among the compared models. Taken together, the offline and online results support EEG as a practical source of implicit emotion cues for response adaptation and suggest that brain-derived signals can improve dialogue generation beyond text-only supervision.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Large language models, Affective computing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xVsCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NeuroPrompt, a framework for emotion-aware dialogue generation that uses EEG-derived emotional information to adapt LLM responses. To address the lack of synchronized EEG–dialogue datasets, the authors introduce a staged training framework consisting of emotion representation learning with an EEG encoder, projection into the LLM embedding space, and EEG-conditioned response generation using soft prompts. The proposed approach is evaluated through offline experiments, human evaluations, and an online user study with real EEG recordings, demonstrating improvements in emotion-aware response generation.

Critical strengths

S1. Meaningful problem setting: Leveraging EEG to capture emotional information that may not be explicitly expressed in dialogue text is an interesting problem with potential practical value for emotion-aware LLM-based interaction.

S2. Evaluation with real users: In addition to LLM-based evaluations, the authors conduct human evaluations and an online EEG study involving 18 participants. These experiments provide valuable evidence of the approach's effectiveness under more realistic interaction settings.

S3. Baseline comparisons and acknowledgment of limitations: The paper includes comparisons with hard-label control and a gold-label oracle, along with ablation studies of the staged training framework. The authors also explicitly acknowledge limitations, including the binary emotion setting and reliance on non-synchronized EEG–dialogue training data.

Critical weaknesses

W1. Insufficient comparison with probability-based EEG conditioning: Although the proposed approach outperforms hard-label control, this comparison alone does not sufficiently establish the benefits of using high-dimensional EEG representations. Since the task involves binary POS/NEG classification, a straightforward alternative would be to use calibrated class probabilities from the EEG classifier as conditioning inputs to the LLM. Furthermore, because the EEG and dialogue data are paired solely through binary emotion labels, a probability-based approach represents a particularly important baseline. The current results do not clarify whether the improvements arise from preserving richer EEG representations or simply from avoiding the information loss caused by hard-label discretization.

W2. Limited methodological novelty: The proposed framework primarily combines existing techniques, including EEG encoders, projectors, soft prompts, LoRA-based LLM adaptation, and staged training. Although applying EEG-based emotional information to dialogue generation is an interesting direction, the methodological contribution beyond existing techniques remains unclear, particularly in light of related studies such as Thought2Text, E²-LLM, and NeuroChat.

W3. Concerns about the fairness of the Text-Only Baseline: The Text-Only Baseline is trained using emotion labels, but the emotion-conditioning input is removed at inference time. This differs from a model trained exclusively on dialogue text from the beginning, and the resulting mismatch between training and inference conditions may affect its performance. A comparison with a text-only model fine-tuned without emotion conditioning, using matched training data and computational budgets, is necessary to establish the benefits of EEG conditioning more convincingly.

Questions for authors

Q1. Probability-based conditioning: Have the authors evaluated a baseline that uses calibrated POS probabilities from the EEG classifier as conditioning inputs to the LLM? In particular, would the proposed method still outperform a probability-based approach that generates soft-prompt embeddings from the classifier's probability output while maintaining the same soft-prompt architecture?

Q2. Methodological novelty: Could the authors clarify the fundamental methodological contribution of the proposed staged training framework beyond existing EEG–LLM integration and emotion-conditioned dialogue generation approaches? In particular, which experimental results demonstrate advantages that cannot be achieved using existing conditional learning techniques?

Q3. Text-Only Baseline: Have the authors compared the proposed approach with a text-only model fine-tuned without any emotion-conditioning input during training, under matched data and computational budgets? If so, does EEG conditioning still provide consistent improvements over this baseline?

Reviewer_hSAxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NeuroPrompt, which combines EEG emotion encoding, embedding projection, and LLM fine-tuning to enable the model to adjust its response style based on EEG signals. Due to the lack of synchronously collected EEG–conversation data, the authors indirectly connect the two modalities through positive/negative emotion labels. The experiments consist of three parts: a blind LLM-judge evaluation on 1,598 test samples, a 500-sample questionnaire with three raters per item, and an online real-time EEG user study with 18 participants. The results show that the proposed method outperforms the text-only and unconditioned baselines, approaching the upper bound of the gold-label oracle.

Critical strengths

1.Using EEG as an implicit affective control signal for LLM dialogue is a well-motivated angle, with a clear distinction from prior work on EEG emotion classification, EEG-to-text, and EEG+LLM analysis. 2.Since users' emotions are not necessarily fully reflected in text, there is a clear rationale for exploring whether physiological signals can help personalize the responses of dialogue systems. 3.The paper's evaluation is comprehensive: in addition to the automatic LLM-judge evaluation on 1,598 test samples, the authors also conducted a 500-sample human questionnaire with three raters per item, as well as an online real-time EEG study with 18 participants. 4.Given the current lack of synchronously collected EEG–conversation corpora, connecting the two modalities through positive/negative emotion labels is a pragmatic approach. The fact that the proposed method approaches the upper bound of the gold-label oracle indicates that the pipeline works reasonably well under this constraint.

Critical weaknesses

1.The average ranks in Table 18 look off. By the paper's own rules, ties are not allowed, so in a three-model comparison, the average ranks should sum to 6. In Table 18, however, the three entries only add up to 4.98 (1.54 + 1.66 + 1.78). 2.The offline experiments are still somewhat unconvincing. In Stage C, the EEG signals and dialogues were not recorded together—they were simply matched up using positive/negative labels. As a result, testing basically checks whether the model can distinguish positive from negative in the EEG and switch its response tone accordingly. This only shows that the control signal works, rather than proving that EEG actually brings fine-grained emotional information into the dialogue beyond binary labels. 3.There still isn't enough evidence showing what advantage the projector really has over passing an emotion signal directly. Stage B maps EEG toward two fixed positive/negative targets, and Stage C also pairs them using binary labels. While the authors compare against passing discrete hard labels and show that the projector works better, they do not compare it with a simpler soft-control baseline that directly passes the classifier's probabilities or confidence scores. 4.The EEG sampling window in the online experiment is problematic. The system only takes EEG from the last few seconds before the video ends, whereas participants only start thinking about how to reply and typing after the video finishes. The paper doesn't report how long participants spent typing, nor were their emotions measured again during typing or rating—which leaves open the question of whether the EEG captured in those final seconds of the video actually reflects the participant's emotional state during the real interaction. 5.Stage C1 clearly outperforms C0, but the issue is that this step changes several things at once: adding another epoch of training, switching the training set entirely to paired data, and averaging multiple EEG segments. Averaging multiple segments was already tested separately in the ablations and didn't bring much gain, leaving the remaining two changes—training for an extra epoch and using paired data alone—still confounded together.

Questions for authors

1.Does Table 18 follow the tie-free ranking rule defined in Appendix D.2? If so, why does the sum of the average ranks equal 4.98 instead of 6? 2.How does your model perform when compared against (a) a fine-tuned text-only baseline that uses no emotion labels at all during either training or inference, and (b) a baseline that passes Stage A's posterior probabilities directly as a soft-control signal? 3.How long was the time gap between the online EEG sampling window and the text input? Did the experiments include any independent emotion detection measurements? Also, have you tested whether shuffling the EEG signals across trials degrades performance? 4.If cluster-robust statistical tests are conducted based on the 799 original dialogue contexts, do the significance conclusions of the main offline experiments still hold?

Reviewer_xgkGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks whether EEG can act as an implicit emotion cue for LLM dialogue generation, avoiding the explicit emotion labels that are unavailable in realistic interaction. Because no synchronized EEG-dialogue corpus exists, the authors assemble one from existing resources: emotion-elicited EEG from SEED (subjects 1–9 train, 10–12 validation, 13–15 test, binary POS/NEG) and dialogue from EmpatheticDialogues, with fine-grained emotions mapped to polarity and additional POS/NEG response pairs generated for contexts whose text does not reveal polarity. The method is staged: an EEGNet encoder trained cross-subject, a lightweight projector mapping the EEG representation into the LLM embedding space, and a Qwen2.5-7B with LoRA consumed as a soft prompt, refined on the paired subset. Evaluation combines a blind GPT-5.4 judge on 1,598 test samples, a 500-sample human questionnaire with 24 raters, and an online study in which 18 participants interacted live with EEG fed to the model.

Critical strengths
  • The online study is the paper's genuine contribution. Collecting EEG, dialogue, the model response and the rating within a single live episode is rare in this literature, and it is the only component of the paper that is not circular with respect to the emotion label: the participant knows their own state and does not know which system produced which response. The reported Wilcoxon tests over participant-level averages are the appropriate treatment.

  • The counterfactual check is well designed and informative. Swapping only the EEG condition while holding the context fixed, and reporting that only 7.2% of responses are identical (8.4% near-duplicates), is the right way to demonstrate that the model uses the conditioning signal rather than ignoring it.

  • The engineering is clean and the ablations are honest. The staged pipeline is fully specified, the projector-versus-hard-label ablation (56.4% win rate for the projector) is reported even though the margin is small, and the Limitations section is candid about label-matched rather than synchronized pairing and about GPT-generated supervision.

Critical weaknesses

W1. The work is organized as a resource contribution, but the resource is two unrelated datasets joined by a binary label, and the title and abstract invite a stronger reading than that. The EEG comes from SEED and the dialogues from EmpatheticDialogues; the two share only a POS/NEG polarity, and the core text-side data consist of POS/NEG response pairs generated by a model for contexts whose text does not reveal polarity. The resource is therefore a machine-generated text dataset plus a label join, not a new neural–language alignment corpus.

W2. The offline metric measures label compliance, and the paper's framing outruns what that metric can support. The judge is instructed that "Speaker emotion label is provided and must be respected," and the EEG condition is label-matched, so the gap between Gold-Label Oracle (33.96% win rate, rank 2.14) and the EEG model (31.93%, 2.23) has the shape of label-transmission efficiency rather than affect transfer. The authors narrow the claim themselves in Section 6.4, to "affect-level EEG control rather than fine-grained neural-language alignment," and for that narrower claim I accept the measurement; what then needs to be argued is where this framework sits relative to any other cheap source of polarity.

W3. The architecture is a standard multimodal bridge whose EEG-specific components are refinements rather than a mechanism, and the generality of the findings is narrow. EEGNet features, a linear projector, LoRA on a 7B backbone and a soft prefix are the established recipe; the multi-window group averaging (K = 3) and the weakly paired objective are sensible refinements but not a new mechanism. Generality is limited on four counts: one backbone (Qwen2.5-7B), three test subjects (SEED 13–15), eighteen participants in the online study, and supervision and judge drawn from the same model family (GPT-generated response pairs, GPT-5.4 judge).

Questions for authors

Why is there no condition in which the emotion is inferred from the dialogue text (by an LLM or a classifier)? That is the comparison on which the claim "beyond text-only supervision" actually rests, and the test contexts were deliberately selected so that their text does not reveal polarity — which makes the comparison both easy to run and highly informative.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.