RLE-Reasoner: Building Trustworthy Emotion Reasoning from Complementary EEG/EOG Evidence through Counterfactual Diagnosis and Diagnosis-Guided Repair

Multimodal large language models (MLLMs) can generate natural-language explanations for emotion judgments, yet fluent reasoning may describe physiological patterns absent from the recorded signals. We propose RLE-Reasoner, a build–diagnose–repair framework that connects emotion reasoning to verifiable EEG/EOG response evidence. In Stage 1, RLE-MLLM combines separate EEG/EOG encoders, Complementarity-Aware Response Fusion (CARF), and a physiological prefix to predict seven-class emotions and generate structured physiological evidence with a natural-language explanation. Seven attributes independently extracted from raw signals supervise the structured EEG/EOG evidence fields without entering the model input; their verbalization supplies the explanation target. In Stage 2, RLE-Eval uses controlled replacement of real EEG/EOG recordings to test whether the generated evidence reflects the current signals. In Stage 3, RLE-PostTrain turns these diagnoses into GRPO rewards that encourage correct physiological descriptions and responses to changed evidence. On a 5,040-trial, twelve-subject dataset with 1,260 held-out test trials, RLE-MLLM 14B achieves 66.52% seven-class cross-subject emotion accuracy and 62.08% matched physiological-state accuracy. Post-training improves overall physiological-grounding accuracy from 63.84% to 74.15% and state Macro-F1 from 46.21% to 58.10%, with matched emotion accuracy changing from 66.52% to 66.66%. Blinded human assessment of free-form text against supplied physiological reference states finds that consistency increases from 58.33% to 69.06%. These results show that physiological evidence can guide the diagnosis and repair of generated emotion reasoning.