- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents AffectVerse, an emotional world model framework for multimodal affective computing. It introduces an Emotion World Module (EWM) that combines cross-modal temporal imagination, MAMA belief aggregation, and belief injection to incorporate predictive emotional representations into multimodal reasoning. Experiments under different datasets show gains over baselines.
Critical strengths
1.The paper explores an interesting direction by introducing predictive world modeling into multimodal affective computing. 2.The proposed EWM provides a structured approach to learning future-aware representations and integrating them into MLLM-based emotion reasoning. 3.The experimental evaluation is comprehensive.
Critical weaknesses
1.The wirting should be further polished to reduce AI style and increase logical coherence, especially the experiment analysis section. 2.The multi-step temporal imagination mechanism is not sufficiently justified. According to Appendix D.6, all rollout steps are supervised using the same pooled future target rather than distinct future horizons. This raises concerns about whether the model actually learns progressive emotional dynamics. 3.The implementation of belief injection is unclear. Eq. (1) extracts hidden states from the Thinker, while Eq. (8) inserts belief tokens into these representations. It is unclear how the injected tokens influence subsequent language modeling if the backbone hidden states have already been computed. 4.The training and inference settings differ considerably. The model learns to predict future segments during training but uses all observed tokens at inference. The effectiveness of the learned predictive mechanism under this setting needs further justification. 5.The temporal alignment between modalities is insufficiently considered. Audio and video are split using the same keep ratio without explicit timestamp alignment, which may lead to inconsistent temporal contexts when modeling rapidly changing emotions. The modality coflict is also lack of consideration. 6.Some experimental claims and reporting require clarification. The reported +3.40-point improvement is calculated over six benchmarks, whereas the nine-benchmark improvement in Table 1 is approximately +2.58 points. Additionally, the results are based on a single reported random seed, making it difficult to assess statistical stability.
Questions for authors
1.Could the authors clarify whether the rollout steps correspond to different future time horizons or repeated predictions of the same future representation? 2.At which transformer layer are the belief tokens injected, and how do they interact with the original LLM hidden states? 3.Can the authors provide a quantitative comparison between samples with significant emotional transitions and samples with relatively stable emotions? 4.How does AffectVerse perform when audio and visual cues provide conflicting emotional information? 5.How sensitive is the method to different temporal sampling rates or longer audiovisual sequences? 6.Could the authors provide an analysis of whether the imagined representations capture emotion transition directions rather than only future emotion categories? 7.Does the performance improvement remain consistent across multiple random seeds? Conducting significance test would be helpful. 8.How would the framework perform when textual information is unavailable or unreliable, considering that text bypasses the EWM? 9.Could the authors further analyze failure cases where temporal imagination leads to incorrect emotional predictions or negatively affects the original backbone's performance? Besides, more cases with emotion trajectory should be provided.