acceptodds
Withdrawn from ICLR 2027

SCHRONO: CONDITIONAL DIFFUSION MODELING VIA SENSORY-CHRONO HIERARCHY FOR MULTIMODAL MENTAL HEALTH COUNSELING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal counseling integrates visual, acoustic, and textual cues to understand a client’s emotions and provide empathetic support. However, two architectural weaknesses remain. First, the two objectives are coupled: perception updates perturb the distribution of generated text. Second, causal attention restricts each video frame to earlier frames, so a late disclosure cannot revise an early reading. We introduce SCHRONO, a framework for multimodal emotion understanding and counseling response generation. It uses a frozen masked-diffusion language model with two components: the Sensory layer trains encoders, projectors, attention pooling, and light heads for emotion and strategy prediction; the Chrono layer takes video, audio, and dialogue history to generate counseling responses through masked diffusion. Separate trainable pathways decouple perception from generation. Bidirectional attention lets each frame access the full video and dialogue context. On the MESC dataset, SCHRONO outperforms the compared methods in emotion recognition and counseling strategy prediction. Its responses also receive the highest average trustworthiness ratings from LLM judges. Our method also recognizes emotions across datasets and generates coherent supportive responses on ESConv without training on that corpus. Further fine-tuning improves 13.1 points for emotion recognition on IEMOCAP. Replacing bidirectional attention with causal attention in the same model lowers response quality, supporting the value of full-context access.

Primary area:
generative models
Keywords:
Diffusion Language Models、Multimodal Dialogue Systems、Emotion Recognition and Strategy Prediction、Parameter-Efficient Adaptation、Bidirectional Attention Topology

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qeeBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCHRONO, a multimodal counseling framework built on a frozen LLaDA diffusion backbone. It introduces a Sensory–Chrono hierarchy, where the Sensory Layer uses visual, acoustic, and textual information for emotion recognition and counseling strategy prediction, while the Chrono Layer uses multimodal features and dialogue context to generate counseling responses. The authors adopt staged training and separate parameter pathways to reduce interference between classification and generation tasks. They also argue that bidirectional attention helps incorporate later disclosures when interpreting earlier multimodal cues in counseling conversations. Experiments on MESC show improved classification performance and higher average LLM-judge scores for generated responses compared with the evaluated baselines.

Critical strengths
  1. Clearly defined research problem. The authors investigate potential gradient interference between multimodal perception and text generation during joint training, and they provide theoretical analyses of generation gradients and correlations across token-level gradients.
  2. Strong performance. SCHRONO outperforms the compared baselines on emotion recognition, counseling strategy prediction, and response generation.
  3. Comprehensive experimental analysis. The authors provide ablation studies on modality contributions, model components, and training strategies, which help clarify the roles and effectiveness of different parts of the proposed framework.
Critical weaknesses
  1. Unclear boundaries between architectural components. The Introduction distinguishes the shared backbone from the discriminative and generative layers, and Fig. 1 places SigLIP, WavLM, the text encoder, projectors, and feature pooling within the Multimodal Input module. However, Section 3.2 describes these feature extraction and processing operations together with the classification procedures of the Sensory Layer, including the per-frame features used for generation. Section 3.3 then introduces these features as inputs to the Chrono Layer. This makes the responsibilities and boundaries of the different modules insufficiently clear.
  2. Lack of direct empirical validation of the gradient-interference hypothesis. The authors identify gradient interference between generation and classification objectives on shared parameters as an important motivation for task decoupling. Although theoretical analyses are provided, the paper does not report empirical statistics on the gradients themselves.
  3. Insufficient justification for why bidirectional attention. The authors motivate bidirectional attention by arguing that early video tokens cannot access subsequent information under causal attention. However, an autoregressive model can integrate the complete available context when generating its final response and reinterpret earlier information. Therefore, the statement in the Abstract (around line 16): “so a late disclosure cannot revise an early reading” is not fully justified if interpreted as a limitation on the model's final contextual understanding rather than on the representations of early tokens.
Questions for authors
  1. Could the authors provide gradient statistics for the generation and classification objectives under a joint-training setting with shared parameters to directly support the gradient-interference hypothesis? Specifically, are there substantial differences in gradient norms or significant conflicts between gradient directions?
  2. What specific challenges in multimodal counseling are addressed by adopting the existing bidirectional attention mechanism of LLaDA? Given that SCHRONO inherits LLaDA's bidirectional attention and a counseling-domain adapter from prior work, could the authors further clarify what distinctive capabilities or incremental methodological contributions the proposed multimodal conditioning design provides for Progressive Disclosure scenarios, compared with existing bidirectional diffusion-based counseling models?
Reviewer_UUYAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCHRONO adapts a frozen masked-diffusion language model for multimodal emotional-support dialogue. A Sensory pathway predicts user emotion and counselor strategy from video, audio, and text features. A Chrono pathway learns multimodal projections and prefix tokens for response generation. In the second training stage, frozen copies of classification projectors separate the trainable classification and generation parameters.

The paper reports improvements on MESC classification, response evaluation by three LLM judges, and blinded human preference studies involving lay participants and clinical psychologists. It also includes component ablations, cross-corpus recognition experiments, ESConv generation, and attention-mask diagnostics.

Critical strengths

The classification improvements are substantial. SCHRONO reports roughly ten percentage points higher emotion accuracy than MultiMood SFT+GRPO, with results over three runs. Modality ablations indicate that the classifier benefits from both video and audio rather than relying only on text.

The response evaluation is broader than a single automated score. It includes three LLM judges, blinded human comparisons, expert ratings, and same-backbone generator ablations. The paper also discloses the counseling adapter’s training provenance and acknowledges that preference scores do not establish clinical validity. These are useful controls and appropriate qualifications.

Critical weaknesses

The central mechanism claims are stronger than the evidence. Disjoint trainable parameter blocks establish the absence of direct cross-task gradients, but freezing the backbone does not guarantee that changing its conditioning embeddings preserves generation quality. Table 22 also shows that the existing counseling adapter supplies most of the improvement over base LLaDA: 5.60 to 7.55, compared with 7.55 to 8.03 from the added conditioning stack. The incremental contribution deserves a matched comparison and paired uncertainty estimates.

The attention-mask intervention demonstrates that this bidirectionally trained checkpoint relies on its original attention pattern. It does not establish an advantage over a suitably trained causal model. The paper acknowledges this limitation, but the motivation and conclusions still overstate the architectural distinction. In particular, a causal decoder’s response tokens can attend to the entire preceding multimodal context even if early frame representations cannot be updated by later tokens.

The theoretical appendix contains incorrect arguments. Equation (8) does not generally define a normalized sequence distribution. For a one-token binary vocabulary with a uniform masked predictor, its right-hand side is 1−t+t/2 for each token, summing to 2−t rather than one. Consequently, the Jensen argument does not establish the claimed NLL bound for the actual diffusion sampler. This does not invalidate the standard masked-diffusion objective, but its derivation here must be replaced.

The attention analysis also needs correction. Lemma B.6 yields a product of layer-wise expansion factors, not generally linear growth with depth or damping. Proposition B.7 does not follow from content-only attention because different output contents can assign different weights to the same prefix. These statements currently provide misleading support for the empirical interpretation.

Questions for authors

Please reconcile the 454 “complete” test responses used for generation evaluation with the 1073 therapist utterances in Table 5. What preprocessing or eligibility rules produce this subset, and are the same example IDs used for every system?

Can you provide paired, dialogue-clustered uncertainty estimates for the generator ablations in Table 22, especially adapter-only versus full SCHRONO? Can the blinded human comparison also include the adapter-only generator?

What result supports preserved conditional generation quality beyond unchanged backbone weights? Please separate this empirical claim from the valid zero-cross-gradient property.

Can a causal baseline be matched for counseling adaptation, external emotion pretraining, frame coverage, and training budget? A bidirectional context encoder with a causal response decoder would also help test the proposed progressive-disclosure motivation.

Please correct the probability definition and likelihood derivation in Appendix B.1, and revisit the attention propositions. Direct gradient measurements would be more convincing evidence for the claimed generation–classification imbalance.

How are human wins and ties aggregated across raters and dimensions? Please provide uncertainty estimates appropriate for the crossed dialogue/rater structure rather than applying a binomial interval to an averaged preference rate without further justification.

Reviewer_vd3NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCHRONO, a multimodal counseling framework built on a frozen diffusion language model. It uses a Sensory layer for emotion recognition and counseling strategy prediction, and a Chrono layer for response generation. The two pathways are designed to reduce interference between perception and generation. The method also uses bidirectional attention to incorporate information from the full dialogue context.

Experiments on MESC show improvements in emotion recognition, strategy prediction, and counseling response quality. The authors also provide ablation studies, cross-dataset evaluations, and human evaluations.

Critical strengths
    1. Good performance on multimodal counseling tasks. The method achieves strong results on both emotion recognition and counseling strategy prediction. It also performs well in response generation, with improvements supported by both LLM-based evaluation and human preference studies.
    1. A practical way to separate perception and generation. Using a frozen backbone with separate perception and generation pathways is a reasonable design. The two-stage training process also shows clear benefits over joint training from scratch.
Critical weaknesses
  1. The main improvements may not come from the proposed method. The model uses an existing BiGraph-Diffuse counseling adapter. According to Table 22, this adapter improves the generation score from 5.60 to 7.55, while the proposed prefix and sensory components further improve it to 8.03. This suggests that much of the improvement comes from the existing adapter. Although the authors provide an ablation study, I am still not fully convinced that the proposed architecture offers a substantial advantage over simpler multimodal adaptation methods.

  2. The benefit of bidirectional attention is not fully established. The paper argues that bidirectional attention improves multimodal counseling. However, replacing the attention mask of a pretrained bidirectional model with a causal mask introduces a mismatch with its pretraining. The observed performance drop may therefore not be caused by the attention mechanism alone. The comparison with Llama-3.1-8B also involves different pretrained backbones. I think the current results show that the model benefits from its original attention structure, but do not fully establish that bidirectional attention is better than a properly adapted causal model.

Questions for authors
  1. Since most of the generation improvement comes from the BiGraph-Diffuse adapter, how much benefit does SCHRONO provide over a simpler frozen-backbone model with multimodal projectors and classification heads?

  2. Can the authors further clarify whether the benefit of bidirectional attention comes from the attention structure itself or from the pretrained model? Would a causal model trained with comparable conditioning and training settings show a similar result?

Clarification that would change my score: please train Llama-3.1-8B or Qwen2-7B with the same projectors, the same 49 conditioning tokens, the same sensory heads, and the same counseling adaptation used for LLaDA, and change only the attention direction. If that matched model still loses by a wide margin, the claim that bidirectional attention improves the counseling replies holds. If the gap closes, it does not. The current 8.03 versus 6.03 comparison changes the backbone, the pretraining, the adapter, and the attention pattern at once.

In Table 22, bare LLaDA scores 5.60 and the counseling LoRA alone reaches 7.55, while the prefix and sensory tokens add about 0.48. Llama with the prefix scores 6.03 and does not receive that adapter; Qwen2-7B at 5.79 was not trained with these modules. What is the judge mean when Llama or Qwen is given the same counseling adaptation and the same sensory tokens? Without that number, the gain cannot be attributed to bidirectional attention.

The drop from 8.03 to 7.10 comes from putting a causal mask on a model pretrained with bidirectional attention. Is there a run in which an autoregressive model is given bidirectional attention under the same training budget? If not, the supported statement is that removing bidirectional attention hurts this LLaDA checkpoint, not that bidirectional attention makes counseling replies better than autoregressive systems.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.