- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the problem of strictly inductive unsupervised multimodal intent discovery and proposes the IntentForge–IntentBridge (IntentFB) framework.
First, the authors employ a frozen Multimodal Large Language Model (MLLM) to generate a "semantic card" for each utterance, devoid of any predefined category labels. Subsequently, IntentForge derives multiple semantic views from these cards to construct a consensus neighborhood graph. By integrating spectral clustering, core-prototype refinement, and anchored propagation, the framework establishes a fixed set of anonymous training categories. Next, IntentBridge transfers this categorization to unseen samples through reliability-weighted student–EMA training, supported by a frozen exemplar retrieval mechanism. During inference, each query is processed independently using only the frozen artifacts derived from the training phase; no relationships between test samples are established, and no test-time updates are performed. The proposed method achieves an accuracy (ACC) of 62.92%, 47.70%, and 39.49% on the MIntRec, MELD-DA, and IEMOCAP-DA datasets, respectively. The appendix provides further analysis regarding reliability, the impact of pseudo-label noise, the stability of representations and decision-making, and the sufficient conditions required for anchored propagation.
Critical strengths
-
The author’s motivation is clear and well-defined. The paper highlights a crucial distinction: while Multimodal Large Language Models (MLLMs) can provide reasonable semantic interpretations for individual utterances, this does not imply they can construct a consistent category system across an entire dataset. Decoupling "per-sample semantic description" from "category system construction" is a well-motivated research approach. This differs from the logic of previous works, which typically perform clustering first and then use representative samples to generate semantic guidance.
-
The paper presents rigorous controlled experiments for the "category partitioning module," accompanied by a thorough analysis of the underlying mechanisms. By keeping factors such as prompts/instructions, semantic features, context, input budget, data partitioning, and random seeds consistent, the authors demonstrate that IntentForge achieves strong performance across multiple benchmarks compared to baseline methods.
-
The appendix provides a direct analysis of the relationship between "reliability" and "pseudo-label error," while also offering a detailed breakdown of the dynamics of "effective supervision noise." This analysis is highly valuable for understanding why the proposed method is effective.
Critical weaknesses
-
The benchmark advantages have not been validated under conditions where models and representation schemes are fully aligned, leading to unfair comparisons. The current evaluation mixes results obtained from local re-runs with those cited directly from the original GLAD/MCSP papers. While methods from various studies retain their respective original models and input configurations, the proposed approach—IntentFB—utilizes MiniCPM-o-4.5 to generate semantic "cards" and employs BGE-M3 for encoding. The appendix acknowledges that discrepancies in models and input configurations persist in these comparisons. Consequently, the substantial performance gains observed on MIntRec and MELD-DA cannot be definitively attributed to the Forge/Bridge algorithms themselves, as opposed to the more robust semantic representation pipeline. The authors must provide a reasonable explanation or additional experimental evidence to address this weakness.
-
The authors should address the contradiction between the "category-collapse" argument and the decline in certain experimental metrics—specifically, the lower macro-recall on MELD-DA/IEMOCAP-DA, as well as the lower ARI, NMI, and FMI scores on IEMOCAP-DA.
Questions for authors
-
The benchmark advantages have not been validated under conditions where models and representation schemes are fully aligned, leading to unfair comparisons. The current evaluation mixes results obtained from local re-runs with those cited directly from the original GLAD/MCSP papers. While methods from various studies retain their respective original models and input configurations, the proposed approach—IntentFB—utilizes MiniCPM-o-4.5 to generate semantic "cards" and employs BGE-M3 for encoding. The appendix acknowledges that discrepancies in models and input configurations persist in these comparisons. Consequently, the substantial performance gains observed on MIntRec and MELD-DA cannot be definitively attributed to the Forge/Bridge algorithms themselves, as opposed to the more robust semantic representation pipeline. The authors must provide a reasonable explanation or additional experimental evidence to address this weakness.
-
The authors should address the contradiction between the "category-collapse" argument and the decline in certain experimental metrics—specifically, the lower macro-recall on MELD-DA/IEMOCAP-DA, as well as the lower ARI, NMI, and FMI scores on IEMOCAP-DA.
-
How were the shared hyperparameters in Table 6 determined? Please explicitly state the development criteria and the specific data used at each step of the model selection process.
-
There are grammatical errors in the paper, the author should revise it.
7 comments
Answer to Question 2
We sincerely thank the reviewer for this thoughtful and constructive observation. We fully agree that higher ACC alone does not establish more complete recovery of semantic categories, and we appreciate the reviewer's careful attention to the limitations revealed by our results. This feedback helps us clarify the scope of our claims and improve the presentation of our findings. (Our Table 2, p.8; Figure 4 and Section 5, p.9.)
The motivating observation in Table 1 and Appendix F concerns the tested direct-MLLM category-generation procedure on MIntRec, which produces duplicated categories and highly concentrated assignments. (Our Figure 1 and Table 1, p.2; Appendix F, p.20.) This motivates separating sample-level semantic description from dataset-level category formation. We agree that IntentFB does not necessarily eliminate semantic merging or recover every ground-truth category across all datasets. In particular, producing nonempty clusters ensures a valid partition but does not guarantee recovery of all semantic categories. (Our Section 3.2, p.5; Appendix B.1, p.14; Rosenberg and Hirschberg, 2007, Section 2.)
We particularly appreciate the reviewer for highlighting the mixed experimental results. On MELD-DA, IntentFB achieves higher ACC than GLAD (47.70% versus 37.79%), while Figure 4 reports lower macro-recall. On IEMOCAP-DA, the ACC improvement is smaller (39.49% versus 38.26%), accompanied by lower macro-recall, ARI (20.11 versus 24.84), NMI (24.55 versus 28.22), and FMI (37.05 versus 37.37). These findings indicate that improvements in ACC do not necessarily translate into uniformly better category recovery or overall partition quality. We agree that this important distinction deserves greater emphasis in the main text. (Our Section 4.2, p.7; Table 2 and Section 4.4, p.8; Figure 4, p.9.)
ACC weights each category's recall by its sample frequency, whereas macro-recall gives categories equal weight. (Confusion-matrix formulation: Narasimhan et al., 2015, Section 2.) This explains how higher ACC can coexist with lower macro-recall, while the latter remains an important indication of uneven category-level performance. Furthermore, the lower ARI, NMI, and FMI on IEMOCAP-DA suggest additional limitations in partition quality. A more detailed attribution of these differences would require matched class-level predictions.
Inspection of available IEMOCAP utterances and Card records suggests one possible challenge for our description-based pipeline. Short responses such as “yeah” can express different conversational functions depending on context and vocal delivery. (Shang et al., 2020, Section 1; Saha et al., 2020, Sections 1 and 3.2–3.4) In the inspected records, responses associated with different dialogue-act labels sometimes receive the same broad description, “to confirm understanding.” [E1] Although other Card fields may preserve additional evidence, the intent description itself may not sufficiently distinguish these functions.
Such semantic ambiguity may affect neighborhood construction and subsequently propagate into the discovered partition and unseen-sample assignments. (Our Theorem 1, p.5; Section 3.3, pp.5–6; Appendix B.1, p.14; Appendix I.5 and Figure 10, pp.26–27.) We regard this as a plausible failure mechanism suggested by qualitative observations, and we appreciate the reviewer's comment for motivating a clearer discussion of this limitation.
Overall, IntentFB connects semantic description, category discovery, and strictly inductive prediction for unseen samples. Our results demonstrate dataset-dependent benefits, with the strongest overall clustering performance on MIntRec, while also revealing limitations in fine-grained semantic discrimination and balanced category recovery. (Our Table 2, p.8; Appendix I.2 and Figure 7, p.23.)
We sincerely thank the reviewer again for identifying this important issue. We apologize that our original wording may have suggested a broader claim than the evidence supports. We commit to explicitly clarifying these limitations in the revised main text, particularly in Sections 4.2, 4.4, and 5. Specifically, we will qualify “effectively addressing category collapse” to refer to the failures observed in our motivating direct-MLLM diagnostic, and clearly acknowledge that semantic merging, uneven category recovery, and dataset-dependent performance remain limitations of the current framework.
We are truly grateful for the reviewer's valuable feedback, which will help us present our contributions and limitations more accurately and transparently.
Answer to Questions 3 and 4
We sincerely thank the reviewer for raising this important question and for the opportunity to clarify our hyperparameter development procedure. The shared hyperparameters in Table 6 were developed through a three-stage process aimed at identifying a unified configuration across MIntRec, MELD-DA, and IEMOCAP-DA, rather than independently optimizing each dataset. Hungarian-aligned ACC was the primary development criterion, while ARI, NMI, and FMI served as complementary diagnostics of clustering quality.
Stage 1: Initial configuration development. We first explored candidate configurations using MIntRec training data, drawn from the official training partition containing 1,779 samples. This stage focused on semantic view construction, anonymous partition stability, and partition-transfer strategies. In particular, we examined fixed versus dynamically updated pseudo-partitions, reliability-guided supervision, and student–EMA learning. Candidate configurations were compared using training-label-informed ACC, with ARI, NMI, and FMI as additional diagnostics. These experiments informed the adoption of fixed anonymous partitions and reliability-guided partition transfer.
Stage 2: Cross-dataset assessment. We subsequently extended the assessment to the MELD-DA and IEMOCAP-DA training data, whose official training partitions contain 7,990 and 7,532 samples, respectively. The same evaluation criteria were applied to examine the behavior of candidate settings across different dataset scales and intent distributions. We also investigated alternative multimodal representation strategies, including temporal feature aggregation and learnable audio–visual sequence alignment, to better integrate acoustic and visual information.
Stage 3: Joint refinement and shared configuration selection. Finally, candidate adjustments were reviewed jointly using training data from all three benchmarks, with ACC prioritized and ARI, NMI, and FMI providing supporting evidence. Rather than selecting separate parameter combinations for individual datasets, we focused on their overall behavior under a common configuration. This process established the shared hyperparameters reported in Table 6.
Key hyperparameter settings. The discovery, sequence, and transfer representation dimensions were all set to 256. The neighborhood sizes for view weighting and graph construction were 20 and 30, respectively, with a reserved text-view weight of 0.42. Reliability-guided curriculum supervision expanded from 20% to 100% over 12 epochs, with EMA momentum set to 0.995. The sequence alignment branch used 20 epochs, a learning rate of 3 × 10⁻⁴, and batch size 64, while the student–EMA branch used 40 epochs, a learning rate of 3 × 10⁻³, and batch size 256. Frozen exemplar retrieval used 16 neighbors with temperature 0.10. These settings were shared across all three benchmarks, with the benchmark-defined number of categories K being 20 for MIntRec and 12 for both MELD-DA and IEMOCAP-DA.
We distinguish unsupervised model fitting from label-informed hyperparameter development. Training annotations were used to evaluate and compare candidate configurations through clustering metrics, but were not provided to semantic Card generation, anonymous partition discovery, or the model-fitting objectives. The development assessments used training data rather than an independently held-out validation partition. Following configuration selection, the final models were fitted on the full official training partitions, with data-dependent artifacts frozen before inference. The reported benchmark results follow the five-run evaluation protocol with seeds {0, 1, 2, 3, 4}, as specified in Section 4.1 and Appendix D.3.
We sincerely appreciate the reviewer's constructive feedback and will clarify the development criteria, the training data used at each stage, and the distinction between unsupervised model fitting and label-informed hyperparameter selection in the revised manuscript accompanying Table 6.
Regarding Comment 4, we sincerely apologize for the grammatical errors in our manuscript and commit to carefully correcting them, while also ensuring that our entire rebuttal is expressed with greater clarity, sincerity, and respect for the reviewers' valuable feedback.
Answer to Question 1
We sincerely thank the reviewer for this insightful comment. We deeply apologize for the confusion caused by our insufficiently clear presentation and commit to clarifying these issues in the revised manuscript.
Our contribution goes beyond representation construction. IntentFB integrates three complementary stages: a frozen MLLM generates sample-level semantic Cards without target category names; IntentForge discovers a consistent anonymous training partition through semantic consensus; and IntentBridge transfers this fixed partition to unseen samples through reliability-guided, strictly inductive prediction. Unlike MCSP, which generates semantic concepts from representatives selected after initial clustering, our Cards are constructed before category discovery, enabling semantic evidence to guide the initial partition.
Regarding comparison fairness, IntentFB and UMC, MCSP, and GLAD address the same task on shared benchmark datasets. All locally rerun baselines, including UMC, use identical training/test splits and a unified evaluation protocol. Literature-reported MCSP and GLAD results retain their original settings. We acknowledge that shared benchmarks do not imply identical representation interfaces: UMC and MCSP use pre-extracted modality features, and directly replacing them with Cards would change their original mechanisms. A common-Card evaluation would therefore require adapted implementations rather than native-system comparisons. We also acknowledge that MCSP's code is publicly available and will correct the outdated statement in our manuscript.
Crucially, Table 3 directly isolates the contributions of Forge and Bridge beyond the semantic frontend. We fix cached Cards, BGE-M3 features, context, input budgets, preprocessing, data splits, category counts, and five matched seeds. Compared with K-means plus nearest-centroid assignment (49.84%/32.09% ACC), the complete framework achieves 62.92%/47.70% on MIntRec/MELD-DA, improving ACC by 13.08/15.61 points. Under nearest-centroid assignment, Forge contributes 11.28/14.10 points; with Forge fixed, Bridge further contributes 1.80/1.51 points. Table 4 additionally validates Bridge under identical Forge partitions.
Thus, Table 2 demonstrates complete-framework performance, while Tables 3–4 provide controlled evidence that Forge and Bridge contribute additional improvements beyond the shared semantic frontend. We do not interpret native-system comparisons as representation-controlled evaluations.
We sincerely apologize again for the insufficient clarification. We commit to explicitly explaining the shared data splits, comparison protocols, baseline-result provenance, and module-level contributions in the revised manuscript and future versions. We are truly grateful for the reviewer's constructive feedback, which will help strengthen the clarity and rigor of our work.