acceptodds
Withdrawn from ICLR 2027

Semi-Supervised Multimodal New Intent Discovery via Progressive Learning and MLLM-Guided Semantic Correction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Semi-supervised new intent discovery aims to identify unseen intents using labeled examples of known intents and unlabeled data. Existing methods primarily rely on text and overlook complementary acoustic and visual information. Moreover, semantic feedback from large language models can preserve incorrect intent groupings when conveyed through summaries of the current clusters or local rewriting. We propose PLSC, a method that combines progressive multimodal representation learning with semantic correction guided by a multimodal large language model (MLLM). To mitigate instability in semi-supervised multimodal clustering when modality cues are inconsistent, the progressive strategy stabilizes shared multimodal representations through supervised contrastive learning using known intent labels before each clustering round and gradually increases the contribution of pseudo-label classification. To correct erroneous intent groupings, we incorporate local semantic relations inferred by the MLLM from representatives as soft constraints in centroid refinement, allowing the corrected centroids to guide subsequent assignments and optimize intent boundaries. Experiments on MIntRec, MIntRec2.0, and MELD-DA demonstrate consistent gains of approximately 1% to 3% over existing methods across different known class ratios, providing a methodological foundation for semi-supervised multimodal clustering.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Semi-supervised new intent discovery, multimodal representation learning, progressive learning, semantic centroid correction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VVMNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PLSC addresses semi-supervised multimodal new intent discovery using labeled examples of known intents and unlabeled examples containing both known and novel intents. It combines multimodal pretraining with iterative clustering, supervised contrastive refreshes using known labels, density-based sample selection, and a gradual increase in pseudo-label classification weight. An MLLM judges whether representative sample pairs express the same intent using text and video. These relations become soft penalties in constrained centroid refinement, with known-class centers held fixed during correction; the corrected centers initialize subsequent clustering. Experiments cover MIntRec, MIntRec2.0, and MELD-DA at different known-class ratios. Component ablations support benefits from progressive training and, in some settings, semantic feedback, although attribution relative to matched alternatives remains incomplete.

Critical strengths
  1. A meaningful task and a concrete feedback mechanism. Discovering new intents from partially labeled multimodal data is practically relevant. PLSC converts MLLM judgments into local same/different-intent relations and uses finite penalties to influence subsequent clustering, rather than requiring generated intent names to define the categories. The feedback interface is explicit, and the appendix documents prompts and computational costs.

  2. Component ablations demonstrate useful conditional gains. The paper does more than report improvements of the full pipeline. At a known-class ratio of 0.25, the pseudo-label weight schedule improves the reported average of four clustering metrics by 2.18, 2.43, and 3.51 percentage points across the three datasets. Holding centroid optimization in place while removing semantic penalties reduces this average by 0.11, 0.30, and 2.50 pp, respectively. These results support the usefulness of progressive supervision and show that semantic feedback can make a meaningful contribution, particularly on MELD-DA in this setting.

Critical weaknesses
  1. The incremental contribution over closely related approaches is insufficiently isolated. UMC already provides multimodal contrastive learning, density-based selection, and iterative clustering. LLM-derived pairwise relations are also established: LANID uses them for representation learning, and Viswanathan et al., Large Language Models Enable Few-Shot Clustering (TACL 2024, Section 2.2), use LLM-generated pairwise must-link/cannot-link constraints in PCKMeans, with weighted penalties for constraint violations. PLSC adds known-class refreshes, gradual pseudo-label supervision, and a specific centroid-feedback loop, but the current component-removal experiments do not establish their advantage over a simple matched combination of these existing ideas. Moreover, the baseline descriptions do not clearly establish matching of modality inputs, backbones, and external-model access. This is an attribution gap, not evidence that the comparisons are necessarily unfair. Given the limited methodological distance, it materially weakens the case for acceptance.

  2. The mechanism evidence is narrower than the motivation suggests. Cross-modal inconsistency is a plausible source of instability, but supervised refreshes and gradual pseudo-label weighting can also help ordinary single-modality clustering. Removing pretraining does not isolate the benefit of additional modalities or establish that cross-modal conflict has been reduced. The direct structural analysis is also modest: in Section 5.2, MIntRec2.0 weighted intent purity changes from 0.4569 to 0.4576 after correction, while silhouette remains nearly unchanged. This supports preservation of structure more clearly than substantial repair of intent boundaries in that setting. Together with the dataset-dependent feedback gains and unreported uncertainty of paired performance differences in the key ablations, the results leave the scope and mechanism of the claimed improvement insufficiently established.

Questions for authors
  1. Clarification of the comparison and incremental value: Please specify the modalities, backbone, and external-model access used by the main baselines. Is there an existing comparison against a simple semi-supervised UMC-based system under matched inputs and supervision, with comparable relational feedback and query budget? If not, one representative matched comparison would be particularly informative; reproducing every related method is unnecessary. This comparison would help establish the method’s incremental contribution.

  2. Clarification of the evidence for the central motivation: Are results available comparing text-only and full multimodal inputs while keeping the training strategy and feedback configuration fixed? Please also provide paired per-seed differences, or their uncertainty estimates, for the existing semantic-penalty ablation. These would help distinguish the contribution of modality inputs from training changes and assess whether the smaller feedback gains are stable. If cross-modal inconsistency is intended only as motivation rather than an experimentally established mechanism, please clarify that scope.

Reviewer_eGGSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PLSC, a three-stage framework for semi-supervised multimodal new intent discovery. Its progressive learning strategy applies a supervised contrastive refresh using known labels before each clustering round and ramps up the weight of the pseudo-label classification loss. It uses an MLLM to make pairwise judgments on representatives selected from each candidate new cluster to refine new intent centroids. PLSC achieves the best average scores on MIntRec, MIntRec2.0, and MELD-DA across all three KCR settings.

Critical strengths
  1. The presentation is clear and easy to follow.

  2. Studying new intent discovery in a semi-supervised multimodal setting is interesting.

  3. The paper includes extensive experiments.

Critical weaknesses
  1. PLSC's novelty is incremental. LANID [1] already uses LLMs to infer semantic relations between samples, while MCSP [2] introduces MLLM guidance into multimodal intent discovery. The progressive learning component also builds on UMC’s density-based sample selection [3]. The paper does not clearly establish their methodological contribution beyond existing methods.

  2. The paper does not clearly explain whether the six baselines in Table 1 receive the same audio and visual inputs as PLSC or use the same backbone models. Without baseline comparisons under similar setting, it is difficult to position PLSC’s superiority.

  3. The authors argue that multimodal information is important for semi-supervised NID, but provide no text-only ablation or ablations removing audio or visual inputs. It is therefore hard to tell which modalities contribute to the gains.

  4. The accuracy of the MLLM’s judgments and its effectiveness in correcting errors have not been verified. The method relies on SAME/DIFFERENT judgments, but the paper does not report how accurate they are. Soft constraints allow for incorrect judgments, but this does not mean that misleading feedback will not harm clustering

  5. The appendix reports MLLM token consumption, but the experiments only use Qwen3.7-Plus as the MLLM corrector, without comparing models of small size. It remains unclear whether these costs are necessary or whether a smaller MLLM could handle such binary relation judgments. The cost–performance trade-off is also not analyzed.

[1] Fan et al. LANID: LLM-assisted New Intent Discovery. LREC-COLING, 2024.

[2] Gu et al. Unsupervised Multimodal Intent Discovery via MLLM-Guided Concept Generation and Semantic Propagation. arXiv:2607.21908, 2026.

[3] Zhang et al. Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances. ACL, 2024.

Questions for authors
  1. Can you clarify the modality inputs, backbone models, supervision, and external resources used by each baseline? Under a matched setup, does using semantic relations for centroid correction outperform using the same relations in a contrastive or triplet loss? Or can you provide baselines comparison under the same multimodal and model setting?

  2. Can you provide results for text-only, text+audio, text+video, and the full model under the same training protocol? Separately, how much does providing video to the MLLM improve its judgments and the final clustering results compared with text-only prompting?

  3. How accurate are the MLLM’s SAME and DIFFERENT judgments, and how many incorrect assignments does semantic correction fix versus introduce? A manual audit or an evaluation with controlled relation noise would help establish the reliability of this component.

  4. Can you compare Qwen3.7-Plus with a smaller MLLM under the same representative pairs and query budget, reporting relation accuracy, final clustering performance, latency, and inference cost? This would help determine whether a cheaper corrector offers a better cost–performance trade-off.

Reviewer_U79NOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PLSC for semi-supervised multimodal new intent discovery using text, audio, and video. Following multimodal pretraining, the method alternates between progressive representation learning and MLLM-guided semantic centroid correction. Progressive learning reinforces known-intent representations and gradually increases pseudo-label supervision, while pairwise semantic judgments from an MLLM serve as soft constraints to refine centroids for subsequent clustering. Experiments on MIntRec, MIntRec2.0, and MELD-DA report improvements over existing methods across different known-class ratios.

Critical strengths
  • The method combines known-intent supervision with MLLM-derived semantic relations to guide multimodal clustering. Incorporating these relations as soft constraints allows centroid refinement to balance semantic feedback with the geometry of the learned representations.

  • The evaluation examines the method’s performance across different datasets and supervision settings, with further analyses of its main components and computational costs, providing a useful assessment of the method’s effectiveness and computational requirements.

Critical weaknesses
  • The methodological advance over closely related work is insufficiently established. PLSC builds on UMC’s multimodal representation learning and density-based selection, MCSP’s representative selection, and established pairwise constrained clustering. LLM-derived pairwise semantic supervision has also been explored in LANID. The paper does not sufficiently explain what new methodological insight this integration provides beyond adapting existing techniques to semi-supervised multimodal intent discovery.

  • The evaluation does not clearly establish the method’s advantage over existing approaches under comparable input conditions. The paper does not clarify whether the baselines receive the same multimodal inputs, and no text-only PLSC variant is provided. This makes it difficult to distinguish improvements attributable to the proposed method from those due to additional modalities. Moreover, closely related multimodal clustering methods, including UMC and MCSP, are absent from the comparison, leaving PLSC’s incremental value over existing multimodal approaches insufficiently established.

  • The role of MLLM feedback in correcting clustering errors is insufficiently validated. The paper does not evaluate the SAME/DIFFERENT judgments against ground-truth labels or examine whether they correctly challenge existing assignments. Since representatives are selected by the largest margin (Eq. 13), the queried samples already have strong geometric support for their assignments. Therefore, accurate judgments may confirm existing relations rather than reveal errors. Given the modest effect of removing the semantic penalties in Table 2 at KCR = 0.75 (0.35, 0.27, and 0.91 percentage points in average scores) and the substantial runtime share of MLLM calls during training (34%–52%), this missing validation is particularly relevant.

  • The aggregate results do not isolate improvements in novel-intent discovery. PLSC repeatedly reinforces known-intent representations through supervised contrastive learning, yet the evaluation reports clustering metrics jointly over known and novel classes. Without separate results for these groups, it remains unclear whether the gains primarily reflect better recognition of known intents or improved discovery of novel ones.

Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.