acceptodds
Under review as a conference paper at ICLR 2027

Towards Robust Open-World Emotion Recognition via Equiangular Prototypes and Progressive Prediction

Abstract

Open-world emotion recognition aims to perform emotion classification using both labeled and unlabeled data, where the labeled data cover only a subset of emotion classes, while the unlabeled data encompass all classes. This setting is crucial for recognizing fine-grained unseen emotions that are difficult to collect under controlled laboratory conditions. However, existing open-world semi-supervised learning methods suffer from two major limitations: (1) they rely on similarity-based clustering for unconstrained aggregation, making it difficult to distinguish representations of different unseen classes in the absence of supervision for unseen classes; and (2) they primarily rely on the predictive confidence of a single model for class decisions, making them susceptible to distributional bias dominated by known classes and consequently prone to biased predictions. To address these issues, we propose a novel open-world emotion recognition framework, termed ROBIN. First, inspired by neural collapse, we introduce the simplex equiangular tight frame structure as equiangular prototypes to guide representations of different classes toward a well-separated distributional structure. Subsequently, we develop a progressive prediction strategy that first employs an auxiliary binary classifier to distinguish between known and unseen classes, and then masks class components inconsistent with the predicted base class, thereby enabling robust fine-grained class recognition. Extensive experimental results demonstrate that ROBIN outperforms existing state-of-the-art methods on the open-world emotion recognition.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Emotion Recognition, Semi-supervised Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_F7wiOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROBIN, an open-world emotion recognition framework designed to recognize both known and unseen emotion classes. The authors identify two limitations: unconstrained similarity-based clustering may fail to separate different unseen classes, while single-model prediction mechanisms may be biased toward known classes. In response, ROBIN introduces equiangular prototypes based on the ETF structure to encourage well-separated class representations. It further adopts a progressive prediction strategy that first distinguishes known from unseen classes using an auxiliary binary classifier and then masks class components inconsistent with the predicted category group to perform fine-grained emotion recognition.

Critical strengths
  1. Open-world emotion recognition is practically relevant because collecting labeled data for every fine-grained emotion is difficult, particularly for rare or previously unseen classes.

  2. The paper highlights the difficulty of separating unseen classes without direct supervision and the potential bias of class predictions toward known classes, providing a reasonable starting point.

  3. ROBIN outperforms existing state-of-the-art methods on the evaluated benchmarks, suggesting potential practical value, although the strength of this evidence depends on the experimental comparisons and ablation studies.

Critical weaknesses
  1. The paper argues that single-model decision mechanisms are susceptible to biases dominated by known classes and proposes a progressive prediction strategy to address this issue. However, it is unclear why the proposed binary classification and subsequent masking operations are sufficient to mitigate such bias.

  2. The paper characterizes the ETF structure as an 'ideal' geometric arrangement for separating emotion classes, but the rationale for its suitability to complex emotion recognition remains unclear. In particular, the ETF constraint imposes equiangular relationships among class prototypes, which may not reflect the semantic similarities or intrinsic structure of different emotions. Moreover, the proposed prototype-based formulation raises questions about how the ETF structure benefits unseen classes that lack labeled supervision. The theoretical discussion in Section 2.2 and the corresponding appendix also appears insufficiently supported by relevant literature. The authors are encouraged to clarify the assumptions underlying Proposition 1, explain why the resulting geometric structure is appropriate for this task.

  3. As mentioned by the authors in Line 096, 'we propose a fine-grained emotion recognition task', the motivation for the specific problem formulation and its distinction from existing open-world settings is not sufficiently discussed. Furthermore, the theoretical analysis appears to rely primarily on a single proposition, which does not seem sufficient to substantiate the broader claims concerning representation separation and robust fine-grained recognition.

  4. As illustrated in Fig. 2, the framework incorporates multiple loss terms, but the roles of several components, particularly the entropy and contrastive losses, appear to be insufficiently connected to the central methodological motivation. At a high level, the framework seems to reduce to a two-stage procedure: first classify samples as known or unseen, and then perform fine-grained classification within the corresponding group. Most importantly, the ETF alignment component is added without sufficiently demonstrating why it is necessary.

Questions for authors

See Weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer F7wi [1/2]

We sincerely thank you for the valuable comments and constructive suggestions on our manuscript. We respond to each concern in detail below and clarify the corresponding revisions we have made.


Q1: Please elaborate on why the proposed binary classification and subsequent masking operations are sufficient to mitigate biased predictions.

Thank you for raising this question. Due to the lack of supervision for unseen classes, the ETF classifier tends to assign higher probabilities to known classes than to unseen classes. To improve prediction robustness, we introduce an auxiliary binary classifier, independent of the ETF classifier, to distinguish known from unseen classes. When the binary classifier achieves high accuracy (up to 99.22% on SEED-VII), the subsequent masking operation reduces the candidate class space by excluding potentially high-probability but incorrect classes, thereby mitigating biased predictions. Further analysis is provided in Appendix E.1.

Q2: The ETF constraint enforces equiangular relationships between class prototypes, which may not accurately reflect semantic similarities or intrinsic structures among different emotions.

Thank you for raising this important concern. We acknowledge that enforcing equiangular relationships between class prototypes may disrupt the semantic similarities among different classes. However, given the open-world setting (limited labeled samples and incomplete class coverage) and the objective (fine-grained emotion recognition), ROBIN prioritizes improving class discriminability, particularly the representation of unseen classes. We will discuss this limitation in the revised manuscript and further clarify ROBIN's design objectives and scope of applicability.

Q3: How does the ETF structure benefit unseen classes in the absence of labeled supervision?

Thank you for raising this important question. We emphasize that ROBIN does not rely solely on the ETF structure to distinguish unseen classes; rather, their discriminative representations are learned through the collaboration of multiple modules. Equiangular prototypes serve as class-position anchors, enforcing inter-class separation through fixed classifier weights. Known classes can be directly aligned with their corresponding prototypes using label indices (0 to K−1K−1). For unseen classes, contrastive learning and a warm-up strategy first enable preliminary clustering (clusters KK, K+1K+1, …), followed by aligning these clusters with their corresponding class prototypes. Meanwhile, mean prototypes maintain intra-class representation compactness. Further details are provided in the Introduction and Appendix E.3.

Q4: Please clarify the assumptions underlying Proposition 1, explain why the resulting geometric structure is applicable and necessary for the current task, and provide sufficient literature support for the theoretical discussions in Section 2.2 and its appendix.

Thank you for your concern regarding the theoretical foundation and applicability of Proposition 1. This proposition shows that fixing classifier weights to a predefined structure can encourage different class representations to cluster around predetermined positions. It is motivated by the well-established neural collapse phenomenon [1, 2], under the assumption that samples from different classes are separable. Neural collapse may emerge naturally when training samples have high-quality labels. However, in open-world emotion recognition, labeled training samples are limited and class coverage is incomplete, making unseen classes difficult to distinguish. Therefore, ROBIN introduces an ETF classifier to provide structural priors through predefined geometric relationships between class prototypes, thereby enhancing inter-class representation separability. Further details are provided in the Introduction and Appendix E.3.

[1] Prevalence of neural collapse during the terminal phase of deep learning training, 2020.
[2] Pca Versus lda, 2001.

AuthorsOpenReview ↗

# Response to Reviewer F7wi [2/2]

Q5: The motivation for formulating the open-world fine-grained emotion recognition task and its distinction from existing open-world settings are insufficiently discussed.

Thank you for raising this concern. We agree that clarifying the differences from existing studies would better highlight our novelty. Existing open-world emotion recognition methods have two main limitations: (1) they require a large number of high-quality labeled samples; and (2) they typically treat all unseen emotion classes as a single category. However, real-world applications often lack sufficient labeled data and require fine-grained recognition of unseen emotions. For example, user satisfaction surveys require fine-grained classification of emotions expressed in user reviews. To address these limitations, we formulate an open-world emotion recognition task that aims to achieve fine-grained classification across different classes with limited labeled samples and incomplete class coverage. To the best of our knowledge, this is the first study to investigate this problem. Further details are provided in the Introduction and Appendices E.2, E.3, and H.3.

Q6: The theoretical analysis relies primarily on a single proposition, which is insufficient to support the broader claims regarding representation separation and robust fine-grained recognition.

Thank you for raising this important concern. Proposition 1 primarily analyzes the role of the ETF structure in open-world semi-supervised emotion recognition. However, achieving representation separation and fine-grained recognition in open-world settings requires the collaboration of multiple modules: (1) ETF-based equiangular prototypes (Lloc\mathcal{L}_{loc}) anchor class positions and enforce inter-class separation through fixed classifier weights; (2) auxiliary binary classifier (Lbin\mathcal{L}_{bin}) distinguishes known from unseen classes to improve prediction robustness; (3) contrastive learning (Lcon\mathcal{L}_{con}) and the warm-up strategy initially cluster unseen-class samples and then align these clusters with predefined prototypes; (4) mean prototypes (Lcal\mathcal{L}_{cal}) maintain intra-class compactness; and (5) entropy regularization (Lent\mathcal{L}_{ent}) encourages diverse prediction distributions to prevent overconfident predictions. We validate the contributions of these modules through ablation studies. We will elaborate on their individual and collaborative roles in the revised manuscript.

Q7: The framework involves multiple loss terms, but the roles of some components, particularly the entropy and contrastive losses, are not sufficiently connected to the core methodological motivation.

Thank you for raising this concern. Please refer to our response to Q6, where we clarify the individual and collaborative roles of these components in achieving representation separation and robust fine-grained recognition.

AuthorsOpenReview ↗

Follow-up on Rebuttal Responses

Dear Reviewer,

Thank you for taking the time to review our work. We sincerely apologize for any inconvenience this may have caused. As the rebuttal period is coming to an end, we would like to respectfully inquire whether you have had the opportunity to review our responses to your comments. We greatly value your feedback. Please feel free to let us know if you have any further questions. We hope that our clarifications have adequately addressed your concerns, and we look forward to any further feedback you may have.

Best regards,
The authors

Reviewer_rmD8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies open-world semi-supervised emotion recognition, where labeled data cover only a subset of emotion classes while unlabeled data contain both known and unseen classes. The authors argue that existing open-world SSL methods suffer from representation collapse among unseen classes and predictions biased toward known classes. To address these issues, they propose ROBIN, a framework comprising two components and assuming that the total number of emotion classes is known. First, inspired by neural collapse, the classifier is fixed to a simplex ETF serving as equiangular class prototypes, complemented by mean-prototype calibration on unlabeled data, to encourage well-separated representations. Second, a progressive prediction strategy employs an auxiliary binary classifier, regularized with Mixup consistency, to separate known from unseen samples, after which fine-grained classification is performed within the predicted group. Experiments on five emotion datasets spanning EEG, eye-tracking, text, and behavioral modalities report improvements over semi-supervised and open-world SSL baselines, supported by ablation, sensitivity, and visualization analyses.

Critical strengths
  1. A practically relevant problem setting: The paper addresses emotion recognition under both limited annotation and incomplete class coverage, with an emphasis on distinguishing multiple unseen emotion categories. This is valuable because it extends emotion recognition toward settings where the available labels cannot fully describe the emotional diversity in the data.

  2. Broad empirical investigation: The evaluation spans different emotion modalities and includes component ablations and sensitivity analyses. This breadth provides a useful basis for assessing the framework’s applicability and the contributions of its components. Subject to clarification of the evaluation issues, the reported improvements would support the practical value of the approach.

Critical weaknesses

Insufficient empirical support for the central claim: Several inconsistencies weaken the evidence that ETF-based alignment improves unseen emotion discovery. Under the stated definition, some average inter-class cosine similarities in Figure 6 fall below the mathematical lower bound −1/(K−1), while Figure 7 shows unexplained differences in evaluation sample counts between ROBIN and its ablations. These discrepancies undermine confidence in the representation analysis and ablation conclusions. Moreover, geometrically separated prototypes do not necessarily correspond to distinct semantic categories: the theoretical analysis assumes the desired class–prototype alignment, and replacing the ETF classifier with a learnable classifier does not isolate the benefit of ETF geometry from simply fixing the classifier.

Questions for authors

What evidence shows that the improvement in unseen-class discovery comes specifically from ETF geometry rather than simply fixing the classifier?

2 comments
AuthorsOpenReview ↗

Response to Reviewer rmD8

We sincerely thank you for the valuable comments and constructive suggestions on our manuscript. We respond to each concern in detail below and clarify the corresponding revisions we have made.


Q1: How can you demonstrate that the performance improvement in unseen-class discovery is attributable to the ETF geometry rather than merely fixing the classifier?

Thank you for raising this important question. We clarify that the ETF geometry is implemented through a fixed classifier satisfying the structure in Proposition 1 (i.e., equiangular prototypes); thus, the two are equivalent in our framework. We further validate its effectiveness through theoretical analysis and ablation studies. Specifically, Proposition 1 provides the theoretical analysis, with its proof presented in Appendix A. In the ablation study (Section 4.4), we replace the fixed classifier with a trainable fully connected layer of the same input and output dimensions. The resulting performance degradation confirms the effectiveness of introducing equiangular prototypes.

W1: Several inconsistencies in the experimental results undermine the credibility of the conclusions.

Thank you for pointing out these inconsistencies. We sincerely apologize for the errors caused by oversights during manuscript preparation. We have carefully reviewed the issues and their causes, as summarized below.

IssuesAnalysis
The GCD confusion matrix in Fig. 3 shows an accuracy of 70.65%, whereas the main text reports 72.19%.The confusion matrix is correct; the accuracy in the main text was mistakenly taken from a different experimental setting.
The number of samples in ROBIN's confusion matrix in Fig. 7 differs from that in the ablation study.The two experiments used different numbers of random seeds, and the reported predictions were aggregated across all seeds: 10 seeds for ROBIN and 8 for the ablation study.
According to Section 4.1.4, the training, validation, and test sets are split at an 8:1:1 ratio, but the number of test samples in the confusion matrix does not match the expected count.The confusion matrix aggregates predictions across multiple random seeds, rather than reporting those from a single run.

We clarify that these issues resulted solely from writing oversights and differences in the number of random seeds used, and do not affect the validity of the experimental results or the main conclusions. We will correct these errors and provide the necessary clarifications regarding the experimental settings in the revised manuscript.

W2: Please explain why some average pairwise cosine similarities in Fig. 6 are below the theoretical lower bound of −1/(K−1)-1/(K-1)?

Thank you for pointing this out. We clarify that Fig. 6 reports the average cosine similarity between unseen classes, rather than all classes. When only unseen classes are considered, their average pairwise cosine similarity can be lower than −1/(K−1)-1/(K-1). Our work focuses on mitigating representation collapse among unseen classes in open-world settings. Therefore, measuring their average pairwise cosine similarity more directly reflects the effectiveness of ROBIN.

W3: The theoretical analysis assumes an ideal class-prototype alignment, whereas geometrically separated prototypes do not necessarily correspond to distinct semantic classes.

Thank you for raising this concern. Inspired by the neural collapse phenomenon, we introduce equiangular prototypes. Neural collapse suggests that, for classification tasks, under ideal training convergence, representations within each class collapse toward their corresponding class mean, while the pairwise cosine similarities between different class means become equal, forming an ETF. In this case, each class has a unique class mean (i.e., a class prototype), establishing the correspondence between classes and prototypes.

AuthorsOpenReview ↗

Follow-up on Rebuttal Responses

Dear Reviewer,

Thank you for taking the time to review our work. We sincerely apologize for any inconvenience this may have caused. As the rebuttal period is coming to an end, we would like to respectfully inquire whether you have had the opportunity to review our responses to your comments. We greatly value your feedback. Please feel free to let us know if you have any further questions. We hope that our clarifications have adequately addressed your concerns, and we look forward to any further feedback you may have.

Best regards,
The authors

Reviewer_taiXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Emotions are far more diverse than a small set of discrete categories can capture. However, collecting data with labels for every emotion is difficult in practice. The paper therefore considers the following setting:

  • Only some emotion categories—the known classes, comprising no more than half of all classes—are identified in advance, and only 5% of the samples belonging to these classes are labeled.
  • Most of the remaining data are unlabeled and contain a mixture of known emotions and emotions never encountered as labeled classes, i.e., the unseen classes.
  • The goal is to (1) accurately classify known emotions and (2) partition unseen emotions in the unlabeled data into distinct clusters, such as “Group 1” and “Group 2.” The number of unseen classes is assumed to be known in advance.

The authors identify two problems with existing methods:

  1. Representation collapse: Without ground-truth supervision for unseen emotions, representations of different unseen classes collapse into a single cluster (Fig. 1).
  2. Biased prediction: The model’s predictions are biased toward the known emotions learned from labeled data.

The core ideas of ROBIN are as follows:

  1. Equiangular prototypes: For K emotion classes, the method predefines K target directions, one for each class. These directions are equally separated in angle and maximally spread out, forming a simplex equiangular tight frame (ETF; Proposition 1). They remain fixed throughout training, while the model learns to map samples toward their corresponding prototypes. This predefined separation is intended to encourage unseen emotions to form distinct clusters rather than collapse into a single group.
  2. Mean prototypes: Because unseen emotions lack ground-truth labels, the method computes the mean representation of samples assigned to each pseudo-label. It then pulls samples toward their corresponding means to form more compact clusters.
  3. Progressive prediction: Instead of directly predicting an emotion class, an auxiliary classifier first determines whether a sample belongs to a known or unseen class. If the sample is predicted as known, unseen classes are excluded from consideration; if it is predicted as unseen, known classes are excluded. The final prediction is then made among the remaining candidates. This mechanism aims to reduce the bias toward known classes.
  4. Additional training objectives: The method also uses regularization to prevent predictions from concentrating on a few classes, along with contrastive learning to encourage similar representations across augmented views of the same sample. Training consists of a warm-up stage followed by the main training stage.

The experiments use five datasets: SEED-V, SEED-VII, GoEmotions, SE-R1510, and SE-F1530. For each dataset, ROBIN’s heads are attached to an established backbone and trained. The method is compared with seven baselines—FixMatch, GCD, TRSSL, OpenCon, OpenNCD, CGMatch, and MOOD—using accuracy and F1 scores for all, known, and unseen classes, averaged over five random seeds. For unseen classes, the predicted cluster indices are aligned with the ground-truth emotion labels using Hungarian matching before evaluation.

Additional analyses include a comparison of known-versus-unseen discrimination on SEED-VII, component ablations on SEED-V, sensitivity to class splits and hyperparameters, changes in inter-class distances during training, and t-SNE visualizations. The authors claim state-of-the-art performance across all evaluated settings.

Critical strengths
  1. A practical and challenging problem setting across heterogeneous modalities. Existing open-set emotion recognition methods often group all unseen emotions into a single “unknown” category (Appendix E.2). This paper instead considers distinguishing among unseen emotions with very limited labeled data. This setting is closer to applications that require identifying distinct new emotions and could serve as a benchmark for future research. Given the open-ended nature of emotion categories and the high cost of annotation, the setting has practical relevance to affective computing. Applying the same framework to EEG, physiological signals, text, and behavioral time series also demonstrates its versatility across modalities.

  2. Substantial performance gains on EEG datasets. In Table 2, ROBIN improves overall accuracy over the strongest baseline by 12.7 percentage points on SEED-V (53.03 vs. 40.36) and 13.5 percentage points on SEED-VII (50.44 vs. 36.93). These gains are substantially larger than the reported standard deviations of approximately 1.4–1.9. The improvements also extend to F1 scores and the more challenging unseen-class accuracy, suggesting that the advantage is not confined to a single metric. Furthermore, the ablation study (Fig. 4) shows that removing the equiangular prototypes causes the largest performance drop, providing some support for the contribution of the proposed design.

Critical weaknesses
  1. Internal inconsistencies in the reported results undermine the reliability of the key findings. The following discrepancies can be identified from the numbers reported in the paper and its figures alone.
  • (a) GCD accuracy in Fig. 3. The confusion matrix [[13157, 5826], [1441, 4339]] gives an accuracy of (13157 + 4339)/24763 = 70.65%, whereas Section 4.3 reports 72.19%. Applying the same calculation to OpenNCD (77.82%) and ROBIN (99.22%) reproduces the values reported in the text.
  • (b) Evaluation sample counts in Fig. 7. The three ablation matrices for SEED-V each contain 23,736 samples, whereas the full ROBIN matrix contains 28,081. These totals should be identical if the methods were evaluated on the same data. Since the Base accuracy calculated from the ROBIN matrix (82.65%) matches Table 5, the main and ablation results may have been obtained on different evaluation sets.
  • (c) The evaluation set does not match the stated protocol. Section 4.1.4 describes an 8:1:1 split, yet the SEED-V confusion matrices contain 23,736–28,081 samples, corresponding to approximately 81–96% of the entire dataset (29,168 samples in Table 9). These counts cannot represent a test set comprising approximately 10% of the data.
  • (d) Indications that Hungarian matching may not have been applied. For the SEED-V unseen-class matrix in Fig. 11, the best of the six possible one-to-one assignments (1→2, 2→3, 3→1) yields an average of (0.65 + 0.38 + 0.55)/3 = 52.7%, while the worst yields 14.7%. However, the reported unseen recall of 32.60% in Table 5 matches neither extreme; instead, it closely matches the diagonal average without reassignment: (0.22 + 0.57 + 0.19)/3 = 32.7%. This raises the possibility that the Hungarian matching described in Section 4.1.3 was not applied during evaluation. Furthermore, each row of the unseen-class matrix sums to one across unseen columns alone, although the Base matrix indicates that 13% of SEED-V unseen samples were predicted as known. Those misclassified samples may therefore have been excluded from the unseen-class evaluation.
  • (e) Apparent artifacts of metric computation are presented as leading results.
    • The SE-R1510 unseen-class matrix, [[0.97, 0.03], [1.00, 0.00]], indicates that the two unseen classes have almost entirely collapsed into a single cluster. Dividing the reported accuracy of 73.76% by 0.97 suggests a majority-class proportion of approximately 76%, meaning that the reported result is comparable to, or worse than, assigning every sample to the majority class. Nevertheless, this value is highlighted in bold as the best result in Table 4. Thus, the very collapse that the paper claims to address remains evident in ROBIN’s own results.
    • The SE datasets contain only one known class, so the Known evaluation measures whether known samples are incorrectly routed to unseen classes, rather than whether different known emotions are distinguished. Under this evaluation, precision should always be 100%, irrespective of the model. However, an entry with no ground-truth samples appears to have been included in the average, resulting in a reported precision of 50.00 ± 0.00 across all five seeds. Recall (= accuracy/2) and F1 (38.78 and 29.31) can also be reproduced exactly from accuracy alone and therefore provide no additional information. Consequently, the rankings based on these metrics in Table 3 are not meaningful.
  • (f) Fig. 6 contradicts both the metric’s definition and the paper’s own theory. Section 4.6 defines the plotted quantity as the average cosine similarity across all class pairs. Under this definition, it cannot fall below −1/(K−1), the lower bound for KK vectors. Proposition 1 likewise identifies this value with maximal separation, and Fig. 6 marks it as the “Optimal” line. Yet, based on the plotted curves, all three SEED-V curves start near −1.0, and both w/o EP and w/o MP remain below the lower bound throughout training. On SEED-VII, ROBIN itself falls below the bound after the early training stage, reaching approximately −0.3 to −0.35. The authors may have averaged over only a subset of class pairs or plotted unnormalized inner products rather than cosine similarities, but either possibility would contradict the description in the text. In particular, taking the paper’s interpretation literally would imply that w/o EP on SEED-V achieves greater separation than the ETF configuration, which is maximally separated by definition. The interpretation that “ROBIN is closest to the optimum” in Section 4.6 is therefore unsupported, undermining the only quantitative evidence offered for improved inter-class separation.
  1. The central claim of discovering distinct unseen emotions lacks supporting evidence, particularly qualitative analysis. The paper’s central contribution is the discovery of distinct unseen emotion categories, but it does not directly demonstrate that such discovery occurs.
  • No qualitative results are provided. There is no analysis showing that the discovered unseen clusters correspond to coherent emotion categories. For GoEmotions, representative sentences from each cluster could readily be presented, yet none are included. Moreover, the results on this dataset raise doubts about whether meaningful clusters have formed. The nine known classes receive labeled supervision, whereas the nine unseen classes do not, so the known-class task would be expected to be easier. Nevertheless, known-class accuracy (20.18%) is lower than unseen-class accuracy (35.32%). This suggests that accuracy may be driven by a few populous classes rather than successful emotion discrimination. Indeed, the macro-averaged unseen recall of 11.88% in Table 5 is close to the expected value of approximately 11.1% for uniform random prediction across nine classes. Even acknowledging the inherent difficulty of this dataset—the fully supervised F1 score is 18.34% in Table 7—these results do not convincingly demonstrate the discovery of distinct unseen emotions.
  • The only visual evidence, the t-SNE plots, is weak. Fig. 10 lacks a legend distinguishing known from unseen classes, and substantial class overlap remains visible in ROBIN’s plot (Fig. 10a). The comparisons are limited to CGMatch and MOOD, which are among the weaker baselines in Table 2, rather than the stronger OpenNCD or OpenCon. Fig. 1 also appears to reuse the plots shown in Fig. 10c/d.
  • Hungarian-matched accuracy and F1 alone do not establish cluster quality. Hungarian matching uses ground-truth labels to find the most favorable assignment, so the meaningfulness of the clusters should also be assessed using metrics that do not depend on this assignment or through qualitative analysis. In the SEED-VII unseen-class matrix in Fig. 11, the maximum entry in each row is only 0.34–0.47. Thus, fewer than half of the samples from any unseen emotion are concentrated in a single cluster; instead, they are dispersed across multiple clusters.
  • The causal link between the proposed components and the reported gains is not established.
    • There is no ablation that removes the second central contribution: the exclusion step following known-versus-unseen classification. Fig. 4 removes only individual loss terms and EP. Furthermore, the ablations are conducted on SEED-V alone.
    • According to Fig. 9, unseen pseudo-label accuracy remains approximately 30–40% at the end of training. Although pseudo-labels need not be accurate from the outset in a k-means-like procedure, the paper does not analyze whether the iterative process actually improves clustering or instead reinforces errors made by the binary classifier.
    • The Base comparison in Fig. 3 is provided only for SEED-VII, where ROBIN achieves its highest Base performance.
    • The statement in Section 4.2 that “SEED-V known-class performance approaches fully supervised performance” compares accuracy on samples from two known classes (89.67%) with accuracy across all five classes (92.88% in Table 7). These are not directly comparable evaluation settings.
  • The baseline comparison is incomplete. The authors state that their experimental setting and warm-up strategy follow Cao et al. (2022) (Sections 2.1 and 3.3). They also cite other open-world semi-supervised learning methods, including Guo et al. (2022), Rizve et al. (2022a), Ye et al. (2024), and Qu et al. (2025) (Section 1 and Appendix H.2), but none of these methods is included in the comparison. Appendix G also does not explain how the closed-world semi-supervised methods—FixMatch, CGMatch, and MOOD—were adapted to predict multiple unseen classes.
Questions for authors

Clarification requests that could materially affect the assessment

  1. Fig. 3 / Fig. 7: The GCD confusion matrix yields an accuracy of 70.65%, whereas the text reports 72.19%. In addition, the SEED-V ROBIN matrix contains 28,081 samples, while the ablation matrices contain 23,736. Which values are correct in each case? Were the results in Table 5 and Fig. 4 obtained on the same evaluation set?

  2. Evaluation set: Do Tables 2–5 report results on the test set or on the unlabeled data used during training? The 8:1:1 split described in Section 4.1.4 does not appear consistent with the sample counts in the confusion matrices.

  3. Hungarian matching: The SEED-V unseen-class matrix in Fig. 11 appears to be shown without label alignment, and the reported unseen score matches the diagonal average before matching. Please clarify the actual evaluation procedure. Were unseen samples incorrectly predicted as known counted as errors or excluded? Was the same procedure applied to all baselines?

  4. SE datasets: The SE-R1510 unseen-class results indicate that the two classes have collapsed into a single cluster, with accuracy comparable to or below that of always predicting the majority class. How is this consistent with the claim of mitigating collapse? Also, why is Known Precision 50.00 ± 0.00 across all seeds when there is only one known class?

  5. Fig. 6: Under the definition in Section 4.6—average cosine similarity across all class pairs—the value cannot fall below −1/(K−1). However, several curves in Fig. 6 fall below this bound. Please clarify:

    (a) What quantity was actually computed: cosine similarity or unnormalized inner product? Please also define the y-axis label, “Rep Mean.”

    (b) Were all K(K−1)/2 class pairs included in the average, or only classes present in a batch or represented in the predictions?

    (c) In which space were the class means computed—backbone features or ETF logits—and using which labels—ground-truth labels or pseudo-labels?

    (d) Please provide a corrected plot computed according to the definition in the text and confirm whether the conclusion that “ROBIN is closest to the optimum” still holds.

  6. Qualitative results: Please provide representative samples, purity, and size for each discovered unseen cluster on GoEmotions. Please also report clustering metrics that do not depend on Hungarian matching, such as NMI and ARI, across all datasets.

  7. Ablation: Please report results across all datasets when predictions are made directly, without the candidate-exclusion step following known-versus-unseen classification. Could you also extend the Base comparison in Fig. 3 and the t-SNE comparison in Fig. 10 to all datasets and include the strongest baselines, OpenNCD and OpenCon?

  8. Pseudo-label quality: Although unseen pseudo-label accuracy is approximately 30–40% in Fig. 9, please provide an analysis demonstrating that the proposed iterative process improves clustering. Is this accuracy measured after Hungarian matching?

  9. Baselines: Why were Cao et al. (2022), from which the experimental setting and warm-up strategy were adopted, and the other cited open-world SSL methods—Guo et al. (2022), Rizve et al. (2022a), Ye et al. (2024), and Qu et al. (2025)—excluded from the comparison? How were FixMatch, CGMatch, and MOOD modified to predict multiple unseen classes?

  10. Hyperparameters: The key loss weight λ_ce varies substantially across datasets, from 0.02 to 0.9. For SEED-VII, the optimum reported in Appendix C.2 exactly matches the default value in Table 1. Were hyperparameters selected using the validation set?

Suggestions for improvement

  1. Report results averaged over multiple class splits and conduct statistical significance tests. Several reported differences fall within the standard deviations, such as SEED-V Known accuracy: 89.67 ± 5.10 for ROBIN versus 89.19 ± 4.85 for CGMatch.

  2. Evaluate settings with more known classes, rather than relying on the SE datasets’ single-known-class setting.

  3. Clarify the rationale for reducing GoEmotions from 28 to 18 classes.

4 comments
AuthorsOpenReview ↗

Response to Reviewer taiX [1/2]

We sincerely thank you for the valuable comments and constructive suggestions on our manuscript. We respond to each concern in detail below and clarify the corresponding revisions we have made to address these issues.


Q1: Several inconsistencies in the experimental results undermine the credibility of the conclusions.

Thank you for pointing out these inconsistencies. We sincerely apologize for the errors caused by oversights during manuscript preparation. We have carefully reviewed the issues and their causes, as summarized below.

IssuesAnalysis
The GCD confusion matrix in Fig. 3 shows an accuracy of 70.65%, whereas the main text reports 72.19%.The confusion matrix is correct; the accuracy in the main text was mistakenly taken from a different experimental setting.
The number of samples in ROBIN's confusion matrix in Fig. 7 differs from that in the ablation study.The two experiments used different numbers of random seeds, and the reported predictions were aggregated across all seeds: 10 seeds for ROBIN and 8 for the ablation study.
According to Section 4.1.4, the training, validation, and test sets are split at an 8:1:1 ratio, but the number of test samples in the confusion matrix does not match the expected count.The confusion matrix aggregates predictions across multiple random seeds, rather than reporting those from a single run.

We clarify that these issues resulted solely from writing oversights and differences in the number of random seeds used, and do not affect the validity of the experimental results or the main conclusions. We will correct these errors and provide the necessary clarifications regarding the experimental settings in the revised manuscript.

Q2: The SEED-V unseen-class confusion matrix in Fig. 11 appears to lack Hungarian matching. Was the same evaluation protocol applied to all baseline methods?

Thank you for raising this concern. We clarify that all methods followed the same evaluation protocol, with Hungarian matching applied before evaluating classification performance involving unseen classes. First, since predicted unseen-class labels have no predefined semantic correspondence with ground-truth labels, Hungarian matching is required to establish their optimal alignment. Second, due to differences in class sample sizes, the overall accuracy cannot be obtained by simply averaging per-class accuracies. Finally, we have no incentive to deliberately omit Hungarian matching to report inferior results.

Q3: For the SE datasets, why is the known-class precision consistently 50.00 ± 0.00 across all random seeds when only one known class is present?

Thank you for raising this question. We agree that clarifying the calculation of known-class precision on the SE datasets would improve the manuscript's readability. Both SE datasets contain three classes, with one designated as known and the other two as unseen. When only one known class is present, according to the macro-averaged precision calculation in sklearn, class 0 has 100% precision, whereas class 1, being an empty class, is assigned 0% precision (i.e., 0/0). Their average is therefore 50%. Thus, the reported 50% results from macro-averaging with zero assigned to the empty class. We will provide further clarification in the revised manuscript.

Q4: The unseen-class results on SE-R1510 appear inconsistent with the claim that representation collapse is mitigated.

Thank you for raising this concern. We acknowledge that ROBIN still has limited discriminative capability between the two unseen classes on SE-R1510. This is mainly attributable to the imbalanced distribution of unseen-class samples, with a sample ratio exceeding 3:1. Such a substantial imbalance may exacerbate representation learning bias and impair discrimination between unseen classes. Nevertheless, ROBIN still outperforms the baselines across all datasets. We will acknowledge this limitation in the conclusion to provide a more objective and comprehensive evaluation of ROBIN.

Q5: Why is the average cosine similarity between all class pairs in Fig. 6 lower than −1/(K−1)-1/(K-1)?

Thank you for pointing this out. We clarify that Fig. 6 reports the average cosine similarity between unseen classes, rather than all classes. When only unseen classes are considered, their average pairwise cosine similarity can be lower than −1/(K−1)-1/(K-1). Our work focuses on mitigating representation collapse among unseen classes in open-world settings. Therefore, measuring their average pairwise cosine similarity more directly reflects the effectiveness of ROBIN. In addition, "Rep Mean" in Fig. 6 denotes the "Representation Mean", where the representations used for cosine similarity computation are the ETF logits. We will further clarify the metric definition and computation scope in the revised manuscript to avoid ambiguity.

AuthorsOpenReview ↗

Response to Reviewer taiX [2/2]

Q6: Please report the performance when predictions are made directly without class masking.

Thank you for this suggestion. We confirm that the requested results are already reported in Fig. 4, corresponding to the ablation setting without the binary classification loss Lbin\mathcal{L}_{bin}. In this setting, the binary classification loss is removed, rendering the auxiliary binary classifier ineffective in class discrimination. The model directly predicts based on the ETF classifier's output probabilities, selecting the class with the highest probability as the final prediction. This setting is consistent with the evaluation protocol suggested by the reviewer.

Q7: Please demonstrate that the proposed iterative process improves pseudo-label quality. Also, is the reported accuracy calculated after Hungarian matching?

Thank you for raising these questions. First, all classification metrics involving unseen classes are calculated after Hungarian matching (see Q2). Second, Fig. 9 shows that as training progresses, the losses gradually decrease, while validation performance consistently improves across all-class, known-class, unseen-class, and base-class classification tasks. These trends demonstrate the progressive improvement in pseudo-label quality during training.

Q8: Why were the cited open-world semi-supervised learning methods not included in the comparisons, including ORCA (Cao et al., 2022), NACH (Guo et al., 2022), TRSSL (Rizve et al., 2022), LPS (Ye et al., 2024), and DSCL (Qu et al., 2025)?

Thank you for your concern regarding baseline selection. First, TRSSL was already included in the original comparisons. Second, ORCA does not support fine-grained classification of unseen classes, while NACH does not release its source code, making fair comparisons difficult. Finally, the original manuscript included seven representative baselines covering semi-supervised learning, open-world semi-supervised learning, clustering-based methods, and contrastive learning, including advanced methods from categories relevant to those mentioned above. We believe these baselines adequately demonstrate ROBIN's advantages. Nevertheless, to further address your suggestion, we have added comparisons with LPS and DSCL. The following table reports classification performance on all, known, and unseen classes.

MethodSEED-V(Acc)SEED-V(F1)SEED-VII(Acc)SEED-VII(F1)GoEmotions(Acc)GoEmotions(F1)SE-R1510(Acc)SE-R1510(F1)SE-F1530(Acc)SE-F1530(F1)
LPS39.4537.9633.4534.3223.1710.5561.8434.2926.8420.80
DSCL35.5130.6831.8231.7422.9211.1857.0235.5225.9619.22
ROBIN53.0349.5050.4452.7331.0511.4072.4650.0443.9540.15
MethodSEED-V(Acc)SEED-V(F1)SEED-VII(Acc)SEED-VII(F1)GoEmotions(Acc)GoEmotions(F1)SE-R1510(Acc)SE-R1510(F1)SE-F1530(Acc)SE-F1530(F1)
LPS69.3869.1266.0065.4915.969.8948.4832.6536.1927.69
DSCL72.2771.7869.9069.3818.6113.5757.5836.5435.2426.06
ROBIN89.6789.5577.7477.0720.1815.1663.3338.7841.4529.31
MethodSEED-V(Acc)SEED-V(F1)SEED-VII(Acc)SEED-VII(F1)GoEmotions(Acc)GoEmotions(F1)SE-R1510(Acc)SE-R1510(F1)SE-F1530(Acc)SE-F1530(F1)
LPS22.5722.1623.5323.1329.568.3962.8838.6122.6518.47
DSCL14.7913.8120.2319.4425.668.0156.9736.8021.7917.89
ROBIN32.3830.3234.6533.4235.329.9873.7642.4549.5248.86

Q9: How were FixMatch, CGMatch, and MOOD modified to predict multiple unseen classes?

Thank you for your interest in the baseline implementation details. To ensure fair comparisons, we adopted a unified evaluation protocol for all methods. Specifically, samples with ground-truth labels smaller than KK (i.e., known classes) are evaluated directly by comparing predicted and ground-truth labels. For samples with ground-truth labels greater than or equal to KK (i.e., unseen classes), we first extract their predicted labels and then apply Hungarian matching to establish the optimal correspondence between predicted and ground-truth classes before calculating performance. We will provide these details in the appendix to improve clarity.

Q10: Were the hyperparameters selected based on the validation set?

Thank you for raising this question. Yes, all hyperparameters for each dataset were selected based on the validation set.

AuthorsOpenReview ↗

Follow-up on Rebuttal Responses

Dear Reviewer,

Thank you very much for your continued time and effort in reviewing our manuscript and for engaging in the discussion. We sincerely apologize for following up again. As the rebuttal period is approaching its end, we would like to kindly ask whether our responses have addressed your concerns. We hope that our clarifications have adequately resolved them. We highly value your feedback and would greatly appreciate any further comments you may have.

Thank you again for your time and consideration.

Best regards,
The authors

Thank you for providing the evaluation code, which clarifies how the routing errors discussed in my previous comment remain incorrect in the accuracy calculation. However, the original results and the rerun evaluation are not yet adequately reconciled, and the additional clustering results still provide limited support for the claimed discovery of distinct unseen emotions. Thank you again for your efforts. I will take all of your responses into account in my final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.