acceptodds
Under review as a conference paper at ICLR 2027

Variational EM Energy Models for Weak Multi-Instance Multi-Label Learning

Abstract

Multi-Instance Multi-Label (MIML) learning trains models on bags of instances, where each bag may associate with multiple labels. However, the presence of noisy instances (defined as weak instances) in bags probably leads to mislabeling and missing labels (referred to as weak labels). We summarize the framework of excluding these weak situations and identifying ground-truth labels in MIML as weak multi-instance multi-label (wMIML) learning. Existing methods address either label noise or instance noise in isolation and do not explore the interplay between noisy instances and noisy labels. Methods that jointly handle data noise and label noise are mostly developed for single-instance data such as images and can hardly cover the multi-instance scenario. In this paper, we address wMIML from an energy-based perspective, casting the removal of both noise sources as a principled inference problem. We propose a variational-EM energy model in which each instance carries a latent support assignment over concepts. The energy measures the coherence between instances, their assigned concepts, and the observed bag labels, so that the two noise sources manifest as violations of one shared notion of coherence and are denoised through a single assignment. Variational EM on this model derives an approximate evidence lower bound from first principles, and the constraints induced by this bound couple label denoising and instance selection through the shared support assignment, so that the two tasks inform each other within a single bound. On ten MIML benchmarks under joint instance and label noise, VEM achieves the best F1 in 20 of 30 settings and stays within the top two on 26 of the 30, its only exceptions being the single-concept benchmark (FashionMNIST at flips 0.2 and 0.3) and two ceiling-saturated Scene settings. It consistently exceeds every baseline in the joint-noise regime where the two degradations must be handled together.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-Instance Multi-Label Learning, Weak Multi-Instance Multi-Label Learning, Variational EM, Energy Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6fxdOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposed a variational-EM energy model for weak multi-instance multi-label learning where bag instances can be incorrect while at the same time label also can be incorrect. The proposed method conducts label denoising, instance denoising, and correlation denoising to combat incorrect label and instance assignment during learning.

Critical strengths

(1) This paper provided their clear research position that doubly degraded weak multi-instance multi-label learning is realistic but understudied field.

Critical weaknesses

(1) Experimental setting is both shallow and narrow, and less credible. First, it seems that the authors flipped each label independently according to label flip rate. This setting ignores the label correlation in multi-label setting, so it is far from realistic noise but rather artificial noise. It is doubtful whether this experiment result could be generalized into general case or real case. Second, several baseline methods (e.g., SCE and GCE) modified cross entropy loss thus it is specialied for single-label learning, not multi-label learning. However, implementation details are missing so it is doubtful whether fair comparison was actually made.

(2) This paper failed to prove the efficacy of their proposed method. In Table 1, the authors did not report all 20 baseline methods exhaustively. It is not enough to report just the best case. Also, it could be observed that the proposed method did not show remarkable performance gain compared to best case baseline. What is more, in some datasets, the performance gain of VEM diminishes when label noise rate increases. This results is contradictory since VEM is proposed to combat wMIML scenario, but the authors did not provide any reason nor analysis about this. Furthermore, ablation results in Table 2 and Figure 2 indicate that the components in their proposed method do not leads to performance improvement. Where does the overall performance indeed come from?

Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer 6fxd (1/3)

We thank the reviewer for the detailed reading. We address each point below, stating the definition, the evidence and the scope of the claim.

1. Noise construction

The corruption is not symmetric per-bit flipping. For each bag, rr is a budget over its positive labels: about 40% are deleted and 60% are moved to a concept that no instance in the bag supports. On the instance side we append 0.5 ni0.5\,n_i real instances drawn from the same corpus and delete none of the originals. Labels are corrupted in training only, and evaluation always uses clean test labels; instance distractors are injected on both sides, each from the pool of its own split, so both see the same noise distribution. Appending rather than substituting is deliberate: substitution would remove genuine evidence and confound instance corruption with label ambiguity, whereas appending keeps the evidence and adds only distractors, which is exactly the coexistence of relevant and irrelevant instances that defines wMIML. At r=0.3r=0.3 the realised per-bit error rate is 2.5% to 21.3%, and 56.1% to 100% of bags are affected. Together the two channels produce missing and spurious labels, the two errors annotators actually make. The three levels are not arbitrary: the comparable protocols we checked use 10 to 40% (Bai et al., IJCAI'20) and 30 to 50% (Xia et al., ICCV'23), and our 0.2/0.3/0.50.2/0.3/0.5 sits inside that range. We stress that this is controlled synthesis, not field-collected noise: its value lies in matching the error types of real annotation and in applying one identical corruption to every method, not in reproducing the noise of a particular deployment.

2. Label correlation

We quantified it. Apart from emotions, whose mean absolute ϕ\phi is 0.125, the other nine benchmarks stay at or below 0.059, and strongly correlated pairs account for at most 1.4%. At that magnitude, per-bit independent sampling and correlation-aware sampling give nearly indistinguishable label distributions. We did try generating correlated noise from the co-occurrence distribution, but co-occurrence in these data is itself weak, so the resulting noise differed little from independent flipping and does not amount to a test of meaningful strength; we therefore report it as a boundary rather than claim coverage. One clarification: we never claimed correlation denoising. The factorized posterior does not model label correlation, as stated in the Limitations; our target is joint denoising on the instance and label sides.

3. Generality

On scale first: ten benchmarks (5 to 374 classes, 800 to 14915 bags), three noise levels and 20 baselines, 30 settings in total, and five metrics (micro-F1, MAP, Hamming loss, one-error, macro-F1). This is a controlled empirical study. A fair comparison requires controlling how noise enters, so that the same construction applies to every method; missing and spurious labels are the two errors annotators actually make, which also makes more prior work comparable. We do include real data. OpenMIC's labels come from the original files, where unsure entries were counted as positive; its 94% coverage (18.8 of 20 labels per bag) is itself real annotation noise, and at that density the all-positive prediction already reaches 0.9696, so the cell offers no room. CAL500 has 8.9 positive labels per instance and near-saturated support. For the same reason, verifying denoising on real noise requires manually verified clean test labels; otherwise one cannot separate gains due to denoising from gains due to fitting the test noise, and such a benchmark does not exist in MIML. Hence, for a fair comparison, we did not add more real datasets.

AuthorsOpenReview ↗

Response to Reviewer 6fxd (2/3)

1. SCE/GCE

Both were designed for single-label classification. We use them on per-label sigmoid binary tasks: GCE or SCE is computed independently on each of the CC label positions (GCE additionally weights positives by posweight\mathrm{pos_weight}), never as a CC-way softmax, and the losses are averaged over positions; the other baselines we use per position (CUE, BML, JAL) follow the same pattern. Concretely, on position cc the logit gives GCE ℓ=(1−pq)/q\ell=(1-p^{q})/q with q=0.7q=0.7 and posweight\mathrm{pos_weight} on the positive side, and SCE ℓ=CE+α SCEsym\ell=\mathrm{CE}+\alpha\,\mathrm{SCE}_{\mathrm{sym}} with α=0.1\alpha=0.1, averaged over the CC positions. The revision will list both formulas and their weight settings in the appendix.

2. Fairness of the comparison

Fairness also depends on the tuning budget. We gave the baselines extra budget: four (lr, hidden width) grid points for GCE and MaxPool each, selected on validation. Taking the strongest baseline per dataset, three of the five datasets are no better than the original value, and the average gain is only 0.46 points, so lr=5×10−4\mathrm{lr}=5\times 10^{-4} is already close to their optimum. After tuning, VEM still leads by 2.5 to 3.0 points on bibtex, CAL500 and LetterCarroll, while on Reuters and enron the lead narrows from +2.4/+1.7+2.4/+1.7 to +0.7/+0.4+0.7/+0.4. The main-table lead therefore does not come from under-tuned baselines, but we report honestly that these two cells are tuning-sensitive. One asymmetry we state ourselves: per-dataset configuration gives VEM +1.71+1.71 points (Appendix Table 6), larger than the +0.46+0.46 for the baselines; the gain concentrates in the five datasets with a structural mismatch, and although each direction comes from the diagnosis in advance, VEM is indeed tuned one level more than the baselines, and we do not report only the half that favours us. At the same time, with no per-dataset selection at all, the unified configuration already beats the best baseline in 8 of 30 settings and wins 208 of 360 pairwise comparisons against the 12 deep baselines; per-dataset selection raises this to 20/30 and 314/360, and each intervention follows a structural statistic (a deeper encoder for sparse features, a lower Hill weight when the support is uninformative, a rebalanced positive weight under saturation) rather than a grid search.

3. Table 1

For space, the main text lists only the strongest baseline per setting; the complete 20-method by 30-setting matrix is in Appendix C, and each cell of the main table also names that baseline (Reuters: MaxPool, MaxPool, Hill across the three levels), so the change of winner is readable from the main table itself. The strongest baseline is not one method: 23 cells without ties are shared by six methods (GCE 9, MaxPool 7, MLDN 3, MIMLVAE 2, Hill and KISAR 1 each), and the other 7 cells have several methods tied for the maximum (13 to 14 at each OpenMIC level, 2 to 3 at each Scene level, 12 at emotions 0.5); counting ties in full, the optimal set across 30 cells involves 15 methods. The winner changes with the level on three benchmarks (Reuters from MaxPool to Hill, bibtex from MaxPool to KISAR, FashionMNIST from MIMLVAE to MLDN), and Scene is tied at all three levels. We therefore compare against the per-cell best baseline rather than requiring a clean sweep.

4. Size of the gain

Under a common 0.5-point tolerance (the 20/30 in the paper counts Δ>0\Delta>0; here the same tolerance is applied to the original model and to the concept-head version), the original model wins 14 cells, ties 12 and loses 4, with a sign test p=0.031p=0.031. On the three benchmarks that keep their lead after fair tuning (bibtex, CAL500, LetterCarroll) the lead at r=0.3r=0.3 is 2.5 to 3.0 points. Adding the concept-level posterior branch of 4.7 raises the mean margin over the strongest baseline from +0.24/+0.84/+0.99+0.24/+0.84/+0.99 to +1.34/+1.69/+2.46+1.34/+1.69/+2.46, with cell counts 17/9/4 against the best baseline (13/14/3 against the original VEM) and p=0.0072p=0.0072; the pooled mean rises from +0.69+0.69 to +1.83+1.83 points. These are not single runs: Appendix Table 8 gives three-seed means and standard deviations for all 30 main-table settings (maximum sd=0.011\mathrm{sd}=0.011), and the concept head uses two seeds per level (Appendix G), with a median per-cell sd\mathrm{sd} of 0.003 and a maximum of 0.011 at level 0.3; the conclusions at the level of means are unchanged. The branch is reported separately as an extension and does not replace the main table, so that the 30-cell comparison and the ablation anchor stay intact; if you consider it should become the main method, we will replace the main table and re-run the dependent analyses in the final version.

AuthorsOpenReview ↗

Response to Reviewer 6fxd (3/3)

1. Gain versus noise level

The observation that the gain diminishes as the noise level rises holds on three benchmarks, and we do not avoid it: LetterCarroll falls from +2.7+2.7 to +0.6+0.6, enron from +1.7+1.7 to +0.5+0.5, and Corel5K from +0.7+0.7 to −2.7-2.7; bibtex holds only partially, from +3.2+3.2 to +2.1+2.1 and then flat. This is not a contradiction: the claim is superiority over the per-cell best baseline at the same noise level, not that the margin grows monotonically with noise. Stacking the ten curves hides three other regimes: OpenMIC has 13 to 14 methods tied at every level and emotions differs by less than 0.05 points, so these two are ceilings where zero gain reflects no room rather than decay; Reuters (+0.06+0.06 to +5.75+5.75) and CAL500 (+1.09+1.09 to +3.92+3.92) move the other way, with the margin widening; FashionMNIST lags throughout, from −4.3-4.3 to −0.0-0.0, which is a structural mismatch. From 0.3 to 0.5, on LetterCarroll VEM goes from 0.558 to 0.397 and the best baseline from 0.531 to 0.391; on enron 0.641 to 0.599 against 0.624 to 0.594; on bibtex both drop by 0.097; on Corel5K VEM drops from 0.392 to 0.360 while the baseline rises slightly from 0.385 to 0.387. The baselines do not become stronger; VEM degrades slightly faster. We have no mechanistic explanation for the faster degradation and report it as it stands. The revision states the coverage condition in 3.4 and separates the two boundary cases, support saturation (CAL500) and single-concept structure (FashionMNIST), in 4.4.

2. Ablation

Apart from the full VEM, Table 2 of the original had only three rows (single-energy posterior, no Hill, plus VEM4): the single-energy and no-Hill rows stay within ±0.003\pm 0.003 and VEM4 averages −0.46%-0.46\%. All three were measured with the structure already present, so near-zero effects do not show that the components carry the gain, nor do they answer where the performance comes from: the structure itself was never ablated. We ran that ablation over 30 settings: replacing energy pooling with mean pooling lowers mean F1 from 0.6713 to 0.6416, below the mean of the per-cell best baselines (0.6644), cuts winning cells (Δ>0\Delta>0) from 20/30 to 10/30 and raises cells lost by more than half a point from 4 to 12. The revision reports the three layers separately: structure −2.97-2.97 points (30 cells), support-aware reweighting −0.42-0.42 (9 cells), contrastive term +0.04+0.04 (30 cells). The loss terms have small marginal effects because they are not independent sources of gain: the support term reweights the negative half of reconstruction, the contrastive term estimates the KL already in the bound, and removing one leaves the others approximating the same quantity. Figure 2 is the per-dataset view of the single-energy row, where the dual-energy and single-energy posteriors differ by ±0.2\pm 0.2 points (five benchmarks positive, one neutral, four slightly negative); the main text has always treated it as a secondary effect relative to reconstruction.

3. Where the performance comes from

From the energy pooling structure, whose contribution rises monotonically with the label space: +1.87+1.87 points for C≤26C\le 26, +3.50+3.50 for 27≤C≤9027\le C\le 90 and +4.45+4.45 for C≥159C\ge 159, and in each bin the version without the structure falls below the best baseline. The locus of efficacy is therefore the structure, not the marginal terms. We also acknowledge two limits: FashionMNIST remains unsolved, and the revision does not improve it (three levels at −5.1/−3.7/−0.6-5.1/-3.7/-0.6, two of them worse than the original); suppressing the max-pooling branch recovers only about 1 point of the 4.3-point gap. And leading on most benchmarks is not a mechanism-level proof: what we provide is that the structure carries the main contribution and that the label-level criterion is identifiable, not a complete causal chain.

4. Presentation

We agree that the paper was hard to follow. The revision adds a Remark in 3.3 separating the modelling choice from the objective, moves the applicability conditions from the analysis into the method section (3.4), rewrites the ablation in 4.5 as a margin attribution and moves the layered and binned tables to the appendix, and adds the realised noise statistics (Appendix F) together with the structural discussion in 4.4.

5. Reproducibility

Every number above can be reproduced; the scripts and result tables will be released with the revision.

Reviewer_dQdEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies weak multi-instance multi-label (wMIML) learning, where both instances and labels may contain noise. The authors propose a variational EM energy model (VEM) to handle these two types of noise jointly. The method uses a shared latent assignment to connect instances with their corresponding labels. It combines label denoising, instance selection, and contrastive learning within a unified framework. Experiments are conducted on ten datasets under different noise levels. The proposed method achieves competitive results against 20 baseline methods and obtains the best F1-score in 20 of 30 settings in the main experiments. The authors also provide ablation studies and discuss the limitations of the method on certain datasets.

Critical strengths
  1. Addressing both instance noise and label noise in multi-instance multi-label learning is meaningful, as these two problems often occur together in real-world data.

  2. A unified variational EM framework is proposed to connect instance selection and label denoising. The theoretical analysis provides support for the method design.

Critical weaknesses
  • The factorized variational posterior �0��0� used in the E‑step is a manually designed heuristic, not the optimal solution derived from maximizing the ELBO. This important note is missing from Section 3.3. Readers could easily mistake this setup for a standard, optimal variational‑EM derivation. Additionally, the training objective �1��1� is not the standard negative ELBO. It includes an extra �2��2� gap term. This key mathematical detail is never stated in the main text, which may mislead readers about what the model is actually optimizing.
  • It cannot clearly tell how much the contrastive term �3��3� contributes on its own. The main‑text ablation table does not include a test where this component’s weight is set to zero.
  • In Section 4.4, the authors offer a plausible explanation for why bags with single‑concept instances work poorly with the fine‑grained support assignment mechanism. That said, their reasoning comes mostly from high‑level dataset structure analysis. They provide no targeted experiments or supporting evidence to directly verify why the model fails in this scenario.
  • The authors claim that the use of latent variables to support the realization of instance selection and denoising. However, the existing experiments mainly evaluate the multi-label prediction performance at the data packet level and do not directly report whether the instance selection is accurate. This point is particularly noteworthy because the noisy instances in the experiments were injected manually, and the authors could theoretically know which instances were replaced. Therefore, it can be considered to add: Precision, Recall, and F1 for the identification of noisy instances.
Questions for authors
  1. Could you explicitly restate the necessary dataset prerequisites for VEM to work well in either introduction or method section, rather than only revealing them after presenting experimental results?

  2. Why is there no main-text ablation for the contrastive loss term? Could you add this ablation and clarify its empirical contribution?

  3. Are there any intuitive algorithmic adjustments you can propose for the two failure regimes, single-concept-instance bags and support-saturated datasets, instead of purely relying on dataset-specific hyper-parameter tuning?

  4. Could you explain that why one-error remains poor for VEM, and what practical fixes could improve top-1 label prediction?

3 comments
AuthorsOpenReview ↗

Response to Reviewer dQdE (1/3)

We thank the reviewer for the detailed assessment. The comments concern the theoretical target, the ablation design, the failure mechanism and the instance-level evaluation; we address them in turn, with the revision, the added evidence and the scope of each claim.

1. Posterior family

The factorized form q(z)=∏j,cqjczjc(1−qjc)1−zjcq(z)=\prod_{j,c} q_{jc}^{z_{jc}}(1-q_{jc})^{1-z_{jc}} is a modelling choice that expresses instance-concept support and avoids an inner sampling loop; it is not the optimum of coordinate ascent on the ELBO. The exact optimum is a different structured family, and Appendix A.5 (E-step Posterior and Dual Energy) states it together with the evidence it must satisfy. We chose this family because it is closed-form, interpretable and needs no inner sampling. Section 3.3 did not make this layer explicit, so a reader could take it for a standard derivation; the revision adds a Remark there that separates the modelling choice from the objective.

2. Training objective

From log⁡p(Y~∣X)=ELBO(q)+KL(q ∥ p)\log p(\tilde{Y}\mid X)=\mathrm{ELBO}(q)+\mathrm{KL}\big(q\,\|\,p\big) it follows that the full objective is a KL-regularized negative bound, differing from −log⁡p(Y~∣X)-\log p(\tilde{Y}\mid X) by exactly 2 KL2\,\mathrm{KL}; the negative ELBO alone corresponds to the reconstruction part only. The identification results therefore concern the bound component that each estimator targets, and do not imply that the chosen family is optimal. Among the three estimators, (c) the contrastive term estimates precisely this KL, which is why Section 4.5 reads it as the posterior gap rather than as a source of gain alongside reconstruction. In the original submission this point appeared only in the appendix; the revision moves it into the Remark of Section 3.3.

3. Contrastive term

Apart from the full model, Table 2 of the original had only three rows (single-energy posterior, no Hill, and the model with VEM4); there was indeed no test with the contrastive weight set to zero. That was an omission in presentation, not missing data.

With one fixed configuration and only the contrastive term switched off, we re-ran all 30 settings: mean +0.04+0.04 points, median −0.01-0.01. Grouped by label space, the 15 cells with C≤26C\le 26 average −0.24-0.24 (removal hurts, so the term contributes there), the 12 cells with C≥90C\ge 90 average +0.43+0.43 (removal helps, so the constraint is too strong there), and enron in between sits at −0.10-0.10, a continuum rather than a cliff. Across the 30 cells, 12 improve, 15 degrade and 3 are unchanged, so the two directions cancel in the mean.

Re-running with weight 0.030.03 for C≤26C\le 26, 00 for C≥90C\ge 90 and 0.010.01 otherwise gives a mean of +0.21+0.21 over the 30 cells (16 wins, 8 ties, 6 losses), with the gain almost entirely from the off group (12 cells, +0.43+0.43, 10 wins, 0 ties, 2 losses); the small-space group gains only +0.07+0.07, inside seed noise. A sign test on the off group gives p=0.039p=0.039 (10 wins, 2 losses), while the small-space group at 6/5/4 is far from significant, so we claim a stable gain for the large-label-space group only, and not for the rule as a whole. The threshold is read off the gap in class counts and carries no optimality claim.

We have revised the description accordingly: the contrastive term is a sensitivity factor that depends on the label space, not a main source of performance. The revision states this ablation in Section 4.5 and gives the full layered table in Appendix B.6.

AuthorsOpenReview ↗

Response to Reviewer dQdE (2/3)

1. Section 4.4: what we can and cannot claim

The original explanation rested on bag structure and label-density statistics, without an intervention that manipulates role independently. We added four: varying the number of concepts per bag, stratifying the real data by instance purity, replacing a fraction of instances by nearest-class instances to change ambiguity, and shrinking instances toward the bag mean to change homogeneity. The readout is the F1 change when the latent support is removed, so a larger negative value means the structure matters more. The first two show no stable monotone relation; the ambiguity dose falls from −8.8-8.8 and settles at −3.9,−5.0,−4.1,−4.1-3.9,-5.0,-4.1,-4.1; the homogenisation dose runs −7.8,−10.2,−13.8,−12.6,+0.4-7.8,-10.2,-13.8,-12.6,+0.4 over three seeds (sd≤1.5\mathrm{sd}\le 1.5).

At full homogeneity the three pooling operators coincide and the contribution vanishes, confirming the dose acts where it should. Before full homogeneity, however, more similar instances make the latent support more important, not less. This does not support the reading that feature similarity makes the structure useless, and it rules out a causal claim in the original direction: the two must be kept apart. The revision therefore frames 4.4 as a structural applicability boundary rather than a verified causal explanation. No benchmark in our suite carries instance-level role labels, so role cannot be manipulated independently of the feature distribution; the four interventions above therefore vary only correlates of role. A genuine test needs MIL benchmarks with role ground truth (MUSK, Fox, Tiger), where features are fixed and only role changes; they are not among our ten benchmarks.

2. Instance-level oracle (P/R/F1)

Under the add-only, same-corpus protocol, injected and genuine instances are exchangeable, so instance-level membership has no stable ground truth. An oracle classifier supervised with the injection mask (three feature representations by four model families) gives AUC 0.487 to 0.528 against chance 0.50; P/R/F1 at different thresholds mostly reflect the threshold position and cannot compare selection ability.

3. Moving the criterion to the label level

A relocated false positive has no instance support, and this asymmetry makes the same criterion measurable at the label level: probes trained on noisy labels only reach 0.71 to 0.80 on the three low-coverage benchmarks (bibtex, Reuters, LetterCarroll), against a clean-label ceiling of 0.83 on the same data; across the ten benchmarks the noisy-label probes span 0.40 to 0.86 (FashionMNIST highest at 0.86, OpenMIC lowest at 0.40). The criterion is not monotone in the final gain: FashionMNIST has 0.86 and VEM still trails the best baseline by 3.9 points there, while CAL500 has 0.49 (clean ceiling 0.51) and leads by 2.5 points. Label-level identifiability is thus necessary but not sufficient.

4. Concept-level extension (Section 4.7)

We add a concept-level posterior head,

\qquad q_c=\sigma\!\left(\beta_q(g_c-b_c)+\log \mathrm{LR}_c\right),$$ trained with $\lambda_g\,\mathrm{BCE}(g,\tilde{Y})+\lambda_q\,\mathrm{KL}\!\left(q\,\|\,\mathrm{Bern}(\sigma(b))\right)$, where $\lambda_g=0.5$ and $\lambda_q=0.05$ are fixed across the ten benchmarks and the three noise levels. F1 still comes from the VEM head; the new head affects the backbone only through the shared encoder. Relative to VEM the gain is $+1.10/+0.84/+1.46$ points over the three levels, and the mean margin over the strongest baseline rises to $+1.34/+1.69/+2.46$; it concentrates on bibtex and LetterCarroll (coverage 7% and 18%), with the other eight benchmarks essentially flat, consistent with the applicability condition in 3.4. Training, early stopping and inference use noisy labels only; clean labels only for evaluation. Put differently, the concept-level $q_c$ is what gains discriminative power: on bibtex, Reuters and LetterCarroll the learnable prototypes give AUC 0.756 to 0.934 (the revision reports a median of 0.82 over the ten benchmarks, and 0.51 on the near-saturated OpenMIC), whereas the instance-level $q$ remains unidentifiable, as the oracle shows. ### 5. Mechanism and positioning It is not an ensemble: F1 comes from the VEM head alone. Cutting the gradient of the auxiliary loss into the encoder leaves the three levels bit-identical to plain VEM (0.5443/0.2527/0.6683; these are below the main table's 0.5581/0.2610/0.6765 because the wrapper's extra parameters shift initialization, but all three conditions share one initialization, so their comparison is valid). The gain therefore travels only through the shared encoder. The branch is an extension and does not replace the main table; if you consider it should be the main method, we will replace it and re-run the dependent analyses.
AuthorsOpenReview ↗

Response to Reviewer dQdE (3/3)

1. Dataset prerequisites

As you suggest, the revision states two directly computable conditions in Section 3.4, before the experiments: the label space must be large enough for instances to separate in role (the gain vanishes at five classes on Scene and is stable from 26 classes upward), and the fraction of the label space covered by a bag must not approach one (no allocation is possible at OpenMIC 94%, Scene 82% and emotions 80%, whereas coverage of 7% on bibtex and Reuters and 18% on LetterCarroll gives the largest gains). Ten benchmarks support these two empirical boundaries but do not pin down the exact thresholds, so we state them as computable statistics rather than a formal treatment.

2. Single-concept bags

Max pooling is easily dominated by one distractor instance here, and the oracle version of the proposed adjustment is to switch that branch off: with wmax⁡w_{\max} going from 0.50.5 to 00, FashionMNIST improves by 0.94 and 0.89 points at r=0.2r=0.2 and 0.30.3 (three seeds), while on the normal benchmarks bibtex moves +0.23+0.23 and LetterCarroll −0.20-0.20, both inside seed noise. The original configuration uses wmax⁡=0.5w_{\max}=0.5 on all ten benchmarks; setting it to zero is a single global change, not a per-dataset choice. It recovers only about 1 point of the 4.3-point gap, however, and the remainder is not something a pooling weight can fix, so we do not claim this regime is solved.

3. Support-saturated data

The intervention already in place is a rebalanced positive weight in the ELBO, which follows directly from the structural diagnosis in 4.4 (near-uniform support and a Hill weight that loses discriminative value) rather than from added capacity; the lead over GCE is +1.1/+2.5/+3.9+1.1/+2.5/+3.9 points at the three levels. To be explicit, this is still a per-dataset weight rather than a transferable rule; the cleaner option, adaptively switching off the Hill reweighting according to the variance of pc=max⁡jqjcp_c=\max_j q_{jc}, is not implemented in this round, so we do not claim a general solution for saturation. We note in addition that the concept head of 4.7 shares one weight pair (λg=0.5\lambda_g=0.5, λq=0.05\lambda_q=0.05) across all ten benchmarks and is not tuned per dataset, with positive mean gains at all three levels.

4. one-error

Uniform-random one-error is about 1−L/C1-L/C. For bibtex, 0.390 corresponds to a random level of about 0.930; OpenMIC has 18.8 of 20 labels positive per bag, so the random level is itself 0.06, and the observed one-error ranges from 0.061 to 0.137, carrying almost no discriminative information there. The errors concentrate on the top-1 versus top-2 boundary: the best true label already sits in the top two for 90.5% of bags on LetterCarroll and 77.7% on bibtex, with MRR 0.824 and 0.741. What is missing is not the ability to find relevant labels but the discrimination of the last position.

Prior correction does not help (LetterCarroll 0.3100 to 0.3100, bibtex 0.3897 to 0.3920), and setting wmax⁡w_{\max} to zero worsens LetterCarroll to 0.3250 and leaves bibtex unchanged while raising bibtex F1 by 1.3 points, which shows that F1 and top-1 act at different positions. The reason is that the frequent classes absorbing top-1 are also the true labels of many bags, so suppressing them removes correct cases as well.

The direction that could work is a listwise or top-1 ranking objective aimed at this error structure, but it would conflict with the threshold optimization behind micro-F1 and would require re-running 30 settings and retuning the weights. In this paper one-error is a diagnostic metric carried over from multi-label practice, not the main optimization target, so we list it as future work; if you consider it should be an evaluation target, we are glad to continue the discussion and add experiments.

5. Reproducibility

The scripts and result tables behind every number above will be released with the revision; we can also provide the result files and run commands during the discussion period.

Reviewer_gAQuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the problem of weak MIML (wMIML), where bags contain both weak instances and weak labels, and the two are coupled since noisy instances are what mislead annotators. To learn under this dual noise, the authors propose VEM, a variational-EM energy model with a single latent support assignment recording which instance supports which concept, so instance and label denoising become two views of the same variable. The resulting ELBO splits into three estimators covering label reconstruction, support-based reweighting, and a contrastive term. The method works well in practice, outperforming majority of baselines across several benchmarks.

Critical strengths
  • The wMIML setting is under-explored. Noisy bag labels and degraded instances have each been studied on their own, but rarely together in the multi-label case.
  • Particularly, the variational EM approach is an interesting way to tackle the dual noise. Both degradations are tied to one latent variable that records which instance supports which concept, so label denoising and instance selection become the same operation instead of two separate fixes.
Critical weaknesses
  • VEM does energy-based EM optimization and is likely more expensive than the baselines, but no runtime or memory comparison is reported.
  • The mean-field posterior assumes instance-concept independence, while labels in these benchmarks can be heavily correlated.
  • In the controlled experiments, label flips and instance injection are applied independently, so the data lacks the instance-label coupling the method is built for.
Questions for authors
  • How much does the mean-field factorization cost you? Would it be possible to model label correlations in the posterior, and how would you do it?
  • Did you measure computational cost against the baselines?
3 comments
AuthorsOpenReview ↗

Response to Reviewer gAQu (1/3)

We thank you for your support of this work and for your constructive comments, which have provided valuable guidance for improving the manuscript. We address each comment below, describe the corresponding revisions, and would be pleased to continue the discussion.

We appreciate your recognition of the wMIML problem formulation and the shared latent support assignment.

W1/Q2: Runtime and GPU memory

Using the same RTX 4090 and a batch size of 64, we measured the median per-step runtime over 100 training steps, the peak GPU memory over an epoch, and the number of trainable parameters. The selected datasets cover different extremes in label-space size and bag size: Scene (5 classes, small bags), LetterCarroll (26 classes), and bibtex (159 classes, large bags). The measurements show that VEM has a higher time cost than the compared baselines, while its memory consumption remains on the same order of magnitude.

DatasetPer-step runtime, VEM/GCE/MaxPoolPeak GPU memory, VEM/GCE/MaxPoolTrainable parameters, VEM/GCE/MaxPool
Scene3/2/2 ms34/33/33 MB398/396/396 K
LetterCarroll3/2/1 ms38/30/22 MB76/69/69 K
bibtex8/3/2 ms51/46/46 MB1154/1114/1114 K

Relative to the compared baselines, VEM requires approximately 1.5–4 times the per-step runtime. This overhead primarily results from the additional E-step, which performs another forward pass over each bag, computes inner products with the CC class prototypes to obtain qq, and returns qq to the loss. The absolute cost of the E-step is determined by the number of instances in a bag; it therefore does not grow with the complexity of the method. This also explains why the ratio is relatively high on bibtex, which has a large label space, and lower on Scene and LetterCarroll, which have smaller bags. By contrast, the backward cost of the baselines depends on the numbers of bags and classes, so the ratio varies with the data configuration.

Peak GPU memory increases by at most approximately 11%, indicating that the spatial cost does not increase by a multiplicative factor. The E-step produces only a bag-by-class intermediate quantity; on bibtex, for example, this is a 25×15925\times159 matrix occupying tens of kilobytes. It does not retain the computation graph, require second-order optimization, or introduce sampling loops. Memory usage is therefore dominated by the encoder and batch size, as in the baselines. The 16 MB increase on LetterCarroll is due to the teacher copy in the dual-energy architecture and is an architectural choice rather than an E-step cost.

The number of trainable parameters increases by 3.6%, 10.1%, and 0.5% on bibtex, LetterCarroll, and Scene, respectively. The 10.1% increase on LetterCarroll reflects its small absolute parameter count, for which the prototype bank of fixed size accounts for a larger fraction; the same component accounts for only 3.6% on bibtex.

We have not reported end-to-end wall-clock time normalized by the number of early-stopping epochs. This is because the optimal epoch counts differ between VEM and the baselines, and VEM typically stops earlier. To report this quantity without obscuring the underlying settings, we have not used a normalization that may be more favorable to our method. If the reviewer would like end-to-end results, we can add a table normalized by each method's optimal epoch count in a subsequent revision.

W2/Q1: Mean-field factorization and training objective

This issue merits careful examination. Section 3.3 of the revision explicitly states that the factorized posterior $ q(z)=\prod_{j,c}q_{jc}^{z_{jc}}(1-q_{jc})^{1-z_{jc}} $ is a modeling choice rather than the exact optimum of the evidence lower bound. The training objective is $ \mathcal L=-\mathrm{ELBO}(q)+\mathrm{KL}(q\Vert p) =-\log p(\tilde Y\mid X)+2,\mathrm{KL}(q\Vert p). $

The method therefore optimizes a KL-regularized negative bound rather than the standard negative ELBO. The theoretical results identify the objective targeted by each estimator; they do not establish optimality of the chosen variational family.

AuthorsOpenReview ↗

Response to Reviewer gAQu (2/3)

Response to Reviewer 1 (2/3)

W2/Q1 (continued): Label correlation

We further examined label dependence on each benchmark. Given the training label matrix YY with rows indexed by bags and columns indexed by classes, we computed the ϕ\phi coefficient, i.e., the Pearson correlation between two binary label variables, for every pair (c,c′)(c,c') and averaged its absolute value. A label pair is considered highly correlated when ∣ϕ∣>0.3|\phi|>0.3.

BenchmarkClassesMean absolute ϕ\phiHighly correlated pairs
emotions60.1256.7%
CAL5001740.0591.4%
Scene50.0520.0%
enron530.0340.4%
LetterCarroll260.0330.0%
Reuters900.0301.0%
OpenMIC200.0260.5%
bibtex1590.0220.6%
FashionMNIST100.0220.0%
Corel5K3740.0150.2%

Across the ten benchmarks, apart from emotions (mean absolute ϕ=0.125\phi=0.125), the mean absolute ϕ\phi does not exceed 0.059. This is also consistent with the absence of a clear gain from explicit co-occurrence modeling on these benchmarks. The third-order interaction test is constrained by estimability: the proportions of estimable label triplets are approximately 1.0%, 3.6%, and 10.5% on Reuters, Corel5K, and bibtex, respectively. Within the present data, the measurable pairwise label dependence is sufficiently weak that it does not substantively affect the independent posterior, while stronger higher-order dependence cannot be estimated at the available sample sizes. Subject to these data limitations, we conclude that label correlation is weak within the measurable range; we do not claim that higher-order dependence is absent in general data.

If label correlation were modeled explicitly, one option is a pairwise Markov random field along the label axis:

q(z)∝∏j,cϕ(zjc)∏c<c′ψ(zjc,zjc′).q(z)\propto\prod_{j,c}\phi(z_{jc})\prod_{c<c'}\psi(z_{jc},z_{jc'}).

Each mean-field sweep would require approximately O(C2)O(C^2) work. A low-rank factorization can reduce this cost to O(Cr)O(Cr), but it may fit noise when the dependencies are weak. These structured variants are not implemented in the present work and will be stated as limitations in the revision.

W3: Instance–label coupling

The comment is valuable in distinguishing two distinct levels: the simultaneous presence of both noise sources and the causal induction of label errors by instance content. This distinction has helped us refine the work further. In the original protocol, the label side removes or migrates positive labels according to a budget rr, while the instance side independently appends instances drawn from the same corpus at half the original bag size. Although the two channels jointly produce the wMIML data, label corruption is not caused by any particular injected instance. The original protocol can therefore evaluate robustness to jointly applied noise, but it cannot by itself establish the causal mechanism in which noisy instances mislead annotators.

AuthorsOpenReview ↗

Response to Reviewer gAQu (3/3)

Response to Reviewer 1 (3/3)

W3 (continued): Source-coupled protocol and results

To align the protocol with the stated motivation, we kept the original experimental environment unchanged and varied only the source of the injected instances. For a spurious positive label cc, the corresponding distractor instances were drawn from bags containing class cc, so that they provide visible support for the incorrect label. We also evaluated a pure instance-noise protocol, in which labels remain clean and only distractor instances are added. The three protocols therefore correspond to jointly applied but independent two-channel noise, source-coupled noise, and pure instance noise.

At r=0.3r=0.3, the margin of VEM over the strongest baseline on bibtex increases from +2.1 to +5.3 F1 points under the source-coupled protocol, and is +4.4 points under pure instance noise. The corresponding F1 decreases after removing the latent support are 17.1, 18.1, and 20.9 points. On LetterCarroll, these decreases are 8.8, 9.1, and 10.6 points, while the margin over the strongest baseline decreases from +2.7 to +2.3 points; we retain this reversal in the reported results. Overall, the latent support structure becomes more important as the protocol more closely matches the premise that instances provide evidence for incorrect labels.

We also audited whether the supplementary protocol leaks label information. For the target bags, the proportion of injected instances drawn from classes in the bag's true-label set was 0%. Moreover, support for incorrect labels increased by only approximately 0.002, indicating that the coupling remains weak. The source-coupled protocol should therefore be regarded as a diagnostic of mechanism consistency, rather than a complete reproduction of the real annotation process.

Revisions and conclusion

In response to these comments, we have revised the manuscript to clarify the roles of the mean-field family and the KL-regularized objective; refer to the main experimental protocol as a jointly applied two-channel controlled corruption, reserving “source-coupled” for the supplementary protocol; include the runtime, label-correlation, and source-coupling results; and state three limitations explicitly: the additional time cost, the unmodeled label correlation, and the incomplete reproduction of causal coupling. We thank you again for your rigorous comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.