- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes COIN, a deep representation-learning method for unbalanced incomplete multi-view clustering, where views are observed at substantially different rates. The method aims to reduce two effects of missingness: changes in a sample’s representation when an observed view is removed, and correlation between the learned representation and observation patterns.
COIN learns a content factor from mask-gated view features and an observation factor from the missingness mask, with reconstruction objectives supporting representation learning. It introduces two regularizers. Nested content invariance aligns the normalized content representations before and after removing one observed view, with sparsely observed views more likely to be removed. Rebalanced decorrelation penalizes linear content–mask cross-covariance using truncated inverse-frequency weights to increase the contribution of rare observation patterns. Final clustering applies K-means to the normalized content factors. The theoretical analysis shows that nested invariance and zero content–mask covariance are distinct properties and clarifies the relationship between these criteria and the implemented losses.
The authors evaluate COIN against twelve baselines on five benchmarks, using one synthetically generated, feature- and label-independent unbalanced mask per dataset at target mean missing rates from 0.1 to 0.5. They report the best ACC, NMI, and ARI in all 75 dataset–rate–metric cells. Supporting experiments include ablations of the two regularizers, hyperparameter sensitivity analyses, and diagnostics measuring representation drift under view removal and residual predictability of observation masks.
Critical strengths
-
The paper distinguishes two properties: the stability of a sample's content representation when an observed view is removed, and the linear correlation between content and observation patterns under a rebalanced distribution. Proposition 1, while elementary, shows that neither property implies the other, which gives a clear rationale for addressing both. COIN translates this distinction into explicit representation objectives, with sampling and weighting that account for unequal view availability and pattern frequencies. Masking consistency has precedents such as CIMLN, but this joint formulation is still a useful conceptual contribution: it specifies which aspects of observation robustness the method targets and makes those claims measurable.
-
Beyond clustering accuracy, the appendix reports representation drift, content–mask covariance, and frozen linear mask-probe results across ablation variants at two missing rates. These ablations provide concrete evidence for the targeted stability effect. The authors also report substantial residual mask predictability in the full model and explicitly distinguish decorrelation from independence. These diagnostics connect the method to observable representation behavior and make the scope of the empirical support clear.
Critical weaknesses
-
Appendix C.5 states that six loss coefficients are selected independently for each dataset-missing-rate combination by maximizing ground-truth ACC, followed by NMI and ARI. Although labels do not enter the training loss, they directly determine which models are reported, including the ablation results in Table 3. Meanwhile, BURG and CIMLN use their official recommended hyperparameters, and an equivalent selection budget is not established for any baseline. The comparison therefore does not clearly separate the benefit of the proposed method from a tuning advantage. Moreover, each dataset–rate combination uses one fixed mask, and averaging over ten kmeans initializations does not quantify variability across mask realizations or representation-training runs. Several baseline results also behave erratically across missing rates (e.g., BURG on MSRCV1 drops from 87.2 to 25.3 ACC, and CIMLN on HandWritten rises from 79.6 to 90.9 between r = 0.1 and 0.2), suggesting run-to-run instability that may inflate the reported margins.
-
Although the diagnostics confirm reduced representation drift and content–mask covariance, they provide limited evidence that these improvements translate into reduced reliance on observation patterns while preserving complementary view information. At 50% mean missingness, the full model’s mask-probe AUC is 0.963 on Washington, compared with 0.678 for reconstruction alone, and remains 1.000 on Synthetic3d. The probes use raw content, whereas the covariance diagnostic and clustering use normalized content, leaving residual mask predictability in the actual clustering representation unmeasured. Moreover, a near-zero covariance norm on normalized content can coexist with strong linear separability. These observations do not contradict the covariance-only claim in Section 3.4, but leave a gap between the measured criterion and the broader motivation of preventing the representation space from being organized around observation patterns. The invariance diagnostic also leaves an important tradeoff unresolved. Because scarce views are dropped more frequently, low drift could partly reflect reduced use of information unique to those views. The reported reconstruction and clustering results do not directly establish that complementary scarce-view information is preserved. A controlled experiment in which a scarce view carries complementary cluster information, with clustering performance reported separately when that view is available and absent, would help assess this tradeoff.
Questions for authors
- Apart from BURG and CIMLN, which use official defaults, how were hyperparameters set for the other ten baselines? Did any baseline receive label-based or per-cell selection comparable to COIN’s?
- What are COIN’s results, including the ablations, when one coefficient setting is used per dataset or across all datasets, chosen without ground-truth labels? Examples would be a fixed default or a label-free internal criterion. How large is the gap between these results and the per-cell oracle-selected results in Tables 2 and 3?
- Can you report means and standard deviations over multiple independently generated masks and multiple representation-training seeds per dataset and missing rate? Several baselines behave erratically across missing rates
- The mask probe uses raw content, while the covariance diagnostic and clustering use normalized content. Can you report probe results on the normalized content actually used for clustering? Can you add a scale-invariant dependence measure, such as canonical correlation between content and mask?
- Can you show that low drift is not achieved by down-weighting the views most often dropped?
- CIMLN already enforces consistency across observation patterns by masking views, but it masks densely observed views, while COIN preferentially drops scarce ones. How does nested invariance differ conceptually and empirically from CIMLN’s masking module?
- All masks are generated independently of features and labels. How does COIN behave when missingness depends on the class or the content?
7 comments
W1 / Q2. Fixed-Configuration COIN and Ablation Results
We conducted a rerun of REC, REC+INV, REC+IND, and Full COIN, employing a consistent global code-default tuple across all five datasets and five missing rates. This evaluation utilized three training seeds per cell, without implementing per-cell label-based selection:
The macro-averages of the 25 dataset-rate cell means are:
| Fixed variant | ACC | NMI | ARI | ACC/NMI/ARI gain over REC | |
|---|---|---|---|---|---|
| REC | 0.5590 | 0.4096 | 0.3668 | 0.3542 | -- |
| REC+INV | 0.6294 | 0.4992 | 0.4686 | 0.0036 | +0.0703 / +0.0896 / +0.1018 |
| REC+IND | 0.6269 | 0.4923 | 0.4579 | 0.2820 | +0.0678 / +0.0827 / +0.0911 |
| Full COIN | 0.6423 | 0.5081 | 0.4786 | 0.0046 | +0.0832 / +0.0985 / +0.1118 |
Fixed-track ablation adheres to the intended module logic. The INV variant significantly reduces nested observation drift, whereas IND enhances clustering performance beyond mere reconstruction. Full COIN achieves the highest average values for ACC, NMI, and ARI among the four variants, while maintaining a near-zero . Consequently, the joint objective integrates clustering improvements with observation robustness under a single, globally fixed coefficient setting.
Gap to the oracle-selected tables. In Table 2 Full COIN, the fixed configuration exhibits a lower accuracy (ACC) in 24 out of 25 cells, with a mean fixed-minus-oracle difference of and a median difference of . This indicates a nonuniform gap across datasets and rates. Regarding the Table 3 ablations, the archived like-for-like comparison is available at . Across 40 method-dataset-rate cells, the mean fixed-minus-oracle differences are , , and for ACC, NMI, and ARI, respectively. The fixed results are lower in 17, 19, and 18 out of 40 cells, respectively. Consequently, per-cell oracle selection significantly enhances the historical Full-COIN headline ACC, whereas the average Table 3 ablation gap is considerably smaller and not consistently negative.
W1 / Q3. Multi-Mask and Training-Seed Stability
We combined three independently generated mask seeds with three representation-training seeds at . Each model execution averages ten K-means initializations internally; these initializations are not considered independent representation-training runs. Consequently, the nine values per dataset-rate cell comprise three mask realizations and three training seeds per mask, rather than nine independent missingness mechanisms.
The corrected fixed external comparison is as follows. Each entry is the mean$\pm$standard deviation over the same mask seeds training seeds, in the order of ACC/NMI/ARI.
| Dataset, | Fixed COIN | BURG | CIMLN |
|---|---|---|---|
| HandWritten, 0.3 | .8438$\pm$.0715 / .7834$\pm$.0460 / .7373$\pm$.0795 | .4764$\pm$.0753 / .4663$\pm$.0560 / .2947$\pm$.0659 | .7972$\pm$.0337 / .7856$\pm$.0185 / .7175$\pm$.0231 |
| HandWritten, 0.5 | .8099$\pm$.0696 / .7387$\pm$.0305 / .6847$\pm$.0610 | .2593$\pm$.0208 / .2150$\pm$.0253 / .0823$\pm$.0206 | .3586$\pm$.1577 / .3138$\pm$.1908 / .2301$\pm$.1742 |
| MSRCV1, 0.3 | .8454$\pm$.0201 / .7393$\pm$.0296 / .6765$\pm$.0355 | .5789$\pm$.0623 / .4659$\pm$.0606 / .3411$\pm$.0686 | .2246$\pm$.0243 / .0569$\pm$.0215 / .0039$\pm$.0132 |
| MSRCV1, 0.5 | .7385$\pm$.0329 / .6046$\pm$.0401 / .5061$\pm$.0621 | .2704$\pm$.0276 / .1219$\pm$.0336 / .0323$\pm$.0152 | .2190$\pm$.0073 / .0491$\pm$.0113 / .0003$\pm$.0047 |
| Synthetic3d, 0.3 | .9390$\pm$.0071 / .7780$\pm$.0189 / .8264$\pm$.0190 | .7271$\pm$.0842 / .4369$\pm$.0988 / .4208$\pm$.1169 | .7415$\pm$.0763 / .4613$\pm$.0726 / .4318$\pm$.0903 |
| Synthetic3d, 0.5 | .8945$\pm$.0262 / .6783$\pm$.0616 / .7153$\pm$.0649 | .5126$\pm$.0559 / .1419$\pm$.0969 / .1365$\pm$.0882 | .4954$\pm$.0906 / .1700$\pm$.1147 / .1200$\pm$.0835 |
| TinyImageNet, 0.3 | .1408$\pm$.0051 / .0939$\pm$.0023 / .0302$\pm$.0021 | .1019$\pm$.0017 / .0585$\pm$.0026 / .0137$\pm$.0008 | .1222$\pm$.0057 / .0866$\pm$.0034 / .0244$\pm$.0019 |
| TinyImageNet, 0.5 | .1288$\pm$.0078 / .0852$\pm$.0046 / .0254$\pm$.0029 | .0923$\pm$.0018 / .0438$\pm$.0041 / .0089$\pm$.0010 | .1048$\pm$.0034 / .0568$\pm$.0023 / .0142$\pm$.0008 |
| Washington, 0.3 | .6143$\pm$.0212 / .2448$\pm$.0162 / .2966$\pm$.0212 | .5771$\pm$.0211 / .2323$\pm$.0156 / .2674$\pm$.0186 | .4975$\pm$.0787 / .2477$\pm$.0429 / .2501$\pm$.0753 |
| Washington, 0.5 | .5494$\pm$.0386 / .1799$\pm$.0375 / .2136$\pm$.0414 | .4941$\pm$.0510 / .1432$\pm$.0282 / .1378$\pm$.0443 | .3443$\pm$.0676 / .1005$\pm$.0603 / .0793$\pm$.0741 |
The standard deviations presented here encompass both between-mask and within-mask training variations. The fixed COIN model surpassed the BURG model across all ten cell means for the ACC, NMI, and ARI metrics. Additionally, it outperformed the CIMLN model in ten instances for the ACC, eight instances for the NMI, and ten instances for the ARI. Notably, the CIMLN model demonstrated superior NMI performance on the Handwritten and Washington datasets at .
We examined the two anomalies highlighted by the reviewer using one frozen mask and three training seeds. For BURG on MSRCV1, the accuracy is at and at . The significant rate-dependent decline remains evident despite minor seed variations at , aligning with its reliance on complete or co-observed samples under conditions of severe missingness. For CIMLN on HandWritten, the accuracy is at and at . This increase is consistent across training seeds and is not attributable to a single-seed anomaly. In the multi-mask experiment, CIMLN exhibits considerable variability in combined mask and training effects on certain cells, including HandWritten at .
W1/Q1: Baseline Hyperparameter Configuration and Selection Fairness
We examined the hyperparameter configurations of all ten baselines, excluding BURG and CIMLN. Each baseline was assessed using a single recorded configuration for each method, adhering to the respective released implementation or its established experimental setup.
The principal configurations were as follows:
The recorded baseline evaluations used these configurations without performing a ground-truth-guided search over the dataset–missing-rate cells.
In contrast to the historical COIN evaluation, which selected configurations independently for each cell using the ACC, followed by the NMI and ARI, we conducted a different evaluation. This evaluation employed a single fixed coefficient configuration across all 25 cells and included repeated comparisons with BURG and CIMLN under shared-observation masks.