- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes semantic view evolution to deal with incomplete multi-view clustering instead of treating the available views as a fixed set of representations, like other papers. Existing methods perform information fusion within the fixed view space, failing to explore complementary semantics. To address this, the paper applies semantic view evolution to learn initial semantic representations via cross-view mutual information learning and then explores new semantic information via view expansion and contraction, ensuring the completion process grows progressively.
Critical strengths
This paper is well-motivated and well-structured, and it solves the problem from fixed view space to evolving semantic view space. It clearly explains the mechanism for expansion and suppression during the completion process and utilizes orthogonal residual fusion to avoid shared information. It shows substantial ablation studies and analysis.
Critical weaknesses
- As the paper mentioned, the global prototype is the mean of active views, which is too simple to deal with complex cases. (e.g., views are really different)
- For the expansion, the computation of the highest complementarity of pairs is expensive. Computational time is concerned.
- For the view contraction, the paper calculates its similarity between each active view and the current global semantic representation, which may capture all the redundancy.
Questions for authors
- The paper involves many pair calculations, so the illustration of efficiency is expected. What is the efficiency compared to other methods?
- Is the computation of pair selection in view of expansion stable or sensitive? If it is sensitive, how will it affect the selected pair and the final performance?
- The analysis of limitation does not illustrate the fragility of pair selection and view contraction in terms of accuracy and efficiency.
3 comments
Q1: During the view expansion stage, our method first computes the cross-view similarity matrix between every pair of active views, with a time complexity of for the matrix multiplication, where denotes the number of samples and denotes the latent representation dimension. Assuming that there are currently active views, all view pairs need to be evaluated to calculate their complementarity scores and identify the optimal pair. Therefore, the overall time complexity of a single complementarity search in the view expansion stage is .
Furthermore, we have added a theoretical complexity comparison of different pairwise search strategies. Although random pairwise sampling and candidate screening can reduce the number of view pairs to be evaluated and thus lower computational costs, they may overlook the most complementary view pair. In contrast, the exhaustive search adopted in our method guarantees the selection of the most complementary pair among the current set of active views. Finally, we measured the actual runtime on six datasets. The results demonstrate that the overall computational overhead introduced by the view expansion process remains acceptable.
| Stage | Hdigit | 100leaves | MNIST_USPS | handwritten | MSRC-V1 | Fashion |
|---|---|---|---|---|---|---|
| Full Method | 715.62s | 448.75s | 408.40s | 413.56s | 674.39s | 896.82s |
| Expansion Only | 63.87s | 51.50s | 22.51s | 275.81s | 178.73s | 213.05s |
Q2: Our method adopts a unified bidirectional conditional entropy criterion for view pair selection. All candidate view pairs are evaluated using the same procedure, and the pair with the highest complementarity score is selected. Therefore, the selection process is based on an explicit evaluation criterion rather than subjective manual decisions or random selection. In addition, although latent representations may vary across different training runs, potentially affecting the ranking of some view pairs, the subsequent semantic fusion process further integrates information from different views. Consequently, variations in individual view pair selections do not necessarily lead to significant changes in the final clustering performance.
Q3: Regarding view pair selection, our method primarily relies on bidirectional conditional entropy to evaluate the complementarity between views, and its effectiveness depends on the quality of the learned latent representations. When the complementarity scores of different view pairs are close, the selection results may become less stable. In addition, view contraction measures redundancy mainly through the similarity between each active view and the global semantic representation, which may have limitations in capturing complex higher-order redundancy relationships. In terms of computational efficiency, view pair selection requires exhaustive evaluation of all active view pairs, and its computational cost increases with the number of views and the sample size. Therefore, improving the stability of view selection, enhancing the capability to identify complex redundancy relationships, and further reducing computational costs in large-scale scenarios are important directions for future research.
Dear Reviewer jhKL,
Thank you for your time and effort in reviewing our paper. Your comments have been valuable in helping us improve its quality.
In response to your concerns regarding the simplicity of global prototype aggregation, the computational efficiency and stability of view pair selection, and the limitations of view contraction, we have provided detailed explanations, additional experiments, and further discussions in our rebuttal.
As the author-reviewer discussion period ends soon, we kindly encourage you to review our responses and share any further feedback you may have.
Thank you again for your time and consideration.
Best regards, The Authors
Thanks for your careful review. We are glad to address your questions one by one.
W1: The use of mean aggregation in this work is primarily motivated by model design and experimental evaluation considerations. The core objective of our method is to explore and integrate complementary semantics through the view expansion and view contraction mechanisms. Introducing a more complex adaptive fusion strategy may introduce additional model factors, making it difficult to distinguish whether performance improvements originate from the core mechanisms or the fusion strategy itself. Therefore, we adopt a relatively simple mean aggregation strategy to construct the global prototype, thereby minimizing the influence of additional design choices on the evaluation of the core mechanisms.
Meanwhile, it is worth noting that mean aggregation for constructing the global prototype is a replaceable implementation choice rather than a fixed constraint of our framework. Depending on the requirements of specific tasks, this module can be replaced with attention-based weighting or adaptive fusion strategies. Furthermore, the global prototype H is continuously updated and refined during the iterative optimization process. Mean aggregation is only one step in the initialization and integration of global semantics. The global prototype is subsequently optimized by incorporating semantic information from all active views, enabling it to progressively integrate complementary information across different views. Therefore, mean aggregation should not be considered in isolation as the final representation process of the global prototype.
W2: During the view expansion stage, our method first computes the cross-view similarity matrix between every pair of active views, with a time complexity of O(N2d) for the matrix multiplication, where N denotes the number of samples and d denotes the latent representation dimension. Assuming that there are currently V active views, all V(V−1)/2 view pairs need to be evaluated to calculate their complementarity scores and identify the optimal pair. Therefore, the overall time complexity of a single complementarity search in the view expansion stage is O(V2N2d).
To further investigate the computational overhead of view expansion, we separately measured the runtime of the view expansion operation alone. Datasets such as Handwritten and MSRC-V1, which contain more views, require the evaluation of a larger number of view pairs to assess their complementarity. Notably, on the Hdigit dataset, which contains a relatively large number of samples, the view expansion operation takes only 63.87s, accounting for 8.93% of the total training time. This demonstrates that our method remains computationally feasible when processing large-scale datasets and that the additional computational overhead introduced by view expansion is manageable.
W3: The primary motivation for adopting this strategy is to measure the degree of information overlap between each active view and the current fused representation from a global semantic perspective, thereby identifying views with relatively low semantic contributions, rather than attempting to characterize all forms of view redundancy. Compared with directly calculating redundancy between every pair of views, this strategy evaluates the relative contribution of each view by taking the current global semantic state into account, while avoiding additional computational complexity.