- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DiHypeSC, a data-informed method for adapting regularization weights \beta for both elastic-net and block-diagonal regularization in subspace clustering. It uses subspace-preserving conditions to guide β selection for elastic-net and block-diagonal regularization. Experiments suggest competitive clustering performance compared with random search and Bayesian optimization, with fewer training runs for hyperparameter tuning.
Critical strengths
The paper is well-written and technically sound. The motivation is clear. The proposed method has strong theoretical support.
Critical weaknesses
Theoretically:
- Line 212. The relative proof for Lemma 1 (and Lemma 5) is found in [2] page 3, not [1].
- Line 196 and 199. The estimator computes (\hat C) using the learned query/key embeddings in SENet, but not fit the standard formulation of C in the comming theories. It makes me confused.
- Theorem1. The restricted weighted norm formulation introduces but not give any denotation, which will certainly affect the interpretability of for Elastic-Net SC problem on line 266-267.
- Lemma 3. The notation of U: "a fixed Laplacian" makes me confused. Does U have any relation with ? Or just choose randomly.
- Lemma 3. As read the PRO-DSC, I think PRO-DSC mainly aims to balance the trade-off between SE loss and TCR loss(Expansion Loss), which belong to Deep Representation Learning topic. I recommend only talk BDR[3] purely in this theory.
- As mentioned in Theorem 1 and Lemma 2, author use weighted norm to replace the Regularization items, which has similar structure studied in [4]. Therefore, I recommend to discuss the connection with [4] and gives more clear geometric meaning of .
Experiments:
- Algorithm 1,2. The is estimated only for , by a matrix inverse computation. Is computed every batch/epoch? I am concerned about the computation cost. Author should report the time comsumming proportion of the added estimation.
- Paper lacks the comparison with baseline setting of SENet and PRO-DSC performance reported in the raw paper, and also should report ACC/NMI for more fair comparison.
- The PRO-DSC method is sensitive to the hard dataset like CIFAR20, I recommend to add and report.
- Theorem 1, Lemma 3 lacks some sythetic experiments to support the conclusion.
I am willing to upgrade my score corresponding to the problem solved as mentioned above.
[1] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., 35 (1):171–184, 2013. doi: 10.1109/TPAMI.2012.88. URL https://doi.org/10.1109/ TPAMI.2012.88.
[2] Mahdi Soltanolkotabi, Ehsan Elhamifar, and Emmanuel J. Cande`s. Robust subspace clustering. The Annals of Statistics, 42(2):669 – 699, 2014b. doi: 10.1214/13-AOS1199. URL https: //doi.org/10.1214/13-AOS1199.
[3] Canyi Lu, Jiashi Feng, Tao Mei, Zhouchen Lin and Shuicheng Yan. Subspace Clustering by Block Diagonal Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
[4] Nimita Shinde, Tianjiao Ding, Daniel P. Robinson, and René Vidal. Geometric Analysis of Nonlinear Manifold Clustering. In Annual Conference on Neural Information Processing Systems, 2024
Questions for authors
See above