- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper concerns the recovery of an unknown subspace from binary classification rules whose parameters lie in that subspace. The main theoretical contribution is an asymptotic minimax bound on subspace estimation error, together with an estimator that achieves it. It also presents experiments demonstrating improved few-shot learning if historical tasks are based on the same subspace as the new task.
Critical strengths
The overall setup and theory of the paper appears coherent as far as I could evaluate.
Critical weaknesses
The writing is challenging to read The setup, model, and results are not properly explained. Instead it seems the paper assumes the reader to already be familiar with the results of the authors and just lists everything up instead of properly explaining it. Example in case: 066 For every fixed d > k ≥3, we compute the Fisher information Ik in one unit subspace-rotation direction and prove ... - What is d, what is k? This is never explained. Examples like these are many in the text. This makes the paper basically unreadable for a human.
Contribution is limited The authors concede themselves that basically their entire setup (or even a more general version) has been established by previous work. An open question and why it is of particular interest is never defined in the paper, instead the contribution section lists loosely connected mathematical results of no clear broader interest.
Disconnect between theory and experiment The theoretical results show recovery for pairwise observations while the experiment uses blocks of extremely long length and do not make use of the block boundaries. In fact it seems the results do not use the theoretically optimal estimator at all. It is never explained how the experiments are supposed to support the theory. How does this relate to the derived minimax bound? Why is this never evaluated?
Questions for authors
Can you explain why the presented results are interesting to a broader audience?
How do the experimental results relate to the theory?