- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends Saul's (2021) passive-aggressive algorithm for difference-of-squares (DoS) classifiers, i.e. low-rank quadratic models with score , to open feature spaces where features can appear, vanish and reappear. Instead of Euclidean proximity, the PA step measures parameter changes under a feature-side metric built up from over the stream. The core trick is that right-multiplying and by (and whitening ) maps this nonconvex QCQP exactly back to the original DoS-PA problem. So the same one-dimensional problem gives the step size, and the rank-one update just moves along instead of . Missing features are zero-filled, new ones are added by padding and , and the metric comes in full, diagonal and sketched versions. Experiments show clear gains over PA and DoS-PA on WDBC with synthetic feature masking, and on the natural IMDB and Crowdsense streams the full variant is competitive with existing open-feature methods.
Critical strengths
The paper is clean to read, and the FULL result on WDBC inc-dec is also a sizable reported gain.
Critical weaknesses
The algorithm is not fully specified for its own setting. Prop. 1 assumes 0 , but sec. 4.4 solves eq. (8) whenever the margin is violated. With , , Eq. (8) is : strictly decreasing, no minimizer. The QCQP is still solvable (boundary branch ), but that branch isn't in the paper and the initialization of is never stated. This matters because of an unmentioned invariant: Eq. (10) is with , , and zero-column expansion preserves rank, so rank forever. In the 30-dim synthetic inc-dec run, block 1 exposes 3 features plus bias, so the factors may be rank despite . Prop. 1 isn't broken, but "new features expand the model" is not what the specified procedure does.
Questions for authors
-
How were and new-feature columns initialized, what is the numerical rank of at the end of the run, and what does the code do when (resp. ) with a violated margin?
-
figure 1 shows the update before the prediction and a slack constraint, could you confirm all numbers are test-then-train with the hard-margin update?
Dear Reviewers,
Could you take a moment to review the authors' responses and let them know whether any concerns remain or require further clarification?
Best,
AC