- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies low-budget active learning through coreset selection. It selects exactly unlabeled samples by minimizing the Sinkhorn divergence between the selected subset and the full dataset in a self-supervised feature space. The proposed FW-Swap algorithm first runs Frank-Wolfe on a continuous relaxation and uses the intermediate active-set vertices as discrete candidates. It then improves each candidate using gradient-guided single-swap local search. The selected samples are labeled and used to train a classifier. The paper also provides a generalization bound and a suboptimality bound, and evaluates the method on image, histopathology, and EEG datasets.
Critical strengths
-
The FW-Swap design is interesting. Although the continuous relaxation has a trivial optimum, the method extracts useful discrete candidates from the Frank-Wolfe trajectory and improves them using gradient-guided swaps.
-
The method shows promising performance in several extremely low-budget settings, which is practically relevant when annotation is expensive.
Critical weaknesses
-
The main theoretical motivation has a feasibility mismatch and a proof error. The optimization problem requires selecting exactly samples. However, the cluster-central-outlier example compares a one-point central configuration with a two-point extreme configuration. They are not simultaneously feasible under the same fixed-cardinality constraint. In addition, the proof defines but later states . The correct expression is . Since this positive bound is used in the subsequent proof, the current argument does not establish the claimed proposition.
-
The experiments do not sufficiently isolate the contributions of the debiased Sinkhorn objective, Frank-Wolfe initialization, and swap refinement. A controlled comparison of Sinkhorn cost versus Sinkhorn divergence under the same optimizer, and FW-only versus randomly initialized Swap versus FW-Swap, would make the contribution much clearer. Some empirical statements are also stronger than the detailed tables support, since FW-Swap is not the best method in every reported low-budget setting.
Questions for authors
-
Can the authors reformulate the cluster-central-outlier example so that all compared configurations belong to the same fixed-cardinality feasible set? Can the authors also provide a corrected proof after fixing the incorrect identity for ?
-
In the mixture-bias proof, if , the natural identity is . Could the authors clarify the parameterization used in the current derivation and confirm whether the final suboptimality bound remains unchanged?
-
The generalization theorem assumes squared Euclidean ground cost, while the experimental section describes Euclidean distance. Which cost is actually used in the implementation, and which theoretical guarantee applies to the experiments?
-
Could the authors provide controlled ablations comparing Sinkhorn cost versus Sinkhorn divergence, and FW-only versus randomly initialized Swap versus FW-Swap, under the same representation, classifier training, and computational budget?
3 comments
Thank you for the response and revisions. The corrections to the theoretical example and proofs, together with the additional ablation studies, address several of my original concerns.
However, the mismatch between the squared Euclidean cost assumed in the theory and the Euclidean distance used in the experiments remains unresolved. The acknowledged baseline implementation errors and the use of ground-truth labels to construct class-balanced candidate pools also weaken the empirical evidence.
These remaining issues affect my confidence in the paper's main claims. I therefore maintain my rating.
We acknowledge the noticeable difference in the empirical results weakens our orginal claims. However, FWSwap remains competitive on most instances and the best performing when the budget is the same as the number of classes (B=10 for both STL-10 and SVHN, B=2 for GasHisSDB and the EEG datasets). Specifically, on SVHN, FWSwap yields the best performance on B=10, 20. For the higher budgets, we achieve similar results to the best method, which in this case would be TypiClust. More so, on the EEG datasets our positioning does not change. In all experiments, we also present the most stable results.
Regarding the class-balanced subsampled dataset, this step is done simply to reduce the pool to a size compatible with our compute power. It is applied once and is independent of the selection process. All other methods receive the same identical pool, and no method has access to the labels during selection. We also point out that all datasets used as reference benchmarks are typically made balanced by selecting from the different classes, we thus follow a standard process.
Regarding the mismatch between the ground cost in theory and experiments, we noticed that the distance performed better and kept it for practice. The reason for the squared one in theory is being able to leverage results from Stromme Rigollet, our manuscript is not about statistical OT theory but about a new method for active learning. The whole generalization bound could be adapted by re-adapting their whole set of results to the Euclidean distance, but this would create unnecessary auxiliary results that are incremental and not the focus of our paper, there is value in conciseness.
Finally, the baseline implementation errors have all been fixed with the correct results now in the available PDF.
Response to weaknesses
The feasibility mismatch can be easily fixed, for instance by putting two atoms as the central outlier. Otherwise even with a single outlier, the biased Sinkhorn will select αL instead of αE. We also corrected the proof, only that last step of Proposition 6 needed to be fixed locally, the result still holds.
We added a complete ablation study to the revised manuscript that properly compares Sinkhorn cost versus Sinkhorn divergence under the same optimizer, and randomly initialized Swap versus FW-Swap. The empirical statement has been revised to be clearer given the results.
Response to questions
See W1.
We fixed a typo, the potentials should have been indexed by λt instead of just t, the integral then uses a change of variable s=λt.
We use the squared Euclidean ground cost in the proof simply to benefit from the Rigollet (2025) result. However, in practice, we find that the AL model achieves better results using a Euclidean ground cost.
This has been added to the revised manuscript in the appendix under section ablation study. In this section, we detail how the FW-Swap algorithm changes with different parameters put to test. We vary the values of both the entropic regularizer ε and the stability mixing term λ. We compare the Sinkhorn divergence to both the Sinkhorn algorithm and the Wasserstein distance on the downstream task. Lastly, we added a table comparing the results given by each: FW-only, randomly initialized Swap, and FW-Swap. Both the parameters and the AL pipeline were fixed for the last two comparisons as they are in the main paper. FW-Swap remains the best on all these baselines.