- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EvoLabel combines two retrieval views with agrement checks and arbitration for in-context classification. It also includes optional label-space edits, but no edits are committed in the reported runs. Experiments across 12 datasets show prediction gains, while the analysis separates these gains from label-space evolution.
Critical strengths
(1) The paper clearly reports that label edits never occur and avoids attributing prediction gains to an inactive component. This is a useful diagnostic result. (2) Comparisions across datasets and backbones support the prediction pipeline. Paired analyses provide evidence that its gains are not solely due to evaluation noise.
Critical weaknesses
(1) Routing and expert choices are selected using the evaluation dataset suite. Additional backbones reuse that suite, so performance on unseen datasets remains uncertain. (2) Zero committed edits demonstrate an activation failure, but provide little evidence about whether executed label changes help or hurt. The conditional analysis does not resolve this central empirical question..
Questions for authors
(1) Do the gains hold when routing choices are selected without access to the target dataset, such as through leave-one-dataset-out evaluation? (2) Can valid label edits be activated and compared with a matched frozen-label system, including errors introduced outside the targeted examples?
Thank you for the detailed rebuttal and additional results. I will keep my current score.