- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CROSS, which is a routing framework for in-context learning that combines a target-aligned hyperbolic nearest-neighbor selector with a diversity-aware determinantal point process (DPP) selector.
A logistic regression router uses 12 features derived from the query, predictions, and selected demonstrations to decide whether to replace the default DPP prediction with the Hyperbolic prediction.
Critical strengths
- Effective use of complementary selectors
- CROSS combines query-local relevance and set-level diversity in a lightweight framework, achieving consistent improvements over the DPP baseline across multiple datasets and LLM backbones without additional model fine-tuning.
Critical weaknesses
- Limited validation of the routing mechanism
- The router compresses 12 features into a single score and applies a global switching threshold, but the paper does not sufficiently discuss whether the scalar score is sufficient to distinguish the retrievers and whether the global threshold is sufficient for different types of queries.
- Beyond final accuracy, the evaluation would benefit from held-out oracle-agreement and routing-regret analyses, threshold-sensitivity experiments, and comparisons with more expressive routers.
- Outdated models and baselines
- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.3 are somewhat old.
- Topic Coverage-based Demonstration Retrieval for In-Context Learning, EMNLP'25 also discusses the trade-off between embedding similarity and diversity. The authors need to compare the proposed method with this paper in a theoretical and empirical manner.
Questions for authors
- Is a scalar score enough to route the retriever?
- Is a global threshold enough for all queries?
- Why not use an ensemble of the retrievers?
- Does the paper evaluate the calibration of the router by comparing its output scores with oracle routing decisions?