- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Few-shot prompting of vision-language models (VLMs) is powerful but fragile — the choice and order of example images in the prompt can make or break performance. Standard practice is k-nearest-neighbor retrieval, which tends to fetch visually similar but redundant examples.
This paper proposes Next Demonstration Prediction (NDP), a learned selector that builds the prompt one example at a time. The idea is simple: first, teach a small neural network to imitate a classic relevance-plus-diversity heuristic (MMR); then, fine-tune it with reinforcement learning (PPO) so that the examples it picks actually maximize the frozen VLM's accuracy on the task — for example, minimizing age-estimation error or maximizing VQA accuracy. A Perceiver module compresses the large candidate pool, and an autoregressive decoder picks each next shot, so the prompt is constructed as an ordered sequence, not just a set. The authors evaluate on 14 benchmarks spanning regression, classification, and VQA, mostly with a Gemma 3 4B model at 16 shots, plus smaller cross-model and transfer experiments. The results are honest and mixed: NDP achieves the best average regression error and VQA accuracy among the image-only selectors it competes against, but loses to DPP on classification.
Critical strengths
- The paper is well organized, the abstract accurately reflects mixed results, and limitations are integrated into the main text rather than buried.
- Treating demonstration selection as an ordered sequential decision problem rather than unordered subset selection is a meaningful conceptual contribution. The permutation experiments (Table 3) provide direct evidence that the learned order carries information beyond the selected set itself — the predicted order (regression MAE 0.62) clearly beats shuffle (0.80) and reverse (0.83).
- The authors conduct extensive evaluations on 14 datasets across three task families, covering K-shot scaling curves () for all datasets in the appendix, three target VLMs, and both in-domain and transfer settings. The qualitative analysis (Figures 3, 4, 9–20) — showing anchor, boundary, global-scale selection dynamics, and NDP spanning the label distribution where k-NN retrieves near-duplicates — is informative and consistent with the quantitative gains.
- The paper explicitly reports that NDP loses to heuristics on cross-dataset transfer, that it trails DPP on classification, and that ICL can hurt relative to zero-shot. This honesty strengthens the credibility of the positive claims and makes the paper useful to practitioners deciding when RL-based selection is worthwhile.
Critical weaknesses
- The authors concede that they did not evaluate the behavioral-cloning checkpoint, so it cannot attribute improvements to PPO versus the MMR initialization. This is a critical gap: MMR alone already achieves strong classification results (82.0 avg, beating NDP's 81.1), and other NDP gains (e.g., 54.2 vs. MMR 53.5 on VQA) are within plausible seed/evaluation noise. Without a BC-only ablation, the paper cannot support its framing of "RL fine-tuning" as the active ingredient, nor the claim that NDP "deviates from the MMR heuristic."
- The paper notes that ~48k–64k VLM generations per policy are required and that matched GPU-hours, latency, and break-even query counts were not measured. For a method whose practical value proposition is "pay training cost once, serve many queries", the absence of any amortization analysis makes the use-case argument (Section 4.7) speculative.
- Reward is zero until step K; the paper offers no analysis of whether the GAE critic actually provides meaningful credit to early selections, nor any diagnostic (e.g., value-function accuracy, per-step advantage magnitudes).
- Cross-dataset transfer fails badly (below random on EmoSet→Food-101), cross-model transfer is mixed (helps on Phi EmoSet, slightly hurts on Qwen UTKFace, which is not uniformly better than in-domain or zero-shot), yet the abstract and conclusion still frame transfer as a contribution ("cross-dataset policy transfer can underperform simple heuristics" is acknowledged but the positive transfer cases are given equal billing).
- Missing related-work positioning.
- Partial2Global [1] and LLM-R [2] are trained, non-RL selectors for (visual) ICL that the paper neither cites nor benchmarks — without them, "learned selection beats heuristics" is not established as "RL selection beats supervised selection".
- The ordering claim, the paper's most novel contribution, is never compared against curriculum-based ordering methods such as CDS [3], leaving open whether a training-free complexity ordering captures most of the permutation-sensitivity effect.
References
[1] Towards Global Optimal Visual In-Context Learning Prompt Selection, NeurIPS 2024.
[2] Learning to Retrieve In-Context Examples for Large Language Models, EACL 2024.
[3] Curriculum Demonstration Selection for In-Context Learning, SAC 2025.
Questions for authors
- Why don't you evaluate the BC-only checkpoint (after Stage 1, before PPO) on any of the 14 benchmarks? If PPO contributes little or nothing beyond the MMR-imitating policy, the paper's central claim — that RL fine-tuning on VLM reward is the active ingredient — would not hold.
- How many independent training seeds or evaluation resamplings were used for the headline Table 1 results? Without variance estimates, the key performance gains cannot be distinguished from noise.
- Is there a substantive reason MMICES (or a minimal text-conditioned variant of your own selector) could not be run on cached embeddings?
- What is the wall-clock latency of the trained NDP selector (Perceiver encode + K decode steps) relative to, e.g., MMR or DPP over the same 8,192-candidate pool, and roughly how many queries would be needed to amortize the ~48k–64k training generations?
- For Qwen-3 on UTKFace, the transferred policy (5.98) slightly beats the in-domain policy (6.28). Do you have any analysis (e.g., embedding-space overlap, dataset statistics) explaining when Gemma-trained selection transfers and when it collapses below random (EmoSet→Food-101)?
2 comments
Thank you for your reply
I thank the authors for the detailed responses to my comments. Given that the authors acknowledge limitations and narrow the statements on most issues, I am inclined to maintain my original rating.
Thank you for recognizing our transparent reporting, permutation analysis, and qualitative study. We address your five weaknesses and questions below.
W1. BC-only evaluation and PPO attribution
We agree that MMR does not substitute for the BC-only checkpoint: a learned imitator may differ from its teacher. Without checkpoint results, we cannot isolate PPO’s contribution.
We will remove claims that PPO is the established active ingredient or that the NDP–MMR gap measures RL’s benefit. The results support the complete pipeline, not PPO’s necessity or independent benefit.
W2. Training cost and amortization
Approximately 48,000–64,000 reward generations per policy constitute substantial training cost, excluding BC, validation, and development.
Without matched training and serving measurements, “train once, serve many” is a possible deployment setting, not demonstrated cost-effectiveness. We will remove implications that query volume necessarily makes NDP cheaper than heuristics.
W3. Sparse reward and critic diagnostics
The critic estimates return from partial histories, and GAE constructs advantages from temporal-difference residuals. This describes the mechanism but does not establish useful credit assignment to early selections.
We lack value-function accuracy and per-step advantage diagnostics and will acknowledge this limitation. We will also correct the statement that PPO clipping “ensures monotonic improvement”; the clipped objective provides no such guarantee.
W4. Transfer failures and framing
Isolated positive cases do not establish general transfer capability. Cross-dataset transfer can underperform random selection, and cross-model transfer is mixed.
We will present transfer as a limitation analysis. Source-specific preferences and domain mismatch remain hypotheses; the results do not establish a reliable criterion for transfer success.
W5. Missing learned selectors and curriculum baselines
Thank you for identifying Partial2Global, LLM-R, and CDS. We will add related-work positioning and acknowledge that superiority over supervised selectors or curriculum ordering is unestablished.
NDP’s history-conditioned policy differs from ranking or feedback-trained retrieval, but this alone establishes no advantage. Likewise, permutation tests show order sensitivity, not superiority over complexity-based curricula or an optimal order.
Q1. Why omit BC-only evaluation?
We have no methodological reason that makes this omitted control unnecessary. PPO’s benefit over initialization remains unresolved; we will attribute results to the combined pipeline rather than infer the missing comparison.
Q2. Training seeds and evaluation resampling
We will explicitly report these counts. We have no additional independent-run results or significance tests to report.
Small headline differences should therefore not be interpreted as established improvements beyond experimental variability. Shot-count curves do not substitute for seed or resampling uncertainty.
Q3. MMICES or text-conditioned retrieval
Question-aware retrieval is feasible with appropriate cached representations. Our image-only restriction holds retrieval features fixed; it does not establish the strongest VQA system.
We will qualify the findings as comparisons among evaluated image-only selectors, not superiority over MMICES or text-conditioned NDP.
Q4. Selector latency and break-even volume
We lack matched selector latency measurements over the same 8,192-candidate pools and cannot provide a numerical break-even count.
For training cost Ctrain and matched per-query costs cNDP and cH, a compute-cost crossover exists only when cH exceeds cNDP:
Q>cH−cNDPCtrain.
Otherwise, increasing volume spreads training overhead but never makes NDP cheaper. Generation counts alone cannot establish compute savings; we will distinguish predictive utility from efficiency.
Q5. Explaining transfer outcomes
Qwen UTKFace transfer MAE of 5.98 versus 6.28 in-domain does not establish reliable superiority without uncertainty estimates. Neither beats zero-shot MAE of 5.49.
We lack embedding-overlap or dataset-statistics analyses explaining this difference or the EmoSet-to-Food-101 failure. Source-specific preferences, model-response differences, and optimization variability are possible explanations, not validated mechanisms.
Thank you again. We hope these clarifications and revisions address your concerns and would appreciate reconsideration of the score if you find them sufficient.