- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies Interactive ReID, where a questioner must select questions based on the evolving candidate set and dialogue history. It proposes RAVEL, a retrieval-aware online reinforcement learning framework that initializes from supervised question generation, directly observes the current Top-4 candidates, and optimizes the question policy using rank feedback from the complete question–answer–retrieval loop. Experiments on Interactive-PEDES and three additional text-based ReID benchmarks report improved retrieval performance, along with analyses of the learned questioning behavior.
Critical strengths
-
The paper frames interactive ReID as a partially observable sequential process, where the value of a question depends on the evidence produced by its answer and the resulting retrieval update. Evaluating question utility through realized retrieval gain rather than static judgments provides a coherent formulation of the problem.
-
The empirical study reveals a weak association between static question judgments and realized retrieval gain, highlighting the limitations of using static supervision to assess question utility. It also provides useful diagnostics of how the learned policy changes its questioning behavior across interaction rounds.
-
The experiments fix the retriever and answerer during policy learning and evaluation, which helps isolate the effect of the questioning strategy within a controlled interaction environment. The additional experiments with alternative retrieval backbones provide evidence that the learned policy is not entirely tied to a single retriever.
Critical weaknesses
-
The Qwen3-VL analysis finds only a weak association between static question judgments and realized retrieval gain. However, this does not show that offline question ordering is ineffective or that online policy learning is necessary. The motivation for retrieval-aware online reinforcement learning is therefore insufficient. 2. Table 3 lacks a direct comparison between SFT with the validity gate and SFT without RL. Since removing the gate causes invalid-action collapse, it remains unclear whether the improvement comes from retrieval-aware RL or from the gate constraining the policy to answerable questions.
-
Table 3 lacks a direct comparison between SFT with the validity gate and SFT without RL. Since removing the gate causes invalid-action collapse, it remains unclear whether the improvement comes from retrieval-aware RL or from the gate constraining the policy to answerable questions.
Questions for authors
-
Are new states generated during policy updates dynamically added to the curated 3,000-state training pool, and if not, how does the current method fully justify the claim of true online learning?
-
Could the authors provide a sensitivity analysis for the manually fixed reward coefficients in Equation 9 to clarify how much the final performance relies on these specific values?
Round awareness and objective. RAVEL rewards the immediate retrieval change for each question-answer step; it does not optimize a return summed across all rounds. Each policy input explicitly includes the round index, current Top-4, and dialogue. Training samples states from different rounds, allowing the LLM to recognize the current round and tailor its question to the stage and evidence. This is our claim of round-aware question generation under a per-step objective. The shared explanation appears in the Overall Response under “Sequential policy and reward horizon.”
Validity gate and Table 3. We corrected the submitted statement that the gate was applied during both training and test-time rollouts. Section 3.4 now says it is used only during RL training, and Table 3's Gate column indicates validity filtering; Supplementary B lists whole-outfit and multi-region rules. Thus, the SFT-only condition and all evaluations have no gate. The Overall Response under “Invalid-question reward is −0.2” reports the audit: broad questions fell from 3,955/36,865 turns (10.73%) in the no-gate transcript to 70/36,865 (0.19%) in the gate-trained ungated evaluation. Removing the remaining answers changed R@1 from 81.581 to 81.500.
Online learning and the fixed state pool. During training, each update samples a question, obtains an answer and rank feedback from the retriever, and updates the questioner. These action outcomes come from fresh interactions during learning; offline RL instead learns from logged actions and outcomes without new interaction. Whether the starting-state pool expands is a separate design choice (Wagenmaker & Pacchiano, 2023). The 0K/1K/3K/5K scores and Supplementary updates appear in the Overall Response under “State-pool scaling”; Supplementary B describes the pool and its construction.
Updated reward and scaling results. The revised reward removes the submitted 0.35 Rank-1 entry term. From the shared no-interaction R@1 of 37.61, the five-round result is 81.76/94.83/97.52/48.18 (R@1/R@5/R@10/mAP; BRI 0.484). Under this reward, transfer results are 82.57/95.96/98.02/73.38 on CUHK-PEDES, 71.30/86.23/91.07/43.63 on ICFG-PEDES, and 73.88/91.64/95.86/57.80 on RSTPReid. Retriever transfer to RDE reaches 86.19/96.85/98.56/49.82 (BRI 0.416). The revised reward was used for the main experiment, dataset and retriever transfer, and training ablations. The Overall Response under “Updated reward and main result” and “Controlled ablations” reports the full results and manuscript changes.
The state-pool comparison reports 0K at 69.44/87.78/92.80/45.48/0.698, 1K at 79.71/93.88/96.96/48.85/0.536, 3K at 81.76/94.83/97.52/48.18/0.484, and 5K at 81.161/94.982/97.423/48.328/0.489 (R@1/R@5/R@10/mAP/BRI). We updated the supplementary state-pool table and construction text to include all four pool sizes. The full comparison appears under “State-pool scaling” in the Overall Response.
Invalid-question reward (−0.2). The no-gate transcript contained 3,955 broad whole-outfit or multi-region prompts in 36,865 turns (10.73%). Removing their answers lowered R@1 from 84.42 to 75.38. After adding these rules to the RL training gate, a gate-trained policy produced only 70 broad whole-outfit questions (0.19%) in its ungated evaluation. Removing those answers changed R@1 from 81.581 to 81.500. This comparison supports the intended role of the penalty: discouraging broad prompts that bypass the localized-question protocol. The Overall Response under “Invalid-question reward is −0.2” gives the transcript audit and full gate-run results; Section 3.4 now states that the gate is used only in RL training, and Supplementary B lists the invalid-question rules.
Within-group repeat penalty (−0.02). GRPO compares eight sampled questions for one state. If all eight are duplicates with the same retrieval outcome, their group-relative advantages are all zero. The first occurrence retains its retrieval reward, while later duplicates receive −0.02, giving tied samples a small relative signal and encouraging question diversity. In the main 3K run, groups averaged 7.93 unique questions out of 8, with a 9.20% repetition rate. Since the deduction applies only to duplicates and is small relative to retrieval changes, a fine-grained coefficient sweep would be difficult to interpret. The full rationale is in the Overall Response under “Within-group repeat penalty is −0.02.”