- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies chat-based person retrieval, where a system progressively acquires appearance evidence through multi-turn question answering. The authors propose EAVI, which combines a Dialogue Evidence Set Retriever (DESR) with a Value-Guided Interaction Agent (VGIA). DESR augments global dialogue-image matching with slot-based evidence sets aligned via optimal transport, while VGIA selects questions from a recorded QA pool and decides when to stop based on the current retrieval state. Experiments on ChatPedes show modest retrieval gains over DiaNA, improved efficiency, and reduced interaction cost through adaptive stopping.
Critical strengths
- This paper focuses on an important variant of person retrieval, where the evidence acquisition is in a progressive chat-based setting.
- This paper provides a new perspective on disentangling the effects of evidence matching and evidence acquisition, and the proposed method follows the same paradigm.
- The proposed method is carefully designed, providing a reasonable fine-grained matching mechanism through anonymous evidence slots and optimal-transport-based alignment, while keeping gallery representations cacheable.
- Experiments show consistent improvements on the ChatPedes benchmark.
Critical weaknesses
- Citation and presentation issues. Some in-paragraph citations appear to use \cite rather than \citep, which leads to awkward formatting in the current draft.
- Concerns about the problem setup:
- Limited realism of the pre-recorded QA protocol. The pre-recorded QA pool is useful for decoupling question selection from question generation and for analyzing different interaction-control abilities. However, it substantially simplifies the realistic ChatPR setting. In practical conversational retrieval, questions should be generated sequentially and conditioned on previously observed answers, rather than selected from a full pool of candidate questions available in advance.
- Potential information leakage from future candidate questions. Even though the controller cannot observe unrevealed answers, seeing the full set of future candidate questions may leak useful information about the target. For example, if a candidate question is “Is the woman carrying a backpack?”, the question text itself already reveals gender-related information.
- Limited evaluation scope. All main experiments are conducted on a single dataset, ChatPedes. This makes it difficult to assess the generality and soundness of the proposed method.
- Unclear contribution of some modules.
- A fitted question prior performs as well as, or slightly better than, the full-state controller across all fixed ASK budgets. This weakens the claim that retrieval-state feedback is broadly useful for value-guided question selection.
- The learned stopping head is not clearly stronger than a simple entropy-based rule, suggesting that the learned stopping module provides limited additional benefit in the current setting.
- DESR is compared with DiaNA using substantially different dialogue encoder sizes. Therefore, the reported system-level efficiency gains may partly come from backbone/model-size differences rather than the proposed evidence-set alignment mechanism itself.
Questions for authors
Based on the claims made before, I think this paper is an analysis disentangling the effects of evidence matching and evidence acquisition, and is highly dedicated to a specific benchmark and setup, which limits the contribution of this paper (clearly below the acceptance threshold). The authors should answer the questions in the "weakness" section.