- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies interactive document retrieval in specialized domains (legal, medical, patent), where a user's initial query omits facts that determine relevance and the system asks clarification questions over several rounds with a fixed retriever.
It makes two claims. First, clarification should become progressively finer: broad questions early, fine-grained questions once the candidate documents are more focused. Second, because users may stop early, the questioning policy should be trained to obtain retrieval gains earlier, which a terminal reward does not encourage.
The proposed framework, CLEAR, has two components. Progressive Residual Facet Discovery (PRFD) encodes documents with residual quantization (RQ) codes and turns fact differences between retrieved document pairs that share an RQ prefix into candidate facets, preferring longer shared prefixes in later rounds. Turn-aware Hierarchical Policy Optimization (THPO) trains a Qwen3-4B questioning policy with stopping-weighted reward-to-go returns and advantages that combine trajectory-, facet-category- and facet-level comparisons.
Critical strengths
1. A training signal derived from the stated objective. The stopping-weighted return is derived rather than asserted. Under a stopping distribution, the first decision's return equals expected quality at stopping minus the initial quality (Eq. 13), and earlier gains are excluded from later decisions with a clear argument (Eq. 16). This ties the learning signal directly to the "earlier gains" goal.
2. Broad and transparent evaluation. The evaluation is broad:
- three domains and seven adapted baselines, with a fixed retriever and frozen candidate pools;
- query-paired bootstrap intervals (Fig. 5, Fig. 7);
- unselected success and failure trajectories (Fig. 8);
- limitations stated explicitly (App. A.3).
Critical weaknesses
1. The paper claims coarse-to-fine clarification, but neither its mechanism nor its experiments show that questions actually move from coarse to fine. Progressive granularity is the paper's first contribution (Sec. 2). It is implemented by preferring document pairs with longer shared RQ prefixes in later rounds (Sec. 3.2). A long shared prefix means two document embeddings are close; it does not mean the fact separating them is fine-grained. Take the paper's own running example (Fig. 1): two theft cases with nearly identical narratives may differ only in whether a threat was used. That fact changes the charge and is arguably the coarsest question to ask, yet such a pair shares a long prefix and would be deferred to later rounds. The paper itself disclaims the link: "Prefix length is a structural preference, not a direct measure of question specificity" (App. A.5).
The behaviour is not measured either. Sec. 4.4 reports only NDCG@10, and no analysis checks whether later questions are finer than earlier ones. Higher retrieval quality may equally come from document-grounded candidates in general. The only direct comparison is a "descriptive cross-run comparison" (Fig. 4, 7), and the patent configuration disables the schedule yet still beats all baselines.
Finally, the premise rests on a single hedged sentence (Sec. 1: fine-grained questions asked early "may rely on unsupported assumptions"). Natural alternatives are not discussed: the inventory's own category→facet hierarchy (App. C.1), a domain taxonomy, or LLM-judged specificity. Without these, the progressive component reads as a generic compression technique applied to a new setting, which weakens the novelty claim.
2. Hierarchical advantage is unmotivated, and the gains are not shown to come from the paper's new components. Neither problem in the introduction (what to ask, gaining earlier) calls for hierarchical advantage. The paper only describes it as providing "the learning signals for policy optimization" (Sec. 1) and reads its ablation post hoc as "consistent with" its role (Sec. 4.5). It gives no reason why summing trajectory-, category- and facet-level comparisons is a better credit signal than ordinary group-relative advantage or a learned critic.
This matters for attribution. The largest component gap comes from document-grounded candidates (Table 2), an idea the paper builds on from CARQ and DocFeatures (Sec. 2). The stopping-weighted return, though well derived, is standard reward-to-go. The components that are new here, progressive prefix selection (W1) and hierarchical advantage, are the ones without a rationale. The paper therefore does not show that its own contributions, rather than inherited ideas, make CLEAR effective.
Questions for authors
Please see the weaknesses.
3 comments
Response to Reviewer 3 (Part 2/2): Alternative Discovery Designs and Hierarchical Advantage
This comment continues our response to W1 from Part 1/2 and then addresses W2.
(3) Clarifying the motivation and comparing more direct alternatives
The phrase "may rely on unsupported assumptions" refers to questions that presuppose an unconfirmed event or attribute: asking about its details too early may depart from the user's actual need. For example, asking "How much compensation was paid?" before confirming that compensation occurred differs from neutrally asking "Was any compensation paid?"
We thank the reviewer for suggesting alternatives. The original manuscript included a prompt-based candidate-discovery control: an LLM proposes candidate facets from retrieved documents and dialogue, and the questioning policy selects a facet and generates a question. The revision adds three alternatives: cosine similarity of continuous representations, hierarchical K-means, and a domain facet hierarchy.
| Method | Core mechanism | R4 NDCG@10 | Mean NDCG@10 over R1–R4 |
|---|---|---|---|
| CLEAR (ours) | RQ document comparisons with the (1,1,2,2) schedule | 0.5344 | 0.4985 |
| LLM Prompt | An LLM discovers candidate facets from documents and dialogue | 0.4449 | 0.4452 |
| Continuous cosine similarity | Similarity of document representations before quantization | 0.4533 | 0.4157 |
| Hierarchical K-means | Cluster-tree paths replace RQ paths | 0.4308 | 0.3984 |
| Domain facet hierarchy | A manual hierarchy orders topics to specific attributes | 0.4263 | 0.3974 |
These experimental results support RQ's role in organizing document comparisons and discovering useful clarification facets.
These alternative-method comparisons have been added to Section 4.4, Table 2, and Figure 4(B), with an accompanying analysis of RQ's role in organizing document comparisons.
Response to W2: Motivation for hierarchical advantage, credit assignment, and component contributions
(1) Connection to "what to ask" and "gaining earlier"
Hierarchical advantage is a design choice that uses the action structure of our task. Specifically, "what to ask" involves priorities at both the category and facet levels: when several candidate facts are available, the model must determine which category of information to clarify and which specific facet within that category to select. These choices can affect convergence and retrieval quality in subsequent rounds. For example, among sentencing factors, the amount involved, the means used, and compensation are distinct facets that can be queried.
Ordinary group-relative advantage can learn these choices from sampled trajectory returns. Our design additionally aggregates returns for choices sharing a category or facet. Training therefore uses both individual trajectory performance and the average performance of related choices, helping the model learn reusable questioning priorities rather than judging a facet's value solely from one sampled outcome.
The connection to earlier gains comes from the stopping-weighted return shared across all levels:
w_k=P(S\geq k).$$ Obtaining the same retrieval gain earlier benefits more possible stopping rounds. Category- and facet-level comparisons use this return with the same auxiliary constraints, and thus learn selection priorities that support retrieval under a limited interaction budget. Their roles are complementary: **the return defines which behaviors are worth learning, while hierarchical comparisons organize the training signals provided by those returns.** We have added this motivation to the "Hierarchical comparisons" paragraph in Section 3.3, explaining how category and facet returns jointly contribute to training. Appendix B.3 provides the implementation details of the comparisons and standardization at each level. **(2) Additional experiments** Following the reviewer's suggestions, we conducted ablations of the hierarchical advantage components and added a Learned critic comparison: | Variant | Advantage configuration | R4 NDCG@10 | Mean NDCG@10 over R1–R4 | |---|---|---:|---:| | Full hierarchical | Trajectory + category + facet | 0.5344 | 0.4985 | | Flat | Group-relative advantage | 0.4592 | 0.4080 | | Learned critic | Learned state-value baseline | 0.4894 | 0.4423 | | w/o facet | Trajectory + category | 0.5084 | 0.4568 | | w/o category | Trajectory + facet | 0.4997 | 0.4561 | These comparisons have been added to the "Advantage controls" section of Table 3 in Section 4.4. The accompanying text analyzes Flat advantage, Learned critic, and the variants without category or facet comparisons to support the role of the hierarchical comparison components.Dear Reviewer zu4u,
We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.
Response to Reviewer 3 (Part 1/2): Progressive Prefix Mechanism and Patent Results
Response to W1: Mechanism, behavioral evidence, and alternative designs for progressive clarification
Thank you for your careful reading and valuable analysis.
(1) RQ organizes document comparisons; structural similarity is not equated with semantic granularity
When two document representations are close, the fact distinguishing them is not necessarily more fine-grained. We introduce shared RQ prefixes to construct a hierarchical document-comparison neighborhood within the current retrieval results. This focuses factual comparison on document pairs that share representation structure but differ in specific facts, allowing us to discover clarification facets that help distinguish candidate documents.
Increasing the shared RQ prefix length does not directly determine the specificity of generated questions. Its main role is to select document pairs for factual comparison: raising the threshold prioritizes pairs sharing more coding levels. Our design aims to identify distinguishing factual differences among documents with similar representation structure; their contribution to retrieval is examined in the controlled comparisons below.
We appreciate the reviewer's example of whether a threat was used. Such a local factual difference can indeed be decisive for the legal characterization of a case. The schedule (1,1,2,2) specifies the minimum shared-prefix requirement for preferred candidates at each round, rather than restricting each round to a particular code level:
(\tau_1,\tau_2,\tau_3,\tau_4)=(1,1,2,2),$$ where $p(d,d')$ is the shared-prefix length. If no candidate satisfies the current threshold, the implementation falls back to candidates that remain available. For example, a pair sharing a prefix of length 3 already satisfies threshold 1 at R1. Provided the other candidate conditions are met, its factual difference can enter the candidate set and is **not forcibly deferred because of its longer prefix**. Later rounds tighten the preferred eligibility of shorter-prefix pairs; they do not make longer-prefix pairs accessible for the first time. If the reviewer's scenario occurs at R1 or R2, our framework would regard the difference between those documents as particularly worth clarifying. The rationale is to retain a broader comparison neighborhood when little user information is available and retrieved documents may differ substantially, then favor unresolved differences between structurally closer cases as answers and retrieval results are updated. Our core objective is to investigate how questioning affects retrieval in subsequent rounds while keeping the retriever unchanged, and how selecting key facets can improve retrieval within a limited interaction budget. Progressive discovery and policy learning jointly serve this objective: discovery adapts the comparison neighborhood and supplies candidate facets, while the policy learns which facets to prioritize based on current and subsequent retrieval gains. Section 3.2 of the revised manuscript now explains the shared-prefix mechanism, minimum thresholds, and candidate fallback, including why longer-prefix pairs can enter early. Appendix B.1 gives candidate construction and ranking details, and Appendix A.7 provides a four-round interaction example. **(2) Progressive behavior, retrieval gains, and the patent configuration** Our objective is not to make questions increasingly specific, but to use shared-prefix lengths to progressively identify document pairs whose differences are most worth asking about. For patent retrieval, we have added an experiment enabling the $(1,1,2,2)$ schedule: | Patent configuration | R4 NDCG@10 | Mean NDCG@10 over R1–R4 | |---|---:|---:| | Schedule disabled | 0.2878 | 0.2757 | | Progressive schedule enabled | 0.3031 | 0.2852 | Final MAP and P@5 in this additional evaluation are 0.3410 and 0.3700, respectively. Enabling progressive scheduling improves both final and four-round mean NDCG@10 over the configuration without round-dependent scheduling. These experimental results support the schedule's role in this configuration. The patent results with progressive scheduling have been updated in Table 1. The legal comparison between progressive and fixed prefixes appears in Section 4.4, Figure 4(A). The new shared-state R3 branching experiment in Appendix A.8 (Table 11) further examines the effect of tightening the prefix threshold from the same retrieval state. The alternative-design comparisons and our response to W2 continue in Part 2/2.