- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses uncertainty decomposition for black-box LLMs within the input clarification ensembling (ICE) framework. The authors argue that existing clarification generators, whether prompted or supervised fine-tuned, ignore the target LLM, so their clarifications can introduce additional epistemic uncertainty that contaminates the decomposition. They propose TACO, which casts clarification generation as a reinforcement learning problem and optimizes the generator with PPO using feedback from the frozen target model. Two rewards guide training: an uncertainty decomposition reward, defined as the batch-level AUROC of ICE data-uncertainty scores, which improves the separation between ambiguous and clear inputs; and an entropy alignment reward, which encourages ambiguity-aware diversity for ambiguous inputs while preserving semantic consistency for clear ones. TACO is evaluated on ambiguity detection (AmbigInst, AmbigQA) and correctness prediction (GSM8K), and the authors report consistent AUROC and Best-F1 gains over baselines including ICE and Spectral Uncertainty. Ablations indicate that both rewards contribute, with the uncertainty decomposition reward driving most of the improvement. Further analyses suggest that TACO lowers clarification-induced model uncertainty while increasing data uncertainty for ambiguous inputs, and that it adapts clarification diversity to input ambiguity.
Critical strengths
-
S1: Addressing a clear gap: ICE-based uncertainty decomposition methods assume that clarifications remove input ambiguity without adding uncertainty of their own. In fact, previous approaches (Hou et al. , Walha et al.) rely heavily on using a frontier model to generate high quality clarifications, therefore side-stepping the critical clarification quality bottleneck at the cost of expensive generation. This is a real gap that is addressed by the paper, where they use RL fine-tuning to get substantial improvements on relatively small clarification models.
-
S2: Consistent substantial gains: TACO improves over all baselines on both ambiguity detection and correctness prediction. On AmbigInst with LLaMA-3-8B, it reaches 90.49 AUROC versus 80.33 for the strongest baseline (Table 1). On GSM8K with Qwen-2.5-7B, it reaches 94.13 versus 89.04. The gains hold across two target LLMs and two generator backbones (Tables 1–2), and they come with lower model uncertainty under clarifications (Table 3).
Critical weaknesses
- W1: Weakly justified claim of target alignment:
- The proposed approach clearly improves the overall quality of the generated clarifications, making them more diverse for ambiguous questions and less diverse for unambiguous ones. However, the paper repeatedly claims that the clarifications are aligned to a specific target model, without sufficient justification.
- Figure 4 does show that the same ground-truth set of clarifications can yield different performance across target models. But this may simply reflect the models’ differing knowledge of the question’s overall domain, i.e., epistemic uncertainty with respect to the original question. It need not reflect how this particular set of clarifications interacts or aligns with each model. Uncertainty of this kind (high epistemic uncertainty to the original question) cannot be reduced by clarifications better adapted to the target model, nor should it be: reducing it would corrupt the epistemic uncertainty signal, which in such cases should remain high.
- To establish the target-alignment claim convincingly, the authors should evaluate how LoRA adapters trained for one target model (e.g., Qwen-2.5-7B) perform when applied to another target model (e.g., LLaMA-3-8B), and vice versa. This would help isolate whether the improvement stems solely from generally better clarifications, or from both better clarifications and genuine alignment to the target model. A complementary case study comparing the clarifications produced by TACO when fine-tuned on different target models would also be informative: do different fine-tuning targets yield substantially different clarification sets?
Questions for authors
- Q1: Value function: How is the value function implemented? Which backbone does it use, and which loss is used to train the critic? These architectural details are omitted from the manuscript and should be included. Moreover, is computed per instance, whereas is computed per batch. For this component of the reward, may increase variance instead of reducing it.
- Q2: Cross-dataset generalization: The method is evaluated on several datasets, but it is not tested whether a clarification LLM fine-tuned on one dataset generalizes to another. This is an important practical question: generalization across domains would remove the need to fine-tune on each target dataset before inference. Please include cross-dataset experiments to address this point.
- Q3: Ambiguity labels on GSM8K: The rewards defined in Section 3.3 require ground-truth ambiguity labels for each question. However, such labels are not available for GSM8K. How was this handled? Are all GSM8K questions treated as unambiguous?
- Q4: Train/test split on AmbigInst: AmbigInst reuses a limited set of instruction templates across many inputs, so the random train/test split used in the experiments leaks templates into the training set. A split by instruction template would give a fairer assessment of the method’s impact on performance.
- Q5: Baseline consistency: Some baseline numbers appear inconsistent. Comparing Tables 1 and 2 for GLM-4-9B and LLaMA-3-8B:
- ICE on AmbigInst in Table 1 (79.54 / 76.78) is identical to Zero-Shot in Table 2.
- ICE on AmbigQA in Table 1 (67.10 / 71.73) is identical to SFT in Table 2. Please clarify this clarify this discrepancy and report the correct values.
- Q6: Typo (minor): There is a typo in Equation (4), the objective formulation.