- Rating
- 3(+1)
- Confidence
- 2(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether teaching LLMs to estimate answer uncertainty and source reliability helps them handle conflicting evidence. It introduces BayesClue, a synthetic environment with reliable, noisy, and adversarial sources. Exact Posterior Distillation trains models to match exact posterior probabilities over answers and source types after each report. The paper reports improved belief prediction in this environment and gains on some external tasks involving conflicting documents and memories.
Critical strengths
S1. Exact posterior targets support controlled training and evaluation. BayesClue computes answer and source-type probabilities after each report under a specified generative model. These targets support supervision of both answer uncertainty and source reliability, and allow evaluation of how predictions change as evidence accumulates.
S2. The experiments show benefits across models and test conditions. The study covers multiple model families, longer evidence histories, larger candidate sets, and shifts in source proportions. Gains on some external benchmarks, without training on their examples, suggest some transfer beyond BayesClue.
Critical weaknesses
W1. The adversarial setting tests only a limited form of adversarial behavior. Adversarial sources have a fixed 2% truth rate and uniformly distributed errors. Once their type is inferred, their reports become useful counterevidence. The synthetic tests do not cover selective deception, reliability changes within an episode, or sources copying one another. Robustness to these forms of conflict remains unclear.
W2. The source categories and reporting accuracies need further validation. The paper varies source-type proportions but keeps reporting accuracies fixed at 95%, 65%, and 2%. It does not report sensitivity analyses for these accuracies or the number of source types. A comparison with a continuous distribution over each source's reporting accuracy could also help justify restricting reliability to three categories.
W3. Source reliability judgment is not directly evaluated on external tasks. Lower source-posterior KL supports learning within BayesClue. External evaluations mainly measure answer quality or calibration and do not isolate the role of source reliability judgment. Tests that change a source's history while keeping its current report fixed would help assess this mechanism.
W4. The effects of source order and report allocation are not isolated. Sources follow a fixed reporting order and contribute unequal numbers of reports in the default setting. The paper does not test randomized source order or balanced report counts. It remains unclear how these choices affect source reliability judgments.
Minor Comments: Although AI use is disclosed, I recommend redrawing the two main figures manually. In Figure 1, text becomes blurry when enlarged, and the background colors behind some labels differ from the surrounding areas. These issues affect readability and visual consistency.
Questions for authors
Q1. How to explain the CONFACT regressions on Qwen3.5? Table 2 reports EPD drops of 18.3 and 10.4 percentage points in macro-F1 for Qwen3.5-4B and 9B. The paper acknowledges variation across models but does not explain these regressions. Please analyze their causes and present base and post-training scores together, with variation across runs?
Q2. What is the added value of full posterior supervision? The paper acknowledges no clear average KL advantage over SAMPLE in Figure 5(b), while Figure 5(c) shows generation gains for some models. Could the authors report confidence intervals for these gains and clarify where full posterior targets offer an advantage over sampled labels? Besides, Figure 5 has overlapping elements. The y-axis title of panel (b) overlaps with data points in panel (a). Please increase the spacing between panels to improve readability.
Q3. Does source supervision still help when the supervision budget is matched? In Table 14, World-only removes source supervision and reduces the number of supervised targets. Does EPD still outperform World-only when both receive the same number of supervised targets?
5 comments
Author Response Part 2
Q3. Matched world-only control.
We conduct the requested experiment and find that with matched supervision, EPD’s source-belief error is 1.2 to 4x lower than world-only training, so source targets teach a model to estimate which sources to trust. The two regimes are comparable on answer beliefs: world-only mostly matches EPD. This is expected, because each world target is already marginalized over source types, so fitting it implicitly rewards accounting for reliability. Source supervision makes that judgment explicit and measurable.
We have revised Appendix C and Table 14 with this text.
External transfer, EPD minus the matched world-only results are below:
| Model | CASEBench Brier ↓ | CONFACT AUROC ↑ |
|---|---|---|
| OLMo-2-1B | −0.18 [−0.25, −0.12] | +.035 |
|
| OLMo-2-7B | −0.18 [−0.34, −0.03] | +.018
|
| Qwen3.5-9B | −0.18 [−0.24, −0.12] | +.001
|
EPD is better on most models. On RAMDocs, EPD leads the matched arm on Qwen3.5-9B by +.110
in discrimination, and by +.082
with every item scored in both answer orders, so the lead is not a position effect.
Figures.
We have regenerated the figures as vector graphics with high resolution, corrected spacing and consistent label backgrounds.
We hope our responses were able to address the concerns of the reviewer and look forward to further discussion and a reassessment of the scores for our work.
References
- Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017). Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. IROS.
Response to Authors' Replies
Thank the authors for the revisions and supplementary experiments, which strengthen the robustness checks and clarify the benefits of exact targets. According to your replies, I still have two questions:
1. Matched world-only control. Does EPD retain its MemBench advantage when the world-only control receives matched supervision and comparable training? Without this result, it remains unclear whether source supervision explains the memory-arbitration gain claimed in Table 14.
2. Source-history test. EPD performs below the base model on the stated source-history test, despite outperforming other belief-training arms. Does this result limit the claim that EPD improves transferable source-reliability judgments, particularly when trust depends on a source’s past performance?
I would also suggest qualifying two claims:
1. Selective deception: Keep robustness claims within the scope of the tested adversarial setting.
2. Benefits of exact targets: Acknowledge that source-belief accuracy is slightly lower than with sampled targets, and generation performance, although better than sampled targets, remains below the base model.
Author Response Part 1
We thank the reviewer for their feedback. Below, we address each of their concerns:
We added the harder adversary you suggest, that lies selectively. The adversary lies about two of the four features and reports the others truthfully. This breaks an assumption of our training setup, so even the exact Bayesian posterior computed under the training assumptions becomes less accurate (+.145 nats). The trained models lose a similar amount (+.07 to +.22 nats), and their predictions stay within .04 nats of that Bayesian posterior. The drop therefore comes from the deceiver violating assumptions that affect any Bayesian reader, not from the trained models breaking down. Sources whose reliability changes mid-episode or who copy each other remain future work.
We added every variation you listed: reliable-source accuracy 0.90 and 0.85, noisy-source accuracy 0.75, adversary truth rate 0.10, and two source types instead of three. The ranking of methods holds in most comparisons. With two types, the rise in source-belief error (.16–.42 nats) tracks the rise in the best achievable error (.23–.33). Our design follows domain randomization (Tobin et al., 2017): the main arm varies the source mix per episode instead of fixing one realistic setting.
We ran the suggested test: we changed a source’s stated track record while keeping its current report fixed. EPD learns source reliability from whether reports agree with each other. This matches how BayesClue is built: reliability is never stated, and source types are resampled every episode, so a source’s history carries no information during training. Among all belief-training arms, EPD retains the use of stated track records best, outperforming sampled targets by .14 to .54.
Report order has a small effect, and training on shuffled reports removes it. Permuting reports leaves the exact posterior unchanged; under permutation, EPD’s answer-belief error rises by at most .037 nats (Qwen3.5-4B: .018 [.011, .027]), and with random sources and unbalanced counts by at most .051. On Granite-2B, training with shuffled report order removes the effect (+.046
+.024,+.069→ +.002
−.011,+.014) with no loss in-distribution.
The drop is a shift toward "yes", with discrimination unchanged (AUROC −.026
−.055,+.002). Details are in our reply to 3NZ9, W2.
Exact targets give more accurate answer beliefs, better generation and better revision; sampled targets are slightly better on source beliefs. Split by query type across all runs:
Mean ± standard deviation over three training runs.
Exact targets revise answers more reliably beyond the training horizon: direction accuracy is 90.2% vs 78.8%, and new-leader selection is 64.3% vs 53.8% (Table 16). Exact targets score higher than sampled targets in all models on MMLU, ARC-Challenge and HellaSwag. We have updated Section 5.4 and Appendix M with the new results.
We have also increased the panel spacing in Figure 5.