- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper looks at interactive dialogue settings where LLMs must ask questions in order to seek information. The authors instantiate a Bayesian teacher model with explicit belief tracking and EIG question-selection and use it to distill a student model (Llama-3.1-8B). Experiments on 3 toy puzzle tasks suggest distillation closes most of the gap between student and teacher.
Critical strengths
- The general question-asking / information-seeking setting is an interesting research problem.
- As someone very familiar with this research area, the literature review appears reasonably complete.
- The ablations on teacher components are interesting.
Critical weaknesses
- The method is not methodologically novel relative to existing LLM + information-seeking work. All aspects of the Bayesian teacher (belief tracking, EIG question-selection) have been studied in prior work, while student distillation is also common both in information-seeking and more general LLM settings. The contributions are not clear, which is compounded by writing issues described below.
- The empirical evaluation is restricted to 3 toy tasks from a single benchmark and one LLM (Llama-3.1-8B-Instruct). Furthermore, the Multi-turn Puzzles benchmark (Badola et al., 2025) does not appear to have been accepted to any venue besides arxiv and to my knowledge has not been widely adopted. Given the relatively thin methodological contribution, I would expect a broader set of empirical contributions on more standard info-seeking benchmarks for comparability to the cited methods.
- Relatedly, while the related work demonstrates clear knowledge of other methods in the space, none of these are included as baselines.
- Overall, the paper is unnecessarily hard to follow; the authors appear to rely heavily on AI-generated prose and figures, which sacrifices human comprehensibility.
- Prose is overly dense and difficult to follow. Many passages bear hallmarks of AI-generated writing; e.g., here is a single sentence from the abstract:
“Exposing the belief state closes much of the gap where the decision is an implicit computation rather than a read-out of the interaction; in matched teacher contrasts, EIG supervision consistently outperforms random querying, whereas greater inference fidelity does not uniformly improve the distilled student; a 70B language-model teacher does not match the Bayesian teacher in downstream student performance, and the learned skill does not transfer to an unseen task family.” - Basic methodological details are not clearly described; e.g., “The teacher keeps a belief in a fixed variational family and updates it after every observation” (L114) — how is represented? how is the variational family chosen? - The paper contains only two figures, both of which are not very illustrative: - Fig 1: A tikz-style schematic. Many aspects of the figure appear AI-generated; e.g., flavor text. Overall, the figure does not clearly convey the pipeline. - Fig 2: Results bar graph. Figure lacks error bars and in the left-hand side the differences between bar heights is not apparent.
Questions for authors
- How did you choose the variational family -- it appears to be chosen specifically for each domain?
- What do you believe are the main contributions of this method relative to existing works?
- Are there any other figures you could include to illustrate details of the method or experimental details (e.g., scaling, ablations)? The current set of figures is very minimal, which contributes to the overly wordy nature of the manuscript.
- Would it be possible for you to rewrite the paper without using generative AI? This is a sincere question; I believe it would significantly improve comprehensibility.
6 comments
W2. Three toy tasks from one benchmark and one LLM (models)
We ran the paper's recipe on Qwen3-8B and Llama-3.2-3B-Instruct and trained a second seed of Llama-3.1-8B-Instruct, for the ELBO+EIG teacher in both formats (A = action-only, B = belief scratchpad). Only the backbone and its chat template changed. We used test subsets fixed before any run: every third Circuit Decoding episode (100), the first 200 Movie Recommendation episodes and the first 100 Word Guess episodes. All comparisons are paired on the same rows.
| Task, metric | Backbone | Base | A | B | B − A [95% CI] |
|---|---|---|---|---|---|
| Circuit, Success@18 | Llama-3.1-8B (paper) | 0.00 | 0.92 | 0.92 | +0.00 [−0.07, +0.07] |
| Qwen3-8B | 0.00 | 0.91 | 0.84 | −0.07 [−0.16, +0.02] | |
| Llama-3.2-3B | 0.00 | 0.81 | 0.91 | +0.10 [+0.01, +0.19] | |
| Movie, Success@10 | Llama-3.1-8B (paper) | 0.040 | 0.060 | 0.295 | +0.235 [+0.165, +0.305] |
| Qwen3-8B | 0.020 | 0.085 | 0.270 | +0.185 [+0.110, +0.255] | |
| Llama-3.2-3B | 0.065 | 0.045 | 0.155 | +0.110 [+0.055, +0.170] | |
| Word, mean attempts | Llama-3.1-8B (paper) | 32.9 | 19.0 | 3.1 | −15.9 [−19.1, −12.9] |
| Qwen3-8B | 32.5 | 15.7 | 3.3 | −12.5 [−15.8, −9.2] | |
| Llama-3.2-3B | 32.7 | 14.4 | 4.1 | −10.3 [−13.4, −7.3] |
The paper's findings hold on both new backbones. Distillation improves every backbone over its base model. Actions alone are enough on Circuit Decoding for the 8B models, and the scratchpad helps the weaker 3B student. On the other two tasks the scratchpad is what turns the gathered information into the decision.
Training seeds (Llama-3.1-8B, same data, same rows).
| Task, metric | Seed 0: A / B | Seed 1: A / B | B − A, seed 0 | B − A, seed 1 |
|---|---|---|---|---|
| Circuit, Success@18 | 0.92 / 0.92 | 0.94 / 0.96 | +0.00 [−0.07, +0.07] | +0.02 [−0.04, +0.08] |
| Movie, Success@10 | 0.060 / 0.295 | 0.065 / 0.290 | +0.235 [+0.165, +0.305] | +0.225 [+0.160, +0.290] |
| Word, mean attempts | 19.0 / 3.1 | 20.8 / 18.2 | −15.9 [−19.1, −12.9] | −2.5 [−7.6, +2.8] |
| Word, attempts, re-decoded | 19.0 / 3.1 | 20.8 / 8.3 | −15.9 [−19.1, −12.9] | −12.5 [−16.9, −7.7] |
| Word, Success@3 | 0.11 / 0.96 | 0.13 / 0.55 | +0.85 [+0.77, +0.92] | +0.42 [+0.30, +0.54] |
| Word, Success@3, re-decoded | 0.11 / 0.96 | 0.13 / 0.77 | +0.85 [+0.77, +0.92] | +0.64 [+0.52, +0.75] |
In the re-decoded rows the scratchpad student's invalid attempts are re-decoded under a trie of the vocabulary. The action-only student is left unchanged.
Circuit Decoding and Movie Recommendation agree to within 0.04 in every cell. On Word Guess the seed-1 scratchpad student fails on one of the three vocabularies in these rows. Its first guess there is not a vocabulary word, a state absent from training, and 38 of the 40 episodes run to the budget. Re-decoding recovers 27 of them and brings the format contrast to −12.5 attempts, within the range of the new backbones. The other 11 fail by repeating one legal guess, the dead end described in the paper. The unconstrained rows remain the reported result, since the paper's protocol does not repair outputs (Section 4).
W2. Three toy tasks from one benchmark and one LLM (tasks)
Choice of benchmark. We chose the benchmark for one property, not for its adoption. Our claims need a teacher that can be checked against a Bayesian reference, which requires a task whose latent state, query space and observation model are fully specified and whose respondent is a deterministic program. The three tasks are the benchmark's tasks with this property. The references let us validate the teacher (Table 6), score every question by its reference EIG (Appendix E.3), apply an oracle read-out (Figure 2a) and attribute student performance to single teacher components (Table 4). The other two tasks lack the property: in Twenty Questions the model under test is the answerer, and Word Chaining has no hidden state.
Standard benchmarks. In the 20-Questions and clarification settings of the cited methods the respondent is typically a language model or a human, so the likelihood is not specified and a Bayesian teacher could not be validated. We therefore have no results there, and we accept that this limits direct comparison. The comparison in the paper is to the benchmark's published frontier-model numbers and is indicative only (Table 2, Appendix B.5).
"Toy". The tasks are small because smallness is what makes exact references possible, but they are not easy. The strongest reported model scores 39.8 on Circuit Decoding and 84.3 on Movie Recommendation, where 50 is a random pick, and the 8B base model scores 0.7 and 50.4. The benchmark is also used by later work: Zou et al. (ICLR 2026) train on simplified variants of its tasks, and on their circuit variant Gemini 2.5 Pro reaches 92% against 40% on the released task (Appendix H). The three tasks span continuous, structured-binary and finite-symbolic latent states, and neither the policy nor the information-gain criterion transfers between them (Appendix G), so they are not three instances of one puzzle.
W3. Baselines from related work
We do not include re-implementations of the cited systems, and we cannot add them within the discussion period. Our controlled arms represent the families of methods in the related work, with episodes, student, recipe and evaluator held fixed. The paper does not label them this way, and it should.
| Family in the related work | Corresponding arm | Result |
|---|---|---|
| Prompted language models without training | Base model and the benchmark's reported models | Strongest reported model 39.8 and 84.3, student 97.4 and 90.0 on Circuit Decoding and Movie Recommendation (Table 2, indicative) |
| Supervision generated by a language model | 70B LLM teacher with action-only students | −0.92 Success@18 and −0.03 Success@10 against the Bayesian teacher's action-only students (Table 4) |
| Imitating a Bayesian model's outputs without its belief (Qiu et al., 2025) | Action-only students of the Bayesian teacher | 0.085 vs. 0.345 Success@10 and 19.2 vs. 3.4 attempts once the belief is written (Table 3) |
| Other query rules under the same belief | ELBO+Random and ELBO+Uncertainty teachers | EIG is better by +0.42 / +0.17 / +0.21 over random and by +0.15 over uncertainty sampling (Table 4) |
| Bayesian loop at inference time (BED-LLM, UoT) | The Bayesian teachers themselves | Teacher 1.00 / 0.41 / 2.29, student 0.93 / 0.35 / 3.37 (Table 2) |
These arms are analogues under our recipe, not the published systems, and outcome-based filtering as in STaR-GATE is not covered. For the inference-time family the teacher row is the idealized case: it runs the posterior update and EIG at every turn with the exact task model, and the student reaches most of that level with one forward pass per turn.
We chose arms over systems because the cited systems differ in backbone, prompts, training data and task at once, so a system-level table would not show which design choice matters. The closest reward-based methods cannot be set against our numbers: Hartmann et al. (2026) and Deng et al. (2026) are concurrent preprints, and Zou et al. (2026) use simplified variants of these tasks (Appendix H).
We thank the reviewer for the detailed review. It raises four weaknesses and four questions, and we answer them as follows.
W1 / Q2. Novelty and main contributions
We agree that variational belief updating, EIG query selection and distillation are each established, and we claim none of them as new. The contribution is what their combination lets us measure. We make three claims.
1. What has to be distilled, and when. Cloning the teacher's actions transfers its query policy but not the computation that turns the interaction into a decision. When the decision can be read off the history (Circuit Decoding), action-only distillation is enough for a strong student: 0.920 vs. 0.927 Success@18 with the scratchpad (Table 3). When it must be computed from a belief (Movie Recommendation, Word Guess), action-only students gather the information and fail to use it: 0.085 vs. 0.345 Success@10 and 19.2 vs. 3.4 attempts. Figure 2a locates the failure: the teacher's questions leave the action-only movie student at 0.085, while a Bayes-optimal read-out of its own transcript lifts it to 0.305. The pattern holds on two further backbones and a second seed (W2).
2. Which part of the teacher matters. The teacher's components can be swapped with everything else held fixed (Table 4). Replacing random querying by EIG changes the student by +0.42 / +0.17 / +0.21 (circuit / movie / word). Replacing variational by exact or particle inference changes it by −0.28 / +0.06 / 0.00. A new experiment shows that most of the −0.28 comes from random tie-breaking in the exact teacher and not from inference fidelity: with deterministic ties the difference is −0.05 [−0.09, +0.00] (our reply to Reviewer ve4v). A 70B language-model teacher yields students with almost no success on Circuit Decoding (−0.92). These attributions require a teacher whose components are explicit.
3. A verifiable teacher as the source of supervision. Training-based methods (Andukuri et al. 2024, Zhang et al. 2025, Chen et al. 2025, Mazzaccara et al. 2024) take their targets from the model itself or from heuristics. Inference-time methods (BED-LLM, UoT) keep the Bayesian loop at test time. Our targets come from an agent validated against exact and particle references (Table 6) and are amortized into the weights.
Closest prior work. Qiu et al. (2025) fine-tune a language model on a Bayesian assistant's recommendations. Their appendix shows the same read-out gap from the other side: a model trained on the assistant's preferences supports 79% accuracy under an external read-out but recommends at 55%. We differ in three ways: our student also learns which questions to ask, the belief target is a posterior summary with per-candidate scores rather than a point estimate, and we identify when the belief has to be written.